- The problem
-
The Medical Science disciplines recently underwent a review in response to the call to reduce assessment effort by ~20%. In 2016-17 all Level 2 courses were single best answer or extended matching questions that can be mechanically graded, except for two courses (BI20M3 - Molecular Biology of the Gene; BI25M7 - Energy for Life) that utilised extended answer essay-style exams. During the assessment reduction review, these two Level 2 courses were identified as ideal courses to save on assessment effort and also increase robustness in our assessment practices as multiple choice exam styles would allow the testing of all areas of each course equally.
- The solution
-
For both courses the essay-style exams allowed a degree of choice with students answering 4 questions out of 10 in each exam, thus allowing students an amount of question spotting and opportunity to only demonstrate competence in less than half of the taught material. For each course a different approach was used as highlighted below, but each course moved from 4 essay scripts per student (approx. 800 scripts per exam with class sizes averaging 200 students) to 100 multiple choice questions, all mechanically graded.
BI20M3 (Molecular Biology of the Gene) - this course adopted a single end of term exam delivered in a main exam venue during the December exam diet. The exam had 100 questions, all single best answer questions with 5 possible answers each. Based on the course having 30 lectures this gave 3‑4 questions per lecture and thus allowed complete course coverage, consequently increasing the scope of assessment to include all areas of the course.
BI25M7 (Energy for Life) - this course adopted four individual in-term exams that followed each of four modules taught. Each exam consisted of 25 single best answer questions with 5 possible answers each. Therefore over the entirety of the course this gave an equivalent number of questions to the BI20M3 course above, with 100 questions over the whole series of exam assessments. Again, this number of questions allowed for complete coverage of the taught material and also had the added benefit of testing knowledge and competence throughout the course rather than all knowledge being tested at the end of the course.
- Evaluation
-
Exam grades for the above courses were analysed from academic years 2015-16, 2016-17 and 2017‑18 to ascertain any discernible differences in student attainment following the change in exam formats. All lecture material and other assessments in both courses has remained unchanged over recent years. Following this analysis it was clear that changing to a single best answer exam format did not negatively impact on student performance, and has increased exam attainment in both courses (see Fig.1 below), but especially in BI25M7; a course that has traditionally been seen as a very challenging course due to its high level of biochemistry content.
From this preliminary analysis, in both courses there was a definite positive shift in attainment with no to very few students getting grade points of 22 (A1) and 21 (A2), but both courses show a marked increase in the number of students achieving these top grades, with less students attaining mid-range grades, suggesting that perhaps the mid-level students are more able to demonstrate their capabilities through the new exam format. Those students with fail grades (grade points less than 9) seem to change little between years, so at least anecdotally the lower end of student attainment does not appear to be affected as much as the top end grades.
Figure 1 - Analysis of exam grades from BI20M3 and BI25M7 courses
Grades from academic years 2015-2018 were analysed to show the number of students (frequency) attaining each grade point. For BI20M3 grades are directly from the end of term exam, and for BI25M7 grades were averaged from the four in-term exams.
Student feedback has also been overwhelmingly positive in light of these changes, especially with the four “mini-exams” that have been employed in the BI25M7 course, with SCEF feedback stating this format of exam should be used in other courses because it allows students to know how well they are attaining in the course as it progresses. The four exam format also forces students to engage with the course content throughout term time rather than cramming for exams during the final few weeks of term, which was seen as a positive by most SCEF respondents. On a negative note, this style of exam does not allow students the ability to show synthesis of their ideas and original thought as it instead is testing recall on a variety of topics. That said, at Level 2 we would expect students to largely still be learning factual information as a foundation to higher level learning in Level 3 and beyond.
- Impact
-
Overall, these changes have seen over 400 hours of effort saved between the two courses, but have also shown a marked improvement in attainment whilst assessing all areas of each course equally to demonstrate overall competence of the students in each course.