MEDICAL EDUCATION

Pdf version

Notes for Primary Care Teachers

ASSESSMENTS – THE BASICS

Table 1. Summary of commonly used written assessment instruments, their objectives, advantages and disadvantages

Assessment instruments	Objectives	Advantages	Disadvantages
Essay Questions a. extended long essay b. short essay	Tests students thought process &  understanding of subject	Easy to set, allows exploration of students’ attitude	Time consuming to mark, Difficult in scoring Problem of reliability, less commonly used now in medical schools.
Objective tests A. Supply – item types i. short answer ii. completion item	Tests student’s ability to recall terms, facts, principles & procedures	Easy to develop. Prevents guessing	Not ideal for measuring complex learning outcome, e.g. problem solving.
B. Multiple Choice Questions (MCQs) i. True/False items	Tests knowledge, factual recall	Easy to mark	Susceptibility to guessing, (reduced through negative marking), tends to assess trivial knowledge, difficult to construct flawlessly, needs training to avoid pitfalls in construction.
ii. Single best items iii. Extended matching items	Tests applied knowledge if constructed well	High reliability, Easy to mark, opportunity to guessing less than True/False items	Measures what students know and understand but do not measure performance, needs training to write good MCQs.
Modified Essay Question (MEQ) & Patient Management Problems (PMP)	Problem solving ability	Uses actual problems and events in clinical practice	Requires some skill in developing
Key Feature problems	Problem solving ability & clinical decision making skills	Good reliability	New, construction is  time consuming, best used in high stakes examinations

Clinical assessments

The observed structured clinical examination (OSCE) is now widely used because it is a reliable approach in assessing basic clinical skills. It is based on a circuit of patient based “stations”. It can vary from 8 to 20 stations, with 5 to 15-30 minutes per station. Both real and simulated patients are used.  The scoring uses rating scales or checklists, by observers and “patients”. The disadvantage of an OSCE is that only isolated aspects of a clinical encounter can be assessed. Some OSCEs include data or visual interpretation stations on clinical problems.

Alternative assessments include the traditional long consultation which allows the observers to be present throughout the doctor-patient interaction. The main disadvantage is poor reliability and it is time consuming, especially when the student numbers are large. One way of overcoming this is to incorporate a number of long consultations into the course that the students must pass before they can proceed to the end-of-course examinations. 

For short cases with real patients (usually a series of 5-6 patients), one or two examiners observe a students performance, ask questions and made a judgment based on student’s performance. Such short cases can be incorporated into OSCE stations.

The oral examination allows exploration of students’ understanding of topics considered relevant to clinical practice.

Individual assessment instruments test different aspects such as knowledge or skills. An examination usually uses a combination of different types of assessments including written and clinical assessments, to test the competencies acquired by students.  An example of this will be shown in the article “Assessments – the Conjoint MAFP/FRACGP examinations” (page 84).

Timing of assessments
For short courses some have a pretest assessment (before the teaching begin) followed by a post test assessment to compare how much was learned from the course. A pretest can also help in determine how much prior knowledge the students have, so that the teacher can plan the lessons to build on it. Usually a combination of formative, continuous and summative assessments is used.

Attitude
This is difficult to assess. To some extent it can be assessed in essay type questions, oral examination and in the communication skills stations of the OSCE. The supervisor’s or GP tutor’s report as part of the continuous assessment may also reveal some attitude problems. Portfolio assessment is another means of assessing attitudes that will be discussed under a separate article.

Involvement in assessments
You may be asked to do one or more of the following:

• Write a report on the student attached to your clinic (usually a standardised checklist is provided)
• Assess his/her performance in your teaching session including case presentation.
• Mark student assignments or projects such as audit or research projects. Usually checklists or guidelines are provided.
• Vet and/or mark written examination questions.
• Participate as an invigilator or examiner in clinical assessments such as OSCE.
• Contribute questions to the existing bank of examination questions. If you are asked to help construct examination questions, ensure the questions prepared by you meet your course objectives. It should cover core knowledge and skills taught by you. You can ask a more senior colleague to guide you.

Conclusion

If you have recently become a primary care teacher, do familiarise yourselves with the different assessment instruments used in the course you are teaching. Reading materials on assessment instruments are also available.1-4 Be ready to participate in any assessment workshop and teaching the teachers courses made available.

Further readings

1. Newble D, Cannon R. A Handbook for Clinical Teachers. 4th edition. MTP Press, 2001
2. Schuwirth LWT, Vleuten CPM. ABC of learning and teaching in medicine: written assessment. BMJ. 2003;326:643-5 [PubMed] [Full text]
3. Smee S. Skill based assessment. In ABC of learning and teaching in medicine. BMJ. 2003;326:703-6 [PubMed] [Full text]
4. Case SM, Swanson DB. Constructing Written Tests Questions for Basic and Clinical Sciences, 3rd Edition. Philadelphia, USA: National Board of Medical Examiners; 2002. [Full text]