Post-hoc analysis of questions asked in computer based tutorials or in audience response system enabled PowerPoint lectures: Testing while Teaching

Abstract

We have carried out the analysis of questions asked in interactive computer-assisted tutorials and audience response system enabled PowerPoint lectures.  This type of post-hoc analysis is not commonly carried out.  However, it can sensitively assess participant knowledge that is critical for optimizing medical education.  Our experience with post-hoc analysis in four studies is presented in this article: 1) A tutorial on electrocardiograms; 2) A tutorial on hematological disorders; 3) Comparison of lectures on hematological emergencies and blood cell morphology; 4) Evaluation of lectures given in an eight month long course on cardiovascular physiology and pathology. In these studies, knowledge was assessed, demographics of knowledge between residents at different stages of training and medical students were determined, and poor students were identified and remediation was recommended.  Although these studies were very different, they strongly demonstrated the educational value of post-hoc analysis of questions asked in tutorials and lectures.  How this information can be used to improve medical education and training programs is demonstrated in this article.

Formal testing and stand-alone exams can provide the same type of information.  The downside of stand-alone exams is that it is difficult to use them frequently.  The reasons are the time needed to prepare and administer the tests and the intimidation of students and trainees by continuous testing.  The inability to test students and trainees frequently hinders the identification of poor students and delays the implementation of improvements in medical education.  Post-hoc analysis is a reasonable alternative to formal testing.  Computer-assisted or on-line tutorials and audience response system enabled PowerPoint lectures can be created to contain an ample number of questions.  There is no need to write additional questions.  Answers are easily stored and available for analysis. Post-hoc analysis of knowledge can be carried out as often as needed.  Since questions asked in these tutorials and lectures introduce topics considered to be important, the questions usually comprehensively cover core and essential subjects.