Title: Quizzing with Flashcards versus Rereading a Handout: A Comparison of Active versus Less Active Study Methods Used for Learning a Novel Anatomy Topic
Authors: Angie K. Maxson, OMS IV; Alan T. Boyd, OMS IV; M. David Gothard, M.S.; Diana C.J. Rhodes, DVM, PhD
Introduction
The use of quizzing as an active study method to enhance memory retrieval may strengthen long-term retention, a process referred to as the “testing effect.” Does quizzing with premade flashcards versus a less active method of rereading an informational handout better prepare Pacific Northwest University (PNWU) students for knowledge tests on newly acquired, novel anatomy material? Additionally, what restudy techniques do PNWU students currently utilize?
Methods
Master of Arts in Medical Science and first year Doctor of Osteopathic Medicine students recruited from PNWU studied an informational handout on ruminant digestive anatomy. Participants then were assigned to a “flashcard group” (FG) or a “handout group” (HG), reviewing ruminant anatomy via their assigned method. Participant demographics and study behavior/preferences were collected before a 15-question test on ruminant digestive anatomy was completed on day 1 of this study, versions of which were administered 1 and 3 weeks later. Data were analyzed with repeated measures ANOVA using SPSSv25.0 software with p<0.05 considered statistically significant. This study was exempt by PNWU IRB (#2021-015).
Results
For each of the 3 ruminant digestive anatomy tests, mean scores were higher in the FG than in the HG. Though these results were not statistically significant, a large effect size (ηp2=0.16) was detected for day 1 test scores (FG 10.67 ± 1.50 vs HG 9.00 ± 2.68). On day 1, participants (n=17) reported normally engaging in the following study methods using the scale of 0=rarely/never; 1=sometimes; 2=often/always with resulting means noted: flashcards/quizzing/practice questions (1.76), note taking (1.76), outlining or concept mapping (1.53), highlighting/underlining lecture materials (1.53), rereading silently (1.06), rereading aloud (0.59), and rewatching recorded lectures (0.24). When asked specifically about the source of questions used for study, questions formed by course instructors were most often reviewed (1.88), followed by self-made questions (1.18), questions created by peers (1.00), and those found on websites or in textbooks (0.71).
Discussion
Traditionally, the testing effect has been demonstrated using simple word pairs. The current investigation evaluated this learning strategy within the more complex subject of medical anatomy and found that on tests, participants using flashcards tended to outperform those that reread a handout. Additionally, it was demonstrated that PNWU students already appear to engage in more active study methods like quizzing. As this study was limited by a small sample size, replication of this research involving a larger sample of professional students may help to better define this phenomenon in medical education.
A question and a comment from a judge: How many participants did your power calculation require for this study? It seems to me to be woefully underpowered, especially since you chose parallel groups, instead of a cross-over design. The comment is: Your long-term results are consistent with other observations that suggest that PNWU students have a retention span of just over 2 weeks. If I read your graphs right, more than 50% of the knowledge present on day 1 was gone 3 weeks later, regardless of whether the students used flashcards or not.
Thank you for your question and comments, Dr. Elliott. To answer your question: An estimated 60 participants were needed to obtain a power of 0.80 (slide 9). Independent of sample size, and for Day 1 Mean Test Scores, a partial etta squared of 0.16 indicates a large effect size, suggesting that there is a difference between groups. We believe that replication of this study with a larger sample size (of approximately 60 participants) will help to demonstrate the presence of the testing effect within the more complex subject of anatomy. To comment on your comment: We asked participants not to restudy ruminant digestive anatomy after Day 1 to control for confounding variables. With repetition of this study, it would be interesting to incorporate restudy sessions between testing periods and to explore the spacing effect. Again, thank you for your time and attention.
As a judge I have the following question. The presenters note while there is existing literature on testing effect, they suggest less has been done on this in relation to ‘complex’ content. Please describe what accounts for complex content, especially as as significant component of anatomical learning is essentially the memorization of structures and functions?
Thank you for your question, Dr. Fritz. Yest, traditionally, word pairs have been used when evaluating the testing effect. For example, Jonsson et al. (2020), provided participants with Swahili – Swedish word pairs, such as “mashua – boat.” Participants either restudied these words when paired together and presented on their screen, or they were provided with the Swahili word and were prompted to type the corresponding Swedish word. Memorization of these word pairs does not require several levels of processing and thus were deemed “simple.” When learning anatomy at the professional level, integration of knowledge is essential for understanding the relationship between structure and function, or lack thereof, and producing a correct response. This task requires several levels of processing, and therefore, the subject is considered more “complex”. For this study, an informational handout on ruminant digestive anatomy was designed at a similar level of difficulty in which material is presented to health science students at PNWU. For example, the handout explains how a ruminant’s “esophagus, unlike in most animals, contains voluntary, skeletal muscle throughout its entire length which allows it to function bidirectionally in ruminants, facilitating cattle in regurgitating their cud back to their mouth for further chewing in rumination.” Participants might not be asked to identify the esophagus, rather, they would be expected to answer questions regarding the relationship between a structure and its function. Thanks again for your question.
Thank you for your presentation. As a judge, I am wondering about the techniques for restudy that students reported versus the techniques you studied. I do not see flashcards or faculty-prepared handouts as restudy options. Was this intentional left off or do students report not using these techniques?
Thank you for your question, Dr. Fritz. Yest, traditionally, word pairs have been used when evaluating the testing effect. For example, Jonsson et al. (2020), provided participants with Swahili – Swedish word pairs, such as “mashua – boat.” Participants either restudied these words when paired together and presented on their screen, or they were provided with the Swahili word and were prompted to type the corresponding Swedish word. Memorization of these word pairs does not require several levels of processing and thus were deemed “simple.” When learning anatomy at the professional level, integration of knowledge is essential for understanding the relationship between structure and function, or lack thereof, and producing a correct response. This task requires several levels of processing, and therefore, the subject is considered more “complex”. For this study, an informational handout on ruminant digestive anatomy was designed at a similar level of difficulty in which material is presented to health science students at PNWU. For example, the handout explains how a ruminant’s “esophagus, unlike in most animals, contains voluntary, skeletal muscle throughout its entire length which allows it to function bidirectionally in ruminants, facilitating cattle in regurgitating their cud back to their mouth for further chewing in rumination.” Participants might not be asked to identify the esophagus, rather, they would be expected to answer questions regarding the relationship between a structure and its function. Thanks again for your question.
Dr. Salido, Please see my response to your question below. Thank you.
Thank you for your question, Dr. Salido. Participants were surveyed on their preferred method of review for basic science lecture materials for the current academic year, including “quizzing” with flashcards containing practice question and “rereading” lecture materials, which would including faculty produced handouts. As you can see from the left graph on slide 7 of the presentation, “quizzing” received the highest mean rating, along with note taking. “Rereading silently” and “rereading aloud” received some of the lowest mean ratings. Again, thank you for your clarifying question.
Wow, nice posts guys