Title: Assessing Knowledge and Confidence in Health Sciences Students in Partner and Individual Restudy Groups Learning Anatomy
Authors: Alan T. Boyd, OMS4(1); Angie K. Maxson, OMS4(1); M. David Gothard, MS(2); Diana C. J. Rhodes, DVM, PhD(1) (1)Pacific Northwest University of Health Sciences, (2)Biostats Inc.
Introduction
Research indicates quiz-based learning may be more effective than passive restudying, a phenomenon known as the “testing effect.” In medicine, appropriate self-assessment of one’s own knowledge is an important quality to limit medical error; thus, it is prudent for health sciences students to learn to assess their confidence accurately. We hypothesized study partners (PRT) would outperform students who studied individually (IND) on anatomy knowledge tests due to increased chance of quiz-style learning. Additionally, we thought PRT would more appropriately indicate their confidence on answer choices since they would have received feedback from their partner regarding their answers.
Methods
Students recruited from Pacific Northwest University initially studied a novel anatomical topic. Then, they were split into groups: IND restudied the material independently and PRT in pairs. Next, students’ knowledge of the topic was assessed using 15-question tests on day 1, week 1, and week 3. Students indicated if they felt confident in their answer for each question. Answers marked as confident that were incorrect were labeled as “misinformed.” Students filled out surveys on study day 1. Data were analyzed using SPSS software via two-sided testing.
Results
Day 1 surveys (n=17) showed PRT and IND were not significantly different for age, gender, or prior anatomy knowledge. In the surveys, PRT reported feeling the material was easier to learn than did IND participants (p = .024). On each of the 3 tests, PRT had lower average scores than IND; the difference between groups was not statistically significant, but there was a moderate effect size (p = .302, ηp2 = .082). PRT tended to have more confidence in their responses (p = .272, ηp2 = .092). The number of misinformed answers was more for PRT compared to IND; while not statistically significant, there was a moderate effect size measured (p = .205, ηp2 = .121).
Discussion
Contrary to our hypothesis, on average, students who studied alone performed better on knowledge tests than those who studied with partners. The findings also suggest that students who studied with partners found it easier to learn and felt more confident while taking the tests, despite performing worse leading to an increase in “misinformed” answers. This study was limited by low power, and thus a follow-up study with a larger sample size is warranted to evaluate if the effect size we saw regarding misinformed answers and lack of testing effect in partners is reproducible and significant.
Questions from a judge: I wonder about the mixed background and abilities of the enrolled subjects, which were either OMS-1 students or those who wish to be. What evidence do you have that the groups (individual vs. group study) were evenly matched at baseline? I think your data are consistent with other observations about OMS knowledge retention: Other data suggest that students remember things they learned for just a little more than 2 weeks; your data suggest that the knowledge decreased from about a score of 10 to about a 2 in 3 weeks’ time.
Dr. Elliott,
Thank you for your question. To your first point about study participants, Masters of Arts in Medical Sciences (MAMS) and first-year osteopathic medical students (OMSIs) were evenly distributed among the partner and individual groups. Furthermore, the topic taught to this student was ruminant digestive anatomy which would have been a novel topic to MAMS or OMSIs. We found no significant difference in the knowledge test scores of the MAMS and the OMSIs at day 1 (p = 0.946), week 1 (p = 0.321), or week 3 (p = 0.947). Additionally, all participants were separated into equal groups using the results of the memory tests. The groups partner vs individual were not significantly different from another in the following categories age (p = 0.285), gender (p = 0.335), memory test scores (p = 0.947), hours of sleep the night before the study (p = 0.901), prior anatomy knowledge (p = 0.236), or previous ruminant digestive anatomy (p = 0.888).
In regard to your observation about declining knowledge retention for 2 weeks, you are correct. There was a significant decrease in test scores with a mean score of 10.0 on day 1 and a mean of 4.73 on week 3 (p < 0.0001).
As a judge I find it interesting that all participants studied the material independently first and were only divided into individuals or pairs in the second round of studying. I would assume since participants had already spent time studying, those who were now ‘partnered’ would spend less time studying in round 2 versus those who were alone and had nothing else to do but attend to the material again. Do you think your results would differ if you had split the groups up from the very beginning?
Dr. Fritz,
Thank you for the question. On Survey 2 participants reflected on distractions from studying they faced while in this experiment. Partners estimated they were distracted for an average of 1.4 min of the experiment whereas individuals estimated they were distracted by texts, emails, or the internet for 7.1 mins. Although not statistically significant (p = 0.07) it suggests that the partners were not more off-task than the individuals. Additionally, the groups were all observed by our study coordinators and subjective information was collected by the coordinators about the behaviors of the participants in the different groups. Based on these observations there appeared to be no off-topic participants they all spent the majority of the time trying to learn the material in both groups.
We were looking at restudy techniques and wanted to evaluate changes in subsequent styles of learning after initial exposure that is why we wanted to give them all exposure as an individual. If partners were paired from the onset that would change the set-up of our study because baseline exposure can be important for forming an informational schema. A schema is an important framework in which you can add more information as you restudy. Altering the baseline exposure would be an interesting experiment, but slightly different from what we are hoping to evaluate.
As a judge I am wondering if study materials or methods were given to the participants or if all participants (individual or partner) were left to their preferred study style (within their assigned individual/partner) . Did you track if they used active or passive learning as individuals or partners?
Dr. Salido,
Thank you for the question. In regard to methods given to the groups, half of the individuals and half of the partners were given flashcards and were instructed to quiz on the material. The other half was given the handout they learned from initially and were instructed to study the material. There was no interaction between the flashcards vs handouts and the partners vs individuals, so we felt it was statistically appropriate to make observations at the partner vs individual level. We tracked the use of active vs passive restudy in these groups by observing each group with study coordinators and by collection and examination of the participant’s restudy materials.
There appeared to be examples of passive and active restudy in both groups. In the individual group, active techniques such as quizzing, drawing, and writing information in a new way were present. Individuals also utilized passive ways of restudying such as highlighting, underlining, and multiple passes of re-reading the handout. In the partner group, there were examples of active techniques such as drawing, quizzing, and creating mnemonic devices. Passive techniques in the partner group included re-reading. Some partner pairs reported being unable to review all of the material in the time allotted for restudy. Additionally, in the partner group there was laughter and relating to the material as well as encouragement and affirmation of their partner. Based on subjective observations from our study coordinators it appeared that the partners did participate in more forms of active restudy.