Reflections versus Extended Quizzes: Which is Better for Student Learning and Self-Regulation?

: Both quizzes and reflections have been found to benefit student learning, but have been typically compared to passive or superficial controls. The purpose of this quasi-experiment is to test the relative effectiveness of brief quizzes followed by reflections compared to longer quizzes. Participants (N = 218) were introductory psychology students enrolled in two different courses, one in which students engaged in weekly brief quizzes and reflections and a second in which students engaged in longer quizzes. Results indicated that the two conditions were similar in effectiveness in terms of learning and self-reports of self-regulation.


Introduction
Instructors who are committed to improving student learning often review the literature and find that many instructional methods may improve student study strategies and subsequent learning. For examples, writing reflections and quizzes have both been shown to be effective methods of increasing learning (e.g., McDaniel, Agarwal, Huelser, McDermott, & Roediger, 2011;Watkins, 2001). However, the benefits of these instructional methods are typically compared to passive controls (e.g., Angus & Watson, 2009;Bannert, 2006;Connor-Greene, 2000;Moos & Bonde, 2015; see Di Stefano, Gino, Pisano, & Staats, 2014 for an exception). Instructional time and student study time are limited, so it would be useful for instructors to know which technique is more effective in an authentic class environment. The purpose of this study is to compare the relative benefits of reflection versus extended practice in a college course.

Benefits of Quizzes
Incorporating quizzes, which are frequent, low stakes learning assessments, has been noted to benefit performance in a variety of disciplines (e.g., Angus & Watson, 2009;McDaniel, Wildman, & Anderson, 2012). Quizzes are thought to be helpful for several reasons under the framework of desirable difficulties (Bjork, Little, and Storm, 2014). Desirable difficulties are challenges that benefit students by improving encoding and processes that support learning (Bjork & Bjork, 2011). Quizzes provide an opportunity for students to practice retrieving and using information from memory (McDaniel et al., 2011). This practice retrieval is beneficial because it can strengthen long-term memory of the material (Roediger & Butler, 2011;Roediger & Karpicke, 2008). In addition, students often overestimate how well they understand content, a phenomenon known as the illusion of competence (Koriat & Bjork, 2005). One way to overcome the illusion of competence is to have students test themselves on the content (Kornell & Bjork, 2007). Unfortunately, students often choose to reread material than to quiz themselves (Karpicke, Butler, & Roediger, 2009) even though rereading material is considered an ineffective learning strategy (see Dunlosky, Rawson, Marsh, Nathan, & Willingham, 2013). For these reasons, requiring quizzes may be helpful for students to improve the monitoring of their learning. Finally, if quizzes occurred frequently throughout a term, they encourage students to study and review the material regularly (i.e., spaced practice) instead of relying on last-minute exam preparation (i.e., massed practice; Connor-Greene, 2000;McDaniel, Roediger, & McDermott, 2007).

Benefits of Reflection
Reflection, the process of examining and contemplating one's knowledge and learning, has also been noted as beneficial for student learning (Dewey, 1933;Peltier, Hay, & Drago, 2005;Rogers, 2001). Reflection provides opportunities to think about performance, consider what strategies are effective, and contemplate how to improve in the future (Mann, Gordeon, & MacLeod, 2009). Reflection is a key process in effectively choosing and monitoring strategies to achieve one's goals, known as self-regulation (Dabbagh & Kitsantas, 2004;Ellis & Davidi, 2005). Specifically, reflection is critical to monitoring behaviors during a task and evaluating the effectiveness of these behaviors (Nussbaumer, Dahn, Kroop, Mikroyanndis, & Albert, 2015). Moreover, self-regulation can be taught with reflections (Paris & Paris, 2001). For these reasons, reflection can improve learning by making it intentional and purposeful (Di Stefano et al., 2015). This is likely why empirical evidence indicates learning benefits with reflections (see Zheng, 2016 for review). For examples, reflections on learning strategies used for mathematics assignments lead to improved course performance (Zimmerman, Moylan, Hudesman, White, & Flugman, 2011). In addition, reflecting on learning strategies improved students' skills to develop their own knowledge (Cacciamani, Cesareni, Martini, Ferrini, & Fujita, 2012). Moreover, prompts to reflect on one's learning process improve performance on transfer tasks (Bannet, 2006;Bannert & Reimann, 2012). In work contexts, individuals who engaged in reflection had lower error rates when learning work skills than their less reflective peers (Roessger, 2014). Finally, choosing to engage in reflective thinking practices is positively associated with academic performance (Phan, 2014).

The Current Study
The discussion on quizzes and reflections indicate that both may improve student learning. However, the amount of time students are willing to apply towards a course is finite. Given restraints, is it better to answer more quiz questions thereby covering more content related to exams or to have a shorter quiz followed by a reflection on learning? The current study addresses this question with a quasi-experiment using two large introductory psychology courses. One course had quizzes followed by a reflection on how they learned the material. The other course had a longer quiz, but no reflection. Because reflection is most effective when regular and ongoing throughout the semester (Bringle & Hatcher, 1999), quizzes and reflections were weekly.
The current study examined the relative effects of quizzes and reflections versus extended quizzing on two sets of dependent variables: performance on exams and self-reported selfregulation. Performance on exams was chosen because it is indicative of overall learning in the course and the variable of interest in class-based quiz studies (e.g., Rezaei, 2015). Self-regulation was of interest to allow for examination into possible changes during the semester as a result of reflections. If the results of the current study indicate that reflection benefited performance compared to extended quizzing, it is likely because reflection improved self-regulation. Typically, studies on quizzes compared conditions with quizzes to control conditions without a comparable learning activity (e.g., Angus & Watson, 2009;Bjork et al., 2014;Connor-Greene, 2000; although often conditions varying the types of quizzes were also compared; e.g., Bunce, VandenPlas, & Havanki, 2006;McDaniel et al., 2011) or quizzes were compared to an ineffective study strategy, such as rereading (McDaniel et al., 2012;McDermott et al., 2014). Moreover, reflection studies typically used passive controls in which students were not given reflection prompts (e.g., Bannert, 2006;Moos & Bonde, 2015; although there may be comparisons in the types of reflection activities as well; e.g., Michalsky & Kramarski, 2015;Roessger, 2014). In contrast, the relative benefits of brief practice followed by a reflection versus extended practice without reflection on mathematical tasks were compared in a lab experiment (Di Stefano et al., 2015). Findings from this work indicated that, although individuals indicated a preference for extended practice over reflection, performance improved with reflection compared to extended practice. Based on these findings, it is likely that reflection would lead to better study strategies and course performance in the current study than would extended quizzes. However, lab results do not necessarily transfer to class environments (see Hulleman & Cordray, 2009). This may be especially true when considering the work by Stefano and colleagues as they examined performance on a single mathematical skill whereas the current study examined performance on understanding a broad range of psychological concepts.

Participants
Participants were 218 introductory psychology students at a large Midwestern university. Only students who completed at least 75% of the quizzes and/or reflections (depending on condition) and completed both pre-and post-self-regulation measures (see Measures) were included in analyses. Approximately 60% of the students identified as female and 40% identified as male with an average age of 19.02 years (SD = 1.89 years). In terms of racial background, approximately 86% reported being Caucasian, 2% reported being African American, 4% reported being Asian, 1% reported being Hispanic, 3% reported being biracial or multiracial, and the remainder did not report racial background.

Procedure
Participants were in two different courses, both taught by the same instructor and both offered in the afternoon. The lectures and exams were the same in both courses. For the course in the quizand-reflection condition, students completed a 5-item, multiple-choice quiz weekly followed by a reflection of how they learned the material for the quiz. Following exams, students reflected on how they learned the material for the exam. For the course in the extended quizzing condition, students completed a 10-item (5 of the items were the same as in the quiz-and-reflection condition), multiple-choice quiz weekly. Information targeted by the quizzes was either on the exams or related to information on the exams. There were 104 participants in the extended quizzing condition and 114 in the quiz-and-reflection condition. All weekly assignments were completed online through the course learning management site. Students were given immediate feedback on their quiz performance. Following best practices in teaching with reflections, reflections were given feedback as well (Aronson, 2011). Only students who completed at least 75% of the quizzes and/or reflections (depending on condition) were included in analyses. At the beginning of the semester, students in both courses learned about effective and ineffective study strategies as part of the cognitive psychology unit. In addition, students were required to read and write a response to a systematic review of study strategies written by Dunlosky and colleagues (2013) at the beginning of the semester. Therefore, they had a background in study strategies upon which to reflect.

Measures
Three unit exams were the measures of course performance. Unit exams were not cumulative. There was also an optional final exam (students could choose to take the final exam to replace a previous, lower unit exam grade) that 112 students chose to take. The final exam was cumulative. Each exam consisted of 40 multiple-choice questions. The Metacognitive Self-Regulation Scale (MLSQ; 12 items) from the Motivated Strategies for Learning Questionnaire was used to measure self-regulation (Pintrich, Smith, Garcia, & McKeachie, 1993). The MSLQ has been used in many research studies and is considered a useful, flexible tool for examining the differential effects of interventions (Duncan & McKeachie, 2005). It was administered to students in class during the second week of the semester and the second-to-last week of the semester.

Results
To compare the relative effectiveness of the quiz-and-reflection condition versus the extendedquiz condition on performance, the effects of condition on both the sum of the unit exams scores and the final exam score were examined. First, a one-way ANOVA was conducted with condition as the independent variable and the number of items answered correctly on the three unit exams as the dependent variable. There was no difference between conditions, F(1, 217) = 1.07, p = .30 (see Figure 1). Then, a one-way ANCOVA was conducted with condition as the independent variable, the number of items answered correctly on the final exam as the dependent variable, and number of items answered correctly on the three previous unit exams as the dependent variable. Again, there was no difference between conditions on exam performance, F(1, 110) = .20, p = .65 (see Figure 2). Previous exam performance was a significant covariate, F(1, 110) = 174.61, p < .001.

Figure 2: Final exam scores by condition (covariate adjusted means and +/-1 standard error bars)
The relative effectiveness of the conditions on metacognitive self-regulation was also examined. A one-way ANCOVA was conducted with condition as the independent variable, scale score at the beginning of the semester as a covariate, and scale score at the end of the semester as the dependent variable. There was no difference between condition, F(216) = .04, p = .84 (see Figure 3). Scale score at the beginning of the semester was significant as a covariate, F(1, 216) = 127.03, p < .001.

Discussion
The purpose of this study was to compare the relative benefits of a brief quiz followed by a learning reflection versus an extended quiz. Based on the findings from this study, there was no difference in the two instructional methods on student learning or self-regulation. These findings may be informative for instructional design. For example, an instructor who wishes to incorporate reflections, but is concerned that less content would be covered as a result, may find these findings reassuring that covering less content in quizzes may not lead to less learning in a course. Conversely, an instructor who teaches large enrollment courses with minimal assistance for grading may appreciate knowing that longer quizzes, which may be automatically graded with feedback through learning management systems, may be as useful as reflections, which may be labor intensive to grade.

Limitations and Future Directions
Given that both quizzes and reflections have been previously found to improve learning (e.g., McDaniel et al., 2012;Di Stefano et al., 2015), it was not surprising that there was no difference in exam performance between the two courses. However, it was anticipated that reflection would increase self-regulation given previous findings (Dabbagh & Kitsantas, 2004;Paris & Paris, 2001;De Steafno et al., 2015). One reason for the similar self-regulation scores between conditions could be that the extended quizzes prompted the students to self-regulate as they had to apply their learning strategies to more content than did students in the quiz-and-reflection condition. Another possibility is that reflection prompted enhanced self-regulation more than extended quizzing, but that the self-report measure was not sufficiently detailed to detect a difference. An interesting idea for future work would be a mixed methods approach and, in addition to the quantitative data from the self-reports, obtain qualitative data by interviewing a sample of students about how they regulate their learning and how quizzing and reflection contribute to self-regulation. The findings from such a study would provide information of both breadth and depth to illuminate the relationships between these instructional methods and selfregulation.
One important limitation of the design in this study is that a control condition without quizzes or reflections was not incorporated. However, such a condition was considered unnecessary. There did not appear to be a need to conduct another empirical study showing that quizzes and/or reflections yielded learning benefits over a passive control or a control with an ineffective study strategy (e.g., rereading). Moreover, it could be considered unfair to students in a control condition to deprive them of an instructional method shown to be effective in the literature. Overall, the design of this study, a quasi-experiment comparing two techniques previously found to be effective, afforded practical guidance for instructors to make decisions about their courses. Given the number of instructional methods found to have benefits over passive or superficial controls, it may be helpful for future studies to compare the relative effectiveness of instructional methods to help instructors determine which methods are most effective. Instructors who wish to incorporate research-based practices into their courses may be overwhelmed by the number of methods noted as effective in the scholarship of teaching and learning literature. To help instructors make informed choices, this manuscript discusses a study in which the relative effectiveness of two instructional methods-quizzing and reflections-are assessed in a quasi-experiment. Although the results indicated similar scores on learning and self-regulation measures for the two methods, these findings provide guidance for instructors. Instructors can apply these findings to know that quizzes and reflections appear to be equally effective and base their decision on what would work best for their particular learning environments.