Discrepancies between Student Perception and Achievement of Learning Outcomes in a Flipped Classroom

In a college algebra course that used flipped/inverted pedagogy, students achieved learning outcomes at a significantly higher rate, as evidenced by results on the final exam. At the same time, student perception on a number of measures decreased significantly, including how interested students were in the course and whether the instructor effectively facilitated learning. This article will draw on a variety of research to suggest reasons for these discrepancies and possible solutions to help improve student perception in learner-centered instruction.


Introduction
The purpose of this article is to describe the results of using a flipped (or inverted) pedagogy in a college algebra course.The article will detail the results of several items on the student course evaluations that measure student perception and compare them with a traditional lecture course.It will also describe the results of the course with respect to student learning outcomes, as measured by the results of a final exam and compared with a traditional lecture course, and highlight the discrepancies between the student course evaluations and the final exam scores.Finally, it will draw on a variety of research to suggest reasons for these discrepancies and possible solutions to help improve student perception in flipped instruction specifically and learner-centered instruction generally.
This course took place at X University, a small, Catholic university that enrolls approximately 2,000 students.Among X University students' ACT math scores, the 25 th percentile is 18 and the 75 th percentile is 25.The students taking college algebra were typically not mathematics majors, but were taking it as a required course for their major.
Flipping the classroom typically indicates that the static content of the course is delivered outside of class, and class time is devoted to active learning experiences (Bergmann and Sams, 2012).In the case of mathematics courses, typically this means that the instructor delivers lectures via lecture videos that students watch at home, and class time is devoted to problem solving.This class is no exception.Flipped pedagogy is one of the ways to employ "learner-centered instruction" (Weimer, 2012).Maryellen Weimer characterizes learner-centered instruction in the following way in her article "Five Characteristics of Learner-Centered Teaching" (2012): 1. Learner-centered teaching engages students in the hard, messy work of learning.2. Learner-centered teaching includes explicit skill instruction.
3. Learner-centered teaching encourages students to reflect on what they are learning and how they are learning it.
4. Learner-centered teaching motivates students by giving them some control over learning processes.5. Learner-centered teaching encourages collaboration.Learner-centered instruction is fundamentally different from traditional, lecture-based instruction.It is reasonable that student perception of learner-centered instruction differs from lecture-based instruction, but it is important to understand those differences.

Methodology
In order to assess student perception, the results of certain items on student course evaluations were compared for flipped and traditional courses.All of the course evaluations examined were taught by the same instructor.The flipped course evaluations all came from flipped college algebra courses.The control group came from college algebra as well as two other 100-level courses taught in a lecture-based style by the same instructor.The other courses were included because the number of students in the traditional college algebra course was prohibitively small (N=15).The response rate to the student course evaluations was high (at least 80% of the students who took the final exam completed the course evaluation).This article will consider student responses to the following items: • "The instructor created an environment conducive to the learning process." • "The instructor encouraged my interest in the course." • "Overall, the instructor effectively facilitated my learning." • "Overall, I rate the course as excellent."These were chosen because they are the most relevant items for assessing student perception of learning in the course.
For each of these items, students were asked to evaluate the statement on a scale of 1-5, where the responses indicated the following statements: 1: Strongly disagree 2: Disagree 3: Neither agree nor disagree 4: Agree 5: Strongly Agree It is important to ask whether student perception, as measured by student course evaluations, is a helpful metric to consider.First, student perception of learning is typically considered to be reliable and valid.In a report entitled "Student Course Evaluations: Research, Models, and Trends" Pamela Gravestock and Emily Gregor-Greenleaf assert, "Several studies demonstrate that students are reliable and effective at evaluating … the amount they have learned [and] the ease or difficulty of their learning experience in the course" (2008).This is not to say that students are reliable and accurate in assessing everything about a course-research suggests that students cannot effectively assess the content knowledge of the instructor, instructor bias, or methods of delivery, for example (Gravestock and Gregor-Greenleaf, 2008).Another review of the literature concerning student course evaluations also suggests that students can and do effectively rate their learning and the effectiveness of teaching behaviors in course evaluations (Gravestock, Greenleaf, and Boggs, 2008).Since the items considered here are among the types of items where student responses are widely held to be reliable and valid, it is reasonable to assume that they should be reliable and valid in this case as well.

Results related to student outcomes
Flipped pedagogy has been widely shown to be effective at increasing achievement of student learning outcomes (Bergmann and Sams, 2012;Khan, 2012;Berrett, 2012;Johnson, 2013;DesLauriers, Shelew, and Wieman, 2011;Van Sickle, 2015).Indeed, a recent meta-analysis of 225 studies that compare STEM classes taught with various pedagogical approaches.It indicated that student learning improved by 6% on average, as measured by examinations, and that students in lecture-based classes were 1.5 times more likely to fail than students in classes that used active learning strategies (Weiman, C.E., 2014;Freeman, Eddy, McDonough, Smith, Okorafor, Jordt, and Wenderoth, 2014).This college algebra course was no exception.When compared to those of a control group, students in the flipped classes performed significantly better on a final exam than students in traditional classes (Van Sickle, 2015).The results are detailed in the tables below.Using a one-tailed t-test to determine the statistical significance in all flipped and all control scores, the flipped classes' scores are significantly higher than those of the traditional classes, with p = .0143.
It is important to note that Control 1-5 were taught by a different instructor than the flipped classes; however, the same instructor that taught the flipped classes taught Control 6, and the results are strikingly similar to the average of the other control groups, as shown in the table below (Van Sickle, 2015).Noting the dates of the courses, one can observe that after four semesters of using flipped pedagogy, the instructor decided to revert to a traditional lecture class.This may seem surprising given that students in the flipped classes were performing significantly better on the final exam than those in lecture classes.The change can be explained by the differences the instructor observed in student perception, as measured by student course evaluations.

Results related to student perception
After teaching four sections of college algebra using the flipped model, the instructor observed that the student course evaluations were less positive than her typical student course evaluations.At that point, the instructor had not taught college algebra using a traditional, lecture-based pedagogy at this university and wondered whether the differences in the final exam scores could be somewhat explained by differences between the instructors.Additionally, the instructor was discouraged by the less positive student course evaluations and decided to try teaching a lecturebased class.The lecture videos from the flipped class were made available to students, but not required, and the majority of class time was devoted to lectures, although some student activities were also incorporated.The tables below show the results on certain items in the student course evaluations.It is difficult to reconcile the positive differences in the final exam scores and the negative differences in the student course evaluations, especially the following item: "Overall, the instructor effectively facilitated my learning."Clearly, based on the final exam scores, the instructor more effectively facilitated student learning in the flipped classes, but according to the students' perceptions, the opposite was true.
It is especially difficult to come to terms with this discrepancy because when it comes to their own learning, students are typically considered to be reliable and valid reporters on student course evaluations (Gravestock and Gregor-Greenleaf, 2008;Gravestock, Greenleaf, and Boggs, 2008).In this case, however, it is clear that they are not accurately evaluating their learning in the flipped versus traditional classes.While students in the traditional classes thought they learned more, the students in the flipped classes actually demonstrated increased achievement of student outcomes.

"Engaging in the hard, messy work of learning"
One possible explanation for this difference is that students perceive learning based on how much they feel they are understanding in the class.During a traditional, lecture-based course, students are able to follow along with the instructor's thinking, and they perceive that they are understanding.When they go home and attempt to work problems, they often have difficulty, but looking back at the class, they feel they understood because they understood the lectures.During a flipped class, the lecture videos are watched outside of class, and class time is devoted to working problems.Since working problems often causes students to have difficulty, they experience this difficulty in class.Looking back at the class, they remember the difficulty they had during class, and they do not feel they understood because they did not understand all of the problems.
Indeed, one of the criteria of learner-centered instruction is that it "engages students in the hard, messy work of learning" (Weimer, 2012).Since students are spending their class time engaged in the "hard, messy work of learning," they are more likely to recall that difficulty when it comes time to report their learning on course evaluations.Additionally, students may interpret learner-centered instructional strategies as "professional shortcomings of the professor as if the professor was … not doing his/her job" (Bishop, Caston, King, 2014).Perhaps the most significant challenge of learner-centered instruction is convincing students of its importance (Doyle, 2008;Bishop, Caston, King, 2014).Honest dialogue with students about how they learn, why they take courses, and what type of instruction is valuable may help to ameliorate this problem.

First experience with learner-centered instruction
Another possible explanation for this discrepancy is that the experimental group only participated in a flipped course for one semester.For most students, it was their first experience with a nonlecture-based mathematics class.Indeed, for most students, it may have been their first experience with learner-centered instruction altogether.Substantial research suggests that learner-centered instruction in American high schools is uncommon (Doyle, 2008).Since course evaluations are comparative in nature, students may have had difficulty evaluating their learning in this course in particular.
It is also possible that over time and in taking subsequent mathematics courses, students may realize that they learned more than they thought, but since course evaluations take place at the end of the semester, they did not have this hindsight.Also, for this course in particular, it is the only required mathematics course for many of the enrolled students.As such, they may never take another mathematics course again.

Lack of preparation outside class
In lecture-style instruction, students who do not do their homework will likely do poorly on quizzes and tests, but when they come to the next lecture, they will still be able to follow along.On the other hand, in a flipped class, students who do not prepare by watching the lecture videos will likely struggle significantly throughout the class session.While there are a variety of ways to hold students accountable for watching lecture videos (Bergmann and Sams, 2012;Van Sickle, 2015), there will still be students who do not prepare.For those students, class will be more difficult, and the problems will be harder to understand.This may lead to students feeling worse about the class, and may reflect negatively in their course evaluations.Instructors must work to find the most effective methods for holding students accountable for their outside preparation.Of course, this will not guarantee that all students will come prepared, but it is of utmost importance to insure that as many students as possible do prepare.

Teacher-student interactions
Learning is not merely an interaction between a student and a curriculum-rather, teacher-student interactions are crucial and are an essential component of learning (Boaler, 2002).From observation, it seems that students who spend more time interacting with the instructor in a learnercentered course may have a more positive view of the course.Some students interact less with the teacher because they need help less often.These students may feel like they are teaching themselves.Other students may need help but do not want to ask for it.These students' understandings will suffer because they will not be getting the help they need, and they may feel frustrated during class.Still others may get less attention from the teacher because of their position in the classroom or other happenstance.For students who, by choice or by coincidence interact less with the instructor, there may be a more negative perception of learner-centered instruction.This is one way in which lecture-based instruction is more equitable, especially for the purposes of studying student perception.In a lecture, all students have the same interaction with the instructor, so when it comes to student perception, it is reasonable to assume that course evaluations, for example, measure different students' perceptions of the same experience.In a learner-centered class, course evaluations are measuring students' perceptions of somewhat different experiences, even though those experiences took place in the same classroom.
In a learner-centered class, it is important to maximize teacher-student interaction, and to make sure those interactions are positive.One helpful framework for teacher-student interactions is by encouraging a growth mindset versus a fixed mindset, as put forth by Carol Dweck in Mindset (Dweck, 2006).Dweck's research suggests that students will put forth more effort when they are praised for their effort, rather than their inherent ability.

Class culture-is it okay to be wrong?
The final explanation for the discrepancy between student perception and student learning in the flipped class is that learner-centered instruction requires students to take risks that are not required of them in lecture-based instruction (Doyle, 2008).In learner-centered instruction, students have to give ideas that may be wrong, try things that are new and difficult, and possibly be wrong, all in front of their peers and their instructor.Some students are naturally more willing to be vulnerable than others, but for those students who find taking these risks difficult, this may dampen their perception of the course.Additionally, for those students who have taken risks, been wrong, and then been embarrassed, there may be a negative association with the course that affects their perception (Brown, 2012).
The instructor can help to establish a classroom culture where taking risks and being wrong is accepted by demonstrating a willingness to be vulnerable, as Brene Brown suggests in Daring Greatly (Brown, 2012).Although every class has its own unique chemistry, the teacher has an important role in cultivating a classroom culture where it is safe to take risks and be wrong.Brown has the following "Engaged Feedback Checklist:" I know I'm ready to give feedback when: I'm ready to sit next to you rather than across from you; I'm willing to put the problem in front of us rather than between us (or sliding it toward you); I'm ready to listen, ask questions, and accept that I may not fully understand the issue; I want to acknowledge what you do well instead of picking apart your mistakes; I recognize your strengths and how you can use them to address your challenges; I can hold you accountable without shaming or blaming you; I'm willing to own my part; I can genuinely thank you for your efforts rather than criticize you for your failings; I can talk about how resolving these challenges will lead to your growth and opportunity; and I can model the vulnerability and openness that I expect to see from you.(Brown, 2012) In learner-centered instruction, the challenge is that teachers are constantly giving feedback.Rather than feedback being something given once in a while during office hours, in learner-centered instruction much of each class period is spend giving feedback.Giving engaged and effective feedback is difficult, but it is essential for creating the types of teacher-student interactions that are necessary for learner-centered instruction to be successful.

Student interest in the course
Another troubling item is "The instructor encouraged my interest in the course."Flipped learning had the largest negative impact on this item, and this is concerning particularly because introductory-level courses like college algebra are often places where departments attract and retain new majors and minors (or fail to do so).
While some may argue that if learner-centered instruction is not encouraging interest in the course, it should not be done, it is important to remember that helping students succeed in course also helps to retain them in their major.Indeed, while there is a shortage of students graduating in STEM fields, there is not necessarily a shortage of students interested in STEM fields.The Pesident's Council of Advisors on Science and Technology (PCAST) issued a report that said, "PCAST found that economic forecasts point to a need for producing, over the next decade approximately 1 million more college graduates in STEM fields than expected under current assumptions.Fewer than 40% of students who enter college intending to major in a STEM field complete a STEM degree" (PCAST, 2012).Increasing retention of STEM majors, even modestly, could fill the gap.Additionally, getting students through required mathematics courses more successfully (and with better prerequisite knowledge to build on in future courses) is likely to help retain students in their STEM majors.
Still, that a flipped course encourages less student interest than a lecture-based course is important to consider, study further, and consider ways to ameliorate.

Questions for future research
It is evident that student perception in flipped pedagogy needs further study.In particular, as flipped learning becomes more common, it would be helpful to know whether students become better evaluators of their learning in a flipped class when it is not their first flipped class, or when they have experienced learner-centered instruction more often.
Another important question is whether certain groups are more effected than others.While this study is too small to consider sub-groups of students, a larger study could consider subsets of students to see whether students with certain characteristics evaluate flipped courses more highly than others.

Conclusion
No matter the explanation for the discrepancy between student course evaluations and achievement of student outcomes, it is clear that it exists.When comparing a lecture-based course to a flipped course, students did not accurately evaluate their learning in course evaluations.It is important for faculty and their supervisors to take this into account when considering course evaluations in learner-centered courses.Realizing that it is difficult to compare these course evaluations to those of lecture-based courses is an important first step.If faculty, department chairs, and those who make rank and tenure decisions understand that student perception is poorer for learner-centered instruction than it is for lecture-based instruction, they can bear it in mind, and faculty will not be penalized for it.As flipped learning specifically and learner-centered instruction generally become more common and better-studied, it is important to continue examining student perception and considering ways to improve it.