You Can Lead Students to Water, but You Can’t Make Them Think: An Assessment of Student Engagement and Learning through Student-Centered Teaching

: The current project conducted an assessment of specific, directed use of student-centered teaching techniques in a criminal justice and criminology research methods and statistics class. The project sought to ascertain to what extent these techniques improved or impacted student learning and engagement in this traditionally difficult course. Overall, the results indicate a modest but notable increase in student engagement and subsequent learning. Results provide empirical evidence that students were significantly engaged with the course and therefore benefited from these pedagogical techniques.


Introduction
Pedagogical research has become a burgeoning field in many academic disciplines as more and more teachers have wanted to know how to improve student-learning outcomes (Albers, 2007;Slunt & Giancarlo, 2004).However, there are some disciplines that have explored the scholarship of teaching and learning more deeply than others.The academic discipline of Criminal Justice and Criminology remains relatively untouched by deeper exploration into this realm of research (Stack, 2007).Instructors within this discipline acknowledge the technological innovations that allow for more engaging and interactive teaching methods (Davis, et al., 2014), however, these improvements have largely focused on experiential learning, service-learning, and the use of internships (Davis, et al., 2014).Team-Based Learning is also a technique that has gained popularity in this discipline, but increased class sizes and reduced resources are likely a culprit as to why this pedagogical technique is not used more frequently (Stamatel, et al., 2013).Although there are likely many who are employing innovative teaching techniques, they are not necessarily being assessed in a research environment, nor are they necessarily being published.This paper intends to add to the growing body of literature on the scholarship of teaching and learning as well as pave the way for further pedagogical research specifically within the discipline of Criminal Justice and Criminology.
The purpose of pedagogical research is to identify the best practices that improve studentlearning outcomes (Haggis, 2009).This requires defining those outcomes and employing one or more techniques that can be empirically assessed as impacting those outcomes.The traditional teacher-oriented model poses a risk to courses that would benefit from greater engagement (Gordon, et al., 2009).In these traditional classrooms, the well-organized and self-motivated students will typically do very good work; there will always be students who may choose not to engage at all and subsequently fail the course, but the rest will fall largely in the middle as "average" (Albers, 2007).Although this teaching-centered method has been employed for ages, only in the last several years have teachers begun to break down these barriers and employ alternative techniques that have a more lasting effect on student-learning outcomes.Further, research suggests that even large-sized classes can benefit from non-traditional, student-centered approaches (Gordon, et al., 2009).
The increased use of student-centered learning has produced a variety of techniques to include the three specifically addressed in the current project: Team-Based Learning, Incentive-Based Learning and Flipped Classroom.Briefly defined, these pedagogical techniques respectively employ facilitated group work, the use of positive-reinforcement for work, and using the classroom for more direct interaction as a result of directed work outside of the classroom.As mentioned previously, it is also important to define the student-learning outcomes that any technique is hoping to impact.An increase in course grades or doing well on examinations are commonly used measures (Haggis, 2009), but there are others such as student engagement, selfassessed learning, and other objective and subjective measures of learning.The concept for this project was derived from an observation that the average criminal justice student dislikes the research methods course and that this course is regularly outperformed by all other courses within the discipline.This personal observation is empirically rooted in the concept that students are generally apathetic about traditional learning methods (Stamatel, et al., 2013).Additionally, Briggs, et al. (2012) identified three concepts that specifically define apathy in research methods courses: Research Disinterest, Relevance Argumentation, and Math Anxiety or "DRAMA."This project seeks to explore a single research question: are student-centered teaching methods effective at improving student engagement and learning?

Literature Review
Pedagogical techniques have been explored recently to help improve teaching methods and subsequently improve student learning (Haggis, 2009), specifically in traditionally difficult courses such as research methods and statistics (Sundt, 2010).There are many techniques that have emerged as effective, but so much of that effectiveness relies heavily upon a variety of variables to include (but not limited to) method of delivery, dynamics of the classroom, demographics of the students, nature of the academic discipline and school program, and episodic incidents that cannot be accounted for during any given class (Haggis, 2009).These approaches are often assessed for effectiveness on a variety of student learning outcomes such as improved final exam scores, final grades, and self-assessed learning (Grant & Green, 2013).Since it appears that these techniques are generally fruitful, this study proffers the idea that the use of any student-centered pedagogical approaches will improve learning outcomes and student engagement, even in a traditionally difficult course.Although this study focused on the pedagogical approaches that primarily utilized elements of team-based learning (TBL), flipped classroom, and incentive-based learning (IBL), these were not specifically assessed for effectiveness, but instead the use of any

Student-Centered Learning
Student-Centered Learning (SCL) typically acknowledges the more active role a student plays in their own learning (Judi & Sahari, 2013).Oblinger and Maruyama (1996) identified concepts that focus on students' ability to actively seek out resources available to them for learning while simultaneously teachers facilitate and employ teaching methods that encourage the use of these resources (Asoodeh, et al., 2012).Early SCL typically relied upon methods such as cooperative learning, critical thinking and dedicated group assignments (Felder & Brent, 1996).Building upon these foundations both classrooms used in the current study utilized a form of team-based learning (TBL), more commonly known as "group work."This approach has proven effective in many capacities as it is generally recognized that students who work in teams or groups will typically outperform students who work individually (Nicoll-Senft, 2009).Additionally, TBL, when used correctly, can outperform the use of the traditional lecture (Jafari, 2014).
TBL facilitates an active learning environment and the approach appears to considerably improve student cooperation, preparation, and a foundation for common-sense thinking that leads to improved critical decision-making (Stamatel, et al., 2013, Gullo, et al., 2015).Despite some of the common perceptions of group work from students (e.g., unwillingness to work with others, feeling that only one person will do all of the work, difficulty in scheduling time to meet, etc.), there is evidence that suggests many students ultimately enjoy group work because they can collaborate with others, divide the workload into more manageable pieces, and work toward a common goal, mimicking many workplaces (Fearon, McLaughlin, & Eng, 2012).TBL facilitates group work in a way that utilizes best practices of both individual and cooperative learning techniques (Jafari, 2014).It is the directed facilitation that sets this apart from other forms of group work and increases the likelihood of student engagement (Gullo, et al., 2015).Additionally, a major benefit of TBL is that it can be employed in combination with a variety of other studentcentered teaching techniques, as is the case in the current project.
To achieve the desired outcome of engaging the students more with the material to facilitate better learning, in addition to TBL, the current project also utilized variations of other pedagogical techniques; specifically, incentive-based learning (IBL) and flipped classroom.IBL is inspired by the use of positive reinforcement for good work, ideally, the use of incentives should motivate and inspire better work, where students compete not with each other but against deadlines and facilitated challenges.Used as a pedagogical technique, it is the use of incentives to encourage work that is above and beyond in some way, either in substance and/or in efficiency in completion (Hout, et al., 2012).As such, an ancillary effect of the use of incentives is that they may inspire faster work, but not necessarily better work.In other words, it may inspire a competitive environment, but not necessarily one more conducive to learning.
From a behaviorist perspective, the use of incentives for learning is a balance between providing motivational incentives but not necessarily a bribe (Astroth, 1994).Further, a brief review of psychological research shows the "Skinnerian" approaches of applying extrinsic incentives does not necessarily overcome the value of natural intrinsic motivation (Grant & Green, 2012).Hout, et al. (2012) found a more reliable way to measure the effectiveness of incentives may be to focus on low-stakes incentives, that is, those that would not increase student motivation in an inflated manner.Tests are a commonly used incentive approach, yet may naturally contradict josotl.indiana.edustudents' intrinsic motivations to learn because they are considered a high-stakes incentive (Hout, et al., 2012).Students may focus too heavily on the reward (the grade), ignore the original intent to learn and subsequently default to avoiding the potential punishment: a poor grade (Grant & Green, 2012).
In general, the use of any "threshold grading" may not be an accurate measure of learning or mastery of the material.Grant and Green (2012) studied the effectiveness of using predetermined grade thresholds, which had been supposedly validated as indicative of mastery of the material, and ultimately found that they did not produce any actual predictive grade performance.In other words, despite an understanding that students knew what it took to get a certain grade level, there was no evidence that it motivated students to perform any better.Babcock (2010) also found the use of incentives indicated that the teaching method may support the incentive more so than student motivation.On the other hand, when grading relied more on commonly achieved selfand peer-evaluations, fluctuations based upon the norms established by student groups were more apparent (Grant & Green, 2012).
This indicates that a more self-defined mastery of material from the student perspective led to greater achievement.Benvenuto (2001) also assessed this structure using a "maximum incentive, minimum discipline" model in a chemistry course and looked at a reward-heavy objective for good work, as assessed by their peers, without the fear of a punitive grade for poor work.Although students generally favored this method of incentive and peer-evaluation, final assessment revealed students did desire some form of punishment for those who did not contribute good work (Benvenuto, 2001).Although the literature may appear contradictory at times, it seems as though the use of incentives could be fruitful when the teaching method balances with self-and peer-defined expectations by the students, and those students have a high intrinsic motivation to learn.
The use of extra credit as an incentive is meant to inspire and motivate additional work from the students, but is considered by some to be one of the most controversial and methods of motivating learning (Fuad & Jones, 2012;Pynes, 2014).Typically, the over-achieving students will participate in any extra credit opportunities, but they are not usually the students who actually need the extra credit, whereas other students rely too heavily on extra credit to save their grade (Pynes, 2014).In a study assessing the use of extra credit in a computer science course, Fuad and Jones (2012) found that using quizzes as extra credit relieved the mental stress that was associated with traditional test-taking perceptions, and thus became a learning tool rather than a punitive measure of learning (Fuad & Jones, 2012).Stack (2007) looked at the effectiveness of extra credit for both specific and general learning outcomes in a criminological theory course.Stack (2007) subsequently concluded that although the extra credit did improve grade performance on directly related quizzes, the overall general assessment of learning (final exam) did not indicate significant grade improvements.Pynes (2014) identifies many of the traditional uses of extra credit as being only weakly related to desired student learning outcomes, thus sabotaging attempts at properly motivating students.
When used appropriately, incentives can motivate students, but may not necessarily improve overall student engagement.Flipped classroom takes another approach to the student engagement issue by requiring students to do pre-assigned homework outside of the class, so that the class time is used to go over the dedicated assignment and material the students independently covered (Novak, 2011).Students are responsible for reading any relevant course materials in order to complete the homework assignment.This method "flips" the lecture from within the classroom to outside the classroom, and the homework is covered more extensively within the class instead josotl.indiana.edu of at home.In order for this method to be successful, the instructor must make the instructions very clear (Novak, 2011).Formica, et al. (2010) and Halloun and Hestenes (1985) found that students who are required to perform on a homework assignment without proper guidance failed to truly learn the objectives of the assignment.Formica, et al. (2010) compared two groups where one group was taught using the traditional class lecture-homework assignment method, while the second group used the flipped classroom method.The results indicated that the students under flipped classroom had a greater understanding of the concepts than the traditional group.The underlying reason for the effectiveness of the flipped classroom method is that the classroom focus is on class discussion and facilitated critical thinking (Slunt & Giancarlo, 2004).In a traditional lecture environment, students are asked to think critically all on their own without ever being given the opportunity to engage with each other in a critical thinking environment (Novak, 2011).
If a technique such as flipped classroom facilitates greater engagement, then learning gaps can possibly be more easily identified and attended to more readily.Natarajan, et al. (2014) used the flipped classroom method in an algebra class to ascertain if there was a specific gain in learning where students can actually reach a level of perfection on the assigned material in the allotted time.A Chi-Square significance test (p < .05)supported the notion that if a threshold of perfection on an assignment is identified, students can work toward that goal, and thus any potential learning gaps can be immediately identified and addressed, significantly improving learning outcomes.Novak (2011) illustrated the positive use of web-based pre-instruction warm-ups as a fundamental component of flipped classroom.Closely related to the Just-in-Time Teaching approach, the use of modern technology allows instructors to disseminate pre-instruction materials to students more efficiently, and subsequently allows instructors to prepare the material "just in time" for class.This approach allows teachers and students to work more as a team ready to begin the lesson with a high level of relevance to the materials to be discussed (Novak, 2011).Further, this interaction allows students to have some control over their learning and to enrich the time spent in class.
It is important that all student-centered learning techniques focus on clear instructions and consistent methods in order to achieve the best results, as well as being facilitated effectively by the instructor.Only a handful have been covered here, not to promote these methods specifically, but rather the concept that any student-centered learning approaches have greater chances at increasing student engagement.Thus, the current project examines the role of these studentcentered teaching techniques and seeks to answer one question: Are student-centered teaching methods effective at improving student engagement and learning?From this larger research question, the following hypothesis is proffered: H1) The use of student-centered teaching methods will improve student engagement and learning.

Participants and Procedure
The current study was conducted in two undergraduate criminal justice and criminology research methodology classes from a medium to large-size, urban university.This university is located in the downtown area of a large city and is described as a commuter school with many nontraditional studentspart time, older, fulltime workers.The sampling process was not random and contained  Bachman and Schutt (2011), and employed some form of team-based learning in combination with another student-centered technique (IBL or flipped classroom).We collected data from 58 undergraduates (22 males, 36 females).Their ages ranged from 18 to 55 (M=24.52).Fifty percent were seniors, 33% junior, and 17% sophomore.Approximately 88% of participants were full time students and 83% were employed.These demographics typically reflect the overall composition of the criminal justice and criminology department student body (Table 1).Following the work of Handelsman et al. (2005), a survey instrument of 24 behaviors and attitudes indicative of student engagement and learning was created.The survey instrument was given to the students in two Criminal Justice research methods courses during the first week of class and then again during the last week of class.The survey was administered during class after an implied consent was given.The survey sought student demographics such as age, gender, and class standing.Likert-type questions were asked regarding student engagement and learning and were ordinal scale measures and coded 0 for "strongly disagree", 1 for "disagree", 2 for "agree", and 3 for "strongly agree" for each question.The survey focused on two main areas: emotional engagement and active engagement.

Exploratory Factor Analysis and Reliability josotl.indiana.edu
To measure a student's engagement to the class, students indicated a level of agreement to the statements below.A factor analysis was conducted on these eight variables in order to reduce the data into measures that are inter-related.The factor analyses extracted two components which accounted for 64% of the variance.The first factor consisted of six items labeled as emotional engagement had 46% of the variance with a coefficient alpha of .829.The second factor consisted of two items labeled as active engagement and had a 20.6% of the variance with a coefficient alpha of .594.

Emotional Engagement Scale:
Active Engagement Scale: Like small groups Attend classes Like to experience learning Class is interesting Motivated Class is fun Enjoy class Engaged

Regression Analysis
The following analysis explores the effect of student-centered teaching on student engagement and learning in a college level research methods class.Using the post-course student engagement questionnaires, multivariate analysis was conducted to determine what attitudes may predict student engagement.
To answer the question concerning which variables could predict a student's class engagement, two linear regression analyses were performed.The first analysis used the Emotional Engagement scale as the dependent variable and the three predictor variables from the survey questionnaire: Class will help my career, I'm a good student, and I'm motivated by grades.The model also included the control variables of full time student, employed, age, and gender.The analysis showed an overall significant model (R 2 = .695;p=.000) with a multiple correlation coefficient of .834,indicating that approximately 70% of the variance within the sample can be explained by this model.Holding all other variables constant, the emotional engagement variable significantly influenced a student's engagement and learning in the classroom.In particular, those students who felt the class helped their career, who felt they were a good student, and were motivated by grades were significantly more likely to be emotionally engaged in the classroom (Table 2).Note: R² = .695,df = 7, n = 51, p = .000 The second analysis used the Active Engagement scale as the dependent variable and the two predictor variables from the survey questionnaire: Come to class prepared, and I like to work in small groups.The model also included the control variables of full time student, employed, age, and gender.The analysis showed an overall significant model (R 2 = .321;p=.000) with a multiple correlation coefficient of .566,indicating that approximately 60% of the variance within the sample can be explained by this model.Holding all other variables constant, the emotional engagement variable significantly influenced a student's engagement and learning in the classroom.In particular, those students who came to class prepared were significantly more likely to be emotionally engaged in the classroom.Conversely, students who like to work in small groups were less likely to be emotionally engaged in the classroom (Table 3).Note: R² = .333,df = 6, n = 51, p = .004 In order to improve student engagement, new approaches involving method of delivery and dynamics of the classroom have become effective tools (Haggis, 2009).Student centered learning techniques such as flipped classroom approaches and team based learning set up, offer students new approaches to become engaged in the classroom.The question remains, can using these techniques positively impact student engagement and learning in the classroom?
It is clear that the student-centered teaching techniques assessed in this project did have a positive impact on student learning outcomes.Using our survey instrument, this study was able to look at student engagement from two perspectives: emotional engagement and active engagement.For instance, emotional engagement was increased when students felt the class would help their career, felt they were a good student, and were motivated by incentives.Looking at active engagement, students felt more engaged when they came to class prepared.However, the analysis also showed that students felt less actively engaged when they worked in small groups.
To further our understanding of student centered learning, we looked at the comments made on university wide student evaluations for both courses.Within these forms we found supports for the statistical findings.For example, students also commented on the teaching style, such as the student who said "excellent teaching style and engaging class projects", "the instructor provided a great classroom atmosphere with multiple avenues of learning", and "the team based learning is new to me, worked out ok." total of 58 students from two separate research methods classes.Each class was instructed by different instructors, but met in the same semester and used the same text, The Practice of Research in Criminology and Criminal Justice by Bradford, Mowder, and Bohte Journal of the Scholarship of Teaching and Learning, Vol. 16, No. 4, August 2016.38 josotl.indiana.edua

Table 1
Demographic Characteristics

Table 2
Linear Regression Model for "Emotional Engagement" Journal of the Scholarship of Teaching and Learning, Vol. 16, No. 4, August 2016.

Table 3
Linear Regression Model for "Active Engagement"