Developing Faculty Communities of Practice to Expand the Use of Effective Pedagogical Techniques

The scholarship of teaching and learning literature is replete with examples of pedagogical techniques that have been demonstrated to be effective in improving learning, motivation, and student success. The extension of these techniques beyond the original context has tended to be slow, difficult, and incomplete. The following paper examines an intervention designed to encourage the exploration and use of a variety of pedagogical techniques by faculty in a traditional, four-year college faculty within the context of the AASCU Re-imagining the First Year Initiative. Small groups of six to eight faculty, joined and created communities of practice. The groups were known as Pedagogical Interest Groups, or PIGs for short. The faculty read about and analyzed a series of pedagogical techniques and committed to introducing at least one technique into their courses to further explore the techniques. When the techniques were successful, the faculty members redesigned entire classes to expand the impact. The communities of practice were successful in encouraging faculty to explore a wide variety of techniques. The average faculty group explored eight different pedagogical techniques. Faculty were able to use the opportunity to experiment with techniques with the support from their colleagues in their PIG. A dozen techniques were explored across the PIGs and dozens of class sections have been completely redesigned. To date, over 2000 students have experienced redesigned courses. Measures of student success, satisfaction, and interest in those sections have increased. The effort has been accompanied by a robust increase in the campus-wide retention rates.

A central goal of the Scholarship of Teaching and Learning is the development and promulgation of effective, evidence-based, pedagogical techniques, methods, and strategies.The ultimate goal of this endeavor is the improvement of learning, motivation, and student success.While the development efforts have led to the identification, creation, and assessment of many techniques, the efforts to extend those techniques beyond the original contexts has not been as successful (Elmore, 1996;DeDe, 2016).One of the factors limiting the expansion of the techniques is simply a lack of knowledge of the mechanics and the advantages of particular pedagogies on the part of college faculty (Dillenbourg, 2017).Another is the limitations of the "better mouse trap" philosophy that underlays the promulgation of pedagogies.This approach posits that better methods and practice are selfexplanatory and self-implementing.Once faculty see the better procedure, they will quickly adopt the new technique (Elmore, 1996).The flaw in this approach is that faculty do not know about how to teach through the continuing formal exploration of pedagogical techniques throughout their careers.In this paper, we explore the use of faculty communities of practice as an explicit instance of faculty development to encourage the exploration of a variety of pedagogical techniques, to increase breadth of pedagogical expertise in our faculty, and, ultimately, to improve student learning and academic success, i.e. build and use "better mouse traps".These efforts are focused primarily on foundational gateway courses which are often a barrier to student success (Flanders, 2017).
This project has been inspired by the Reimagining the First Year Initiative (RFY) sponsored by the American Association of State Colleges and Universities (AASCU).RFY is grounded in the belief that Colleges and Universities can do a better job of meeting the needs of our students and of responding to the current societal and academic mandates (McBride & Kanekar, 2015).RFY suggests that many models exist that point to better ways of reaching the needs of our students.Schools should study those models and endeavor to replicate their already learned lessons.These models concern a wide variety of practices that include both pedagogical and non-pedagogical components.
Over the past several decades, considerable scholarly effort, reflection, experimentation, and analysis have been expended in trying to develop pedagogical techniques that are effective in reaching the needs of a wide range of students.This can include students who may not have enjoyed a solid foundation in high school, who may be from groups who may not have attended college in previous generations, who enroll in less selective institutions, who are first-generation college students in their families, or who are from underrepresented groups.These techniques contrast with the traditional lecture and textbook based systems that have characterized higher education for some time (Hainline, et al, 2010).Often, the faculty of universities have attended schools that primarily used traditional lecture and textbook based systems, use those experiences as their models, and have not received appreciable formal training in teaching and little more than incidental exposure to modern alternative techniques.

Faculty Communities of Practice-Pedagogical Interest Groups (PIGs)
In recent years, communities of practice as a vehicle for faculty development have grown in popularity (e.g.Abigail, 2016, Teeter, et al, 2011, Wenger-Traynor 2011, 2015).A community of practice is simply a group of people who form to pursue a common goal.They share a domain of interest, they engage in joint activities and discussions, and they develop a shared repertoire of resources/practices (Wenger-Traynor, 2011).We developed a series of faculty communities of practice to explore, learn about, and assess pedagogical techniques.Since our communities shared an interest in pedagogy, we referred to the groups as Pedagogical Interest Groups, or PIGs for short.This name was created by the first of the community of practice groups in this project and continues to the present.Our communities had six to eight faculty in them, usually from different disciplines.Initially, they were invited to participate by their Dean, though eventually faculty began asking to join a PIG.Each group selected the particular pedagogical techniques that they wanted to study.Each group read about and analyzed different techniques.Most frequently, faculty have heard about particular pedagogical techniques, were intrigued by them, but have not actually studied them.Overall, the members of the PIGs have studied about a dozen techniques.This includes: Active Learning, Belongingness, Collaborative Learning, Community-Based Learning, IU Career EDGE (Exploration, Development, Graduation, Completion) Modules, GoFar (achievement motivation intervention), Growth Mindset, Inclusive Learning, Nudges, Problem-Based Learning, Social Media Learning, Stereotype Threat, and Transparent Teaching.
Within the Communities of Practice, faculty read articles about how to arrange and use particular techniques, the academic contexts in which they had been deployed, and assessments of their efficacy.The members of the PIGs evaluated the techniques through a series of analyses.Did they have enough information to replicate the techniques?Did the techniques produce appreciable change in measures of student learning or in measures of student success?Were these techniques consistent with the general approach to knowledge and learning pursued by the individual instructor?Could these techniques be accepted by our students?The study phase for the PIGs lasted for about a semester.By the end of the semester, each professor in the PIG selected one technique to attempt within a class.The initial extension was developed to help consider some of the questions posed above.For instance, could we arrange the technique?Could this technique work within our context?Thus, one could consider the first exploration as similar to a pilot study.At the end of the semester, individual teachers and the individual Communities of Practice (PIGs) assessed the pilot studies using self-defined appropriate criteria such as attendance, completion of a project, or grades or quality of an assignment.In addition, the Dean's office examined course grades, DFW rates, and retention rates.If the technique was successful, the teachers were encouraged to redesign the course using that technique as a basis.
Part of the specific context of these efforts are the characteristics of the school and of the student body.The university is a comprehensive regional state university with a diverse student body of approximately 3800 and offers Associate, Baccalaureate and Master's degrees in a variety of undergraduate and graduate programs.The student body is 46% underrepresented minority, 43% first generation, 70% female, 66% traditional aged and 67% fulltime.Over the past 5 years, the 6-year graduation rate has ranged from 24 to 27% and first to second year retention rate ranges from 64 to 67 percent.

Findings
One of the most impressive outcomes of this project is that all the techniques improved aspects of student learning and with some techniques, the change was immediate.For instance, during the exploration of collaborative learning in a foundational mathematics courses, it was obvious from the first day that students were responding positively to the new techniques.About 100 students were enrolled in three sections of this class.Prior to the intervention, students had spotty attendance and were not always focused on the activities within the classroom.When the students were doing their first collaborative assignment, all attention was focused on the problem, the students were deep in conversation with each other, and engagement with the content was obvious.This pilot lasted two and a half weeks.During that time, there was nearly perfect attendance.Grades were higher on the examination at the end of this section and the mathematics faculty began redesigning the entire class to include collaborative learning assignments throughout the semester.We have used Communities of Practice for about two years now.We have formed eight PIG groups, with 46 faculty from 13 different disciplines participating.So far, they have introduced 12 different techniques into 15 different classes.To examine their efficacy, faculty collected data on individual techniques as appropriate to the procedure.For instance, mathematics collected attendance data and exam grades.Others collected project completion rates, understanding of course material, or measures of the depth and sophistication of papers.The activities were all aligned with the learning outcomes, and the measures of efficacy consistently designed and employed across semesters.In all cases, standard measures of student success: course grades (course GPA), DFW rates, and one-year retention rates were collected.The measures of student success are displayed in Table 1 for courses that were offered after a complete redesign.These same measures from comparison courses are also displayed.These measures were collected from the same courses that were targeted for interventions during the last semester prior to the start of the RFY project.Typically, this was from the Fall Semester 2015, but for two courses, the class is not taught in the Fall and so we used data from the Spring 2016.The average grade in the redesigned classes was significantly higher than in the previous semester: t(910)=4.581,p<.001.DFW rates were one-third lower in the redesigned courses: χ 2 (1)=80.508, p<.001.The one-year retention rate was 10% points higher in the redesigned courses: χ 2 (1)=68.001, p<.001.One final observation can be reported: the effort has been accompanied by an increase in the campus-wide retention rate.The one-year, fall-to-fall, first-time freshman retention rate calculated by the university for reporting to the Integrated Post-secondary Education Data System (IPEDs) has averaged 65.6% over the five years prior to the start of this project (STD=1.74).This past year, the one-year, fall-to-fall, first-time freshman retention rate was 67.9%.This was not an all-time high retention rate, but at 2.3 percentage points higher than the previous year, was an unusually good one.Based on chance variations it would occur about once every 10 years (p=9.68%).

Discussion
The primary objectives of the study were demonstrated.1) Faculty members were able to use faculty communities of practice to acquire an understanding of an array of pedagogical techniques.Cumulatively, the faculty read, analyzed, and added information about many techniques that they did not possess before.2) Faculty members were able to introduce success-oriented pedagogies into gateway courses.Students experienced redesigned courses in Introductory Accounting, Biology, College Skills, Composition, Economics, Geology, History, Mathematics, Psychology, Sociology, and Statistics.3) The redesigned courses were associated with improved student learning.Many of the interventions were accompanied with efforts to track aspects of student success such as direct comparisons of answers on quizzes from students who had learning within a traditional classroom and from a redesigned classroom.Overall, students from the redesigned classrooms demonstrated more mastery of the course material.4) If students can learn more effectively, this should lead to improved grades within their classes, higher levels of student success, and a lower DFW rate.In fact, the redesigned courses were associated with better grades.The mean grade (as measured by a 4-point scale with 4 as an A) in those classes were more than a third of a grade higher than in those classes prior to the redesign.The DFW rate was significantly lower in the redesigned courses (24% versus 36%).5) Over a longer time horizon, the improved student learning could lead to increased retention and graduation rates.Over a one-year period, the retention rate for students who enrolled in the traditional-style classes was 68%.Over the same duration, the retention rate for students who enrolled in the redesigned courses was 78%.
One of the important aspects of these faculty communities of practice is their size and their freedom to explore.The average size of each PIG was 6-8 faculty, which provided ample discussion time by all participants as well as a variety of techniques explored (as each faculty member in the PIG selects a different technique to present to the group).Additionally, the members of the PIGs decide collectively which techniques to learn about/explore.They are therefore, inherently interested and invested in the process and outcomes, which supports previous findings by Wenger-Traynor (2011) that indicate significant success factors for communities of practice are self-governance and a sense of ownership.
Another positive attribute of the PIGs was the extent to which it afforded communication between faculty and administrators and between faculty members.One example is that the PIGs provided an effective way to transmit information about the values of the university as well as performance feedback.The efforts to improve student success are greatly enhanced by providing specific targeted feedback.For instance, our faculty are typically unaware of student success measures such as the DFW rate for their class or the retention rate of students enrolled in their classes.Our faculty have also been unaware of what sorts of techniques and strategies other faculty might use to address similar issues.The faculty did not have clear or consistent information as to the values and goals of the university such as whether the school is worried about grade inflation, or whether the school wants to use gateway courses as a de facto selection mechanism, for example.The PIGs served as a venue to discuss meaningfully these issues within small groups.This supports research by Wenger (2002) that suggests that communities of practice work well when the strategic goals and needs of the university intersect with the interests of the members.
The observed improvements in academic success are notable.However, the history of reform efforts in higher education suggests that improvements will not be sustainable without altering fundamental characteristics of the system (Elmore, 1996).In short, the academic, administrative, and incentive structures that enable and promote the features of the traditional classroom pedagogies are potent and persistent.To maintain the continuation and increased exploration of innovative curricula and pedagogy, campus administrative and faculty leadership needs to be engaged with the process and supportive of the goals.One example of academic, administrative, and incentive structure that can overwhelm a reform effort such as this one is the routine faculty evaluation systems.These can include promotion and tenure reviews, teaching award and course development grant criteria, performance evaluations, and salary reviews.Hainline (2010) suggests that linking measures of teaching effectiveness and recognition for innovative teaching are needed to sustain pedagogical transformations.
Scaling the innovations developed by the PIGs is imperative for student success on our campus.DeDe (2016) suggests that successful scaling requires depth (effectiveness), sustainability, spread, shift (decentralization of ownership) and evolution over time.The PIGs have demonstrated effectiveness of the techniques in our classrooms on our campus.They have sustained their efforts across multiple semesters so far.We are now working on the concepts of spread and shift.We are adding new PIGs every term.The number of faculty interested in being a member of a PIG is greater than our ability to add new PIGs at this point.One third of our full-time faculty who teach gateway courses are currently participating in PIGs and this certainly contributes to DeDe's concept of spread of practice.Additionally, we are piloting incentivized workshops for adjunct faculty to extend the products/techniques developed by faculty who participated in the PIGs to all faculty that teach the redesigned courses.These workshops will provide professional development opportunities as well as an opportunity for the adjunct faculty to contribute to creating the products used in the classrooms.
We believe that the processes that the PIGs have followed to date are well suited to our campus context.The successes are being expanded and sustained across terms as outlined above.We believe that these steps recognize that "designing an innovation for sustainability and scale is a multistage iterative process that involves teachers as co-evaluators and designers."And we hope that it will prevent "implementing a top-down intervention as a method of scaling up…(which) is a recipe for continued failure" (Dee Dee, 2016).
Hoyert and O'Dell Journal of the Scholarship of Teaching and Learning, Vol. 19, No. 1, February 2019.josotl.indiana.edu Hoyert and O'Dell Journal of the Scholarship of Teaching and Learning, Vol. 19, No. 1, February 2019.josotl.indiana.edu

Table 1 : Measures of Student Success
Hoyert and O'Dell Journal of the Scholarship of Teaching and Learning, Vol. 19, No. 1, February 2019.josotl.indiana.edu