Using IRB Protocols to Teach Ethical Principles for Research and Everyday Life: A High-Impact Practice

: Undergraduate research as a high-impact practice demonstrates many positive benefits for students, but little research has delved into the impact of ethical training for research, in particular submitting Institutional Review Board (IRB) protocols to determine if the study meets ethical standards for the treatment of human subjects. This study explored if students in two experimental and one nonexperimental research methods class benefited from increased knowledge of research ethics and how to apply them in daily-life situations if they participated in various aspects of IRB protocol procedures either as part of a class-based research project or by completing an IRB protocol activity for developing a hypothetical program to help families. Some students in all three classes had previously engaged in a 4-hr online extended training [the Collaborative Institutional Training Initiative (CITI) Program] in research ethics focused on the Belmont Report principles of beneficence, respect, and justice, but not in IRB protocols. Students were given a pre-and posttest to assess knowledge in both research and daily-life settings for applying the Belmont Report research ethics principles. Results indicate students gained greater knowledge of research ethics when they completed IRB protocol training during a class-based undergraduate research or program-design project, even if they had already completed some extended case-based training in the CITI Program. Results are discussed in terms of the value of using modified IRB protocol approaches as a high-impact practice to teach ethics in research and daily life to students.

High-impact practices (HIPs) generally refer to a group of 11 practices designed to increase student engagement in the learning process and involvement with faculty by promoting active, hands-on activities in a collaborative and mentored environment that lead to deep and extended dives into the material and skills to be learned (Kuh, 2008;Kuh, O'Donnell, & Schneider, 2017). They include activities such as learning communities, study abroad programs, senior capstone experiences, writingintensive courses, participation in undergraduate research, and others.
Increasingly, undergraduate research is being highlighted as an important opportunity for learning through an HIP (Kuh, 2008) by engaging students in faculty-led research and encouraging students to develop their own research projects in independent studies and senior capstone projects. This HIP can also involve introducing research into existing course instruction through course-based undergraduate research experiences (CUREs; Brownell & Kloser, 2015;Corwin, Graham, & Dolan, 2015;Segarra & Gomez, 2014;Teixeira-Poit, Cameron, & Schulman, 2011). Research on undergraduate research and CUREs has shown that students benefit from this mentored experience through increased knowledge and skill acquisition (Adedokun et al., 2014;John & Creighton, 2011;Stanford, Rocheleau, Smith, & Mohan, 2017), feelings of achievement in project ownership and selfefficacy (Adedokun et al., 2014;John & Creighton, 2011;Sandquist, Cervato, & Ogilvie, 2019), higher retention and graduation rates (Stanford et al., 2017), and greater commitment to the profession (Adedokun et al., 2014 ;Helm, & Bailey, 2013). Corwin et al. (2015) noted several benefits specifically of CUREs in their review of research on outcomes, including strong support for increased student Ritchie Journal of the Scholarship of Teaching and Learning, Vol. 21, No. 1, April 2021. josotl.indiana.edu perception in content knowledge, technical skills, and analytical abilities, in addition to some increased perceived improvement in collaboration and communication skills. There is also evidence that students were more likely to persist in science education and professions. Moreover, there is some evidence students think CUREs give them increased access to faculty as mentors and allies (Alkaher & Dolan, 2014;Kallgren & Tauber, 1996). Though much of this research focuses on research processes, lab skills, commitment to education, and critical thinking abilities, it can also impact students learning about responsible conduct in research and ethical treatment of participants.
However, even in the social sciences, the focus in undergraduate research is often on the overall research process rather than on how the research helps students learn about ethical treatment of subjects. Moreover, guidelines developed by the American Psychological Association (APA, 2013, p. 26) highlight the need to teach students not only how to "apply ethical standards to evaluate psychological science" but also how to develop "positive personal values" and "treat others with civility." Yet, most undergraduate psychology students receive superficial instruction in primarily research methods courses, focused mostly on a few cases of unethical research (Ruiz & Warchal, 2014). This instruction hardly leads to the deep understanding and change in values promoted by the APA guidelines, much less the highly integrative and mentored activities promoted by HIPs. In fact, instruction that is integrated throughout the curriculum (Ruiz & Warchal, 2014), uses extended complex case-based approaches (Watts, Medeiros, Mulhearn, Steele, Connelly, & Mumford, 2017) and uses critical thinking processes for teaching ethics (Kienzler, 2001) is needed and has been shown to be most effective in promoting knowledge of ethical and responsible research conduct.
CUREs are one way to give students this intensive HIP instruction that can lead to better critical thinking processes in research ethics (Kienzler, 2001;Olszewski, 2019). Particularly, I propose teaching students research ethics by having them actually follow the process used by scientists, such as preparing an Institutional Review Board (IRB) protocol for research with human subjects (Hubbard & Ritchie, 1995;Olszewski, 2019). IRB protocols require researchers to address each of the three principles from the Belmont Report on research ethics with humans by answering a series of questions to determine if research participants are being treated ethically. These include beneficence, which encourages researchers to consider how to maximize benefits while minimizing the harm of research participation; respect, which emphasizes the need to consider research participants as autonomous in the decision-making process and has led to guidelines for obtaining informed consent and avoiding coercion of participants; and justice, which holds researchers to fairness and lack of bias, focusing on protecting subjects from exploitation. Each of these principles requires critical thinking about how researchers can treat participants ethically while developing the study protocol and methodology. But undergraduate students have rarely been involved in this early stage of research development and thus have not benefited from the critical analysis and hands-on experience of preparing such protocols.
More instructors have been turning to these hands-on experiences, including writing and defending IRB-like protocols, when teaching research ethics (e.g., Danowitz, Brown, Jones, Diegelman-Parente, & Taylor, 2016;Diaz-Martinez, et al., 2019;Kallgren & Tabuer, 1996;Olimpo, Diaz-Martinez, Bhatt, & D'Arcy, 2017;Olszewski, 2019;Segarra & Gomez, 2014). The few existing studies on the benefits of using IRB-like procedures have shown that students reported an appreciation for ethical issues (Kallgren & Tabuer, 1996) and were more likely to address issues of informed consent and risks to participants in study design (Segarra & Gomez, 2014). Interestingly, Mabrouk (2016) found students who were involved in undergraduate research activities may tend to understand ethical concepts but not necessarily be able to apply them to their own research. In addition, Olimpo and colleagues (2017) in their review of ethics instruction reported on several studies showing few outcomes for ethical understanding due to the limited amount of time and depth given to research ethics instruction. On the other hand, using IRB protocols to teach research ethics is a Ritchie Journal of the Scholarship of Teaching and Learning, Vol. 21, No. 1, April 2021. josotl.indiana.edu mentored experience requiring greater depth than just a few hours of instruction. Olszewski (2019) argued that completing IRB protocols will help students learn not only ethical research conduct and professional skills, but also how to manage large projects. In addition, he pointed out that participating in the IRB review process develops the professional skills necessary for dealing with issues of data security, client confidentiality, and social justice as well as recognizing both risks and benefits when making decisions. The process of completing an IRB protocol under the tutelage of a faculty member would give students a more mentored and in-depth experience in learning professional ethical practices. Using IRB protocol development as a tool to teach research ethics would be consistent with the view that HIPs lead students to experiential, active, applied, and mentored learning environments that delve deeply into content and are extended over time (Kuh, 2008). Thus, involvement in actually making ethical decisions in an IRB protocol should increase not only the students' knowledge and understanding of research ethics, but also the ability to apply them beyond classroom environments and to their own research, and perhaps even to their daily life and future profession.
Though IRB procedures differ across institutions, they share certain criteria that make researchers, and thus students, think critically about beneficence, respect, and justice as they write in-depth procedures for their studies. These critical analyses and writing components are essential to HIPs (Kuh, 2008). These three principles also figured prominently in Hudson and Diaz Pearson's (2018) qualitative research into how college students think about morality. They noted that elements of respect, doing no harm, and justice were 3 of 10 themes identified as important to college students' moral identities. Thus, the IRB protocol should relate well to students' own moral perspective, making it easier for students to apply it to everyday moral decisions, which might eventually lead students to further develop their own ethical and moral standards in life. The IRB protocols at our institution for all research projects include both an online, in-depth exposure to classic studies and instruction on research ethics similar to those reported by Olimpo and colleagues (2017), as well as writing critical responses to multiple questions about the three Belmont Report research ethics principles (BREPs), addressing the topics of study procedures, risks and benefits, data security, informed consent, and others.
Another component of completing IRB protocols for CUREs is that they are grounded by a relationship with the course instructor, who then can serve as a mentor to help students better understand how the three BREPs apply to their own projects. In fact, there is some evidence suggesting students do receive greater mentorship through CUREs, leading to a greater appreciation of research ethics (Kallgren & Tauber, 1996). Students often work with the instructor as they make decisions, receive feedback, and make changes to their protocols.
This study arose from my normal assessment practices to determine teaching effectiveness in research methods classes. In these classes, I use various forms of IRB protocol development to give students that hands-on experience. I developed three hypotheses based on findings from using undergraduate research experiences generally, as well as those specific to learning ethics within a classroom environment using undergraduate research (e.g., Kallgren & Tauber, 1996;Olimpo et al., 2017). First, I expected students would learn from traditional methods of class discussion of core cases, but greater learning would occur from the more in-depth process of IRB training workshops and on-line modules. At our institution, this was completed using the Collaborative Institutional Training Initiative Program (CITI Program, 2019) an approximately 4-hr on-line training program. Second, I hypothesized that students would achieve even greater learning of ethical principles from IRB mentored experiences, especially if they had in-depth traditional training from the CITI Program previously. Last, I expected that students would gain a greater ability to understand how research ethical principles apply to daily-life issues after IRB protocol training.

Participants
Data were collected from students (n = 97) across two semesters from three different types of classes at a Midwestern regional campus, whose student body is mostly female, traditional-aged, Caucasian students. At this campus, 65% of the students are women, 28% are minorities, and 72% are under the age of 24. In addition, 79% of the undergraduate students are full-time, but only 8% of students live on campus. The majority of students graduated from high schools close by.
The first type of class was a sophomore-level course designed to introduce students to basic experimental methods (EXP) and APA format for writing manuscripts. Three to four students complete an extensive group project designed to teach the mechanics of research in psychology, including ethical procedures. As part of the group project, students collectively complete an IRB protocol that is not submitted to an IRB committee but is closely mentored by the instructor. Over two semesters, 31 students took this class, only two of whom had completed previous CITI training. The second type of class focused on teaching nonexperimental methods of psychological and professional research (NONEXP) to mostly advanced students roughly 1 year from graduation. There were 35 students from two semesters, and a majority had completed CITI training (n = 26), either to work in a professor's research lab or in an earlier class. In this class, students completed data collection for a qualitative interview project on how family spaces (e.g., family and living rooms) impact family cohesion. As part of this project, students completed training on an IRB-approved protocol closely mentored by the instructor, typical but much more in-depth compared to how students would normally be trained as research assistants in a professor's lab. Groups of students used the existing IRB protocol to answer questions about the three Belmont Report principles and how the study safeguards the rights of participants under each principle. Last, one section of an upper division interdisciplinary psychology course in marriage and family (MF), which completed group projects to design a hypothetical program to serve at-risk families, was included for comparison (n = 31); six of these students had completed previous CITI training. These students were required to use questions similar to those of an IRB protocol to demonstrate the ethical treatment of potential clients.

Materials
Testing materials. Materials included a pre-and posttest as well as an in-class exam. The pre-and posttest consisted of the same 10 multiple-choice questions for measuring knowledge of the BREPs but were administered without announcement a little over 3 months apart. Some questions required students to distinguish between the definitions of the principles, but most required the student to think critically while applying the principles to hypothetical research scenarios. These questions changed between the classes. Students received one point for each correct answer. All versions of the testing materials had at least three questions each for justice and beneficence but had four questions for respect. An example of a question assessing knowledge of respect for research ethics is "Hayden carefully administers informed consent of his subjects because he wants to make sure they are making an autonomous decision about whether to participate or not. He is observing which principle of the Belmont Report?" Students then chose between beneficence, justice, or respect as the answer. An example of a question about beneficence is "T'rshelle completes a section of an IRB protocol which asks her to name all the risks to her subjects as well as any direct rewards they might receive from participating in the research. She is working on this ethical principle." The in-class exams were similar in nature but often did not have 10 questions, and thus these scores were reconfigured to a 10-point scale. Usually, there were six to eight questions on these tests. The questions concerning applications of the BREP to daily life were also similar, though there were four questions for respect, four for justice, and three for beneficence, resulting in a total of 11 questions. Like the in-class exams, the daily-life answers were rescaled to 10 points. An example of a justice question from the daily life-scale is "A wise boss tries to make decisions based on what is best for all his or her employees rather than playing favorites. Bosses who show favoritism at the cost of others are breaking this basic principle." Modified IRB protocol. The IRB protocol for the NONEXP students was the one already approved by the IRB, but training students on the approved protocol was achieved by having students answer questions in small groups of three or four students, similar to the modified IRB protocol used by EXP students for their group projects, while the instructor circulated among these groups. The modified protocol for MF students was similar except the questions related to design, implementation, and safeguarding data gathered for a program rather than a research study. EXP students were placed in groups of three or four students to design their research projects and complete the modified IRB protocol across two class sessions. This protocol began by having students write a brief description of their proposed study purpose and procedures. Then, students listed potential risks to participants and methods they would use to minimize these risks. Students also listed potential benefits for participants. These questions focused on the beneficence principle. Students also answered questions about participant recruitment. Since most students were going to use friends and family, they were required to justify this procedure and how they would safeguard the confidentiality of the participant as well as how they would minimize potential coercion to do the research. Students wrote out a recruitment script and prepared the informed consent form that they planned to use. These questions mostly focused on the respect principle. One question specifically required students to address the potential vulnerability of their participants to being exploited since they often used friends and family. This addressed issues of justice. Students also answered questions concerning how they would manage the data collected and maintain the confidentiality of their participants. These protocols were submitted to the instructor, who made comments that students then responded to as a group in the next class meeting. NONEXP students answered questions about the existing IRB-approved protocol in small groups focused on how they would specifically complete their part of the research project. Last, the modified IRB protocol for the MF students was similar, though participant recruitment did not focus on family and friends but rather on community agencies. The protocol also still focused on how students would collect and safeguard data from program participants.

Instructional Procedures
In each class, I began with a knowledge pretest to assess prior exposure to the BREPs; the methodology classes (EXP and NONEXP) also answered questions about how to apply these principles to everyday moral issues. This pretest was administered within the first week of class prior to research ethics instruction. Then, I taught students using a typical approach of introducing the three principles from the Belmont Report and how they were developed from standard examples of problematic studies, many of which were discussed in detail in their textbooks. Key studies discussed include the Milgram Study and the Tuskegee Institute Study in terms of how they gave rise to the three principles. This information was usually covered in one class period, approximately in the second or third week. Students were then tested on knowledge of these principles for research, but not everyday moral issues, in standard in-class exams using multiple-choice questions. Then, I showed students how these principles are addressed in IRB protocols, approximately in the fifth week of class. EXP students Ritchie Journal of the Scholarship of Teaching and Learning, Vol. 21, No. 1, April 2021. josotl.indiana.edu wrote parts of a protocol for a group research project on various topics, while NONEXP students discussed in detail an existing IRB protocol concerning a qualitative study on how families use family spaces such as the family room to enhance family cohesion and bonding, which the students implemented as part of the class. MF students, also in small groups, completed the modified IRB concerning development of a program for at-risk families. These procedures took at least two class periods. Students in the methods classes then completed the research from approximately Week 5 to Week 12. MF students presented their projects to the class near the end of the term. Last, students were given a posttest at the end of the semester (the 16 th week) to retest their knowledge of both the BREPs and their application to everyday moral issues.

Archival Research Procedures
Data gathered from students were used as a classroom assessment to increase the effectiveness of my instruction. Aggregate assessment results across the entire class were then shared with students via a class management software (Canvas) announcement only after I had completed the term and turned in their grades. At that point, their names and identifying information had already been stripped from the class data file to complete the aggregate assessment. These assessment results were used to make changes to the class and reported in my annual evaluations. About a year later, I had an exempt IRB protocol approved to combine these class de-identified data sets into one data set to analyze as an archival database to answer the research question of whether these IRB protocol procedures increased student knowledge of ethical research principles. These analyses were presented at a conference for educators (Ritchie, 2015) approximately 4 years ago.

Results
Analyses were conducted using three different repeated measures analyses of variance (ANOVAs), which I discuss in conjunction with the three hypotheses.

Does Greater Learning Occur After In-Depth Traditional Training?
The first hypothesis was that traditional methods of instruction would be related to greater understanding of the BREPs, but especially when that instruction was in greater depth congruent with the training necessary for submission of IRB protocols through CITI training. BREP test results for 97 students were submitted to a 2 (test times) × 2 (CITI training [with or without]) × 3 (courses) repeated measures ANOVA. For research knowledge questions, a significant effect was found for test time, F(1,91) = 30.04, p < .001. As shown in Figure 1, over all three classes, students showed improvement from the pretest (M = 4.36, SD = 2.14) to the posttest (M = 6.25, SD = 2.35). In addition, those with CITI training scored better across the two tests compared to those without CITI training regardless of which class they were in, F(1,91) = 8.68, p = .004. No other significant effects were found.

Does Greater Learning Occur with IRB Mentored Activities?
For one section of the EXP class, data were available not only from the pre-and posttest but also from an in-class test that followed traditional instruction but was given before IRB training and protocol development began. Fourteen students participated in this class and their data were submitted to a repeated measures one-way ANOVA on training type (pre-IRB-training, traditional instruction, and post-IRB-training), followed by post hoc paired samples t tests. A significant effect of training type was found, F(2,26) = 4.66, p = .019. Post hoc t tests indicated that post-IRB-training (M = 5.59, SD = 1.74) differed significantly from pre-IRB-training (M = 3.41, SD = 1.27), t(13) = -3.86, p = .002, and tended to differ from traditional instruction (M = 4.04, SD = 2.57), t(13) = -1.95, p = .073. There was no difference between pre-IRB-training and traditional instruction, t(13) = -.77, p = .455.

Are These Learning Effects Transferable to Everyday Ethical Issues?
Two research methods sections (EXP: n = 22; NONEXP: n = 14) also completed a measurement on applying the BREPs to everyday-life decisions as both a pretest and a posttest. Data were submitted to a 2 (test times) × 2 (courses) repeated measures ANOVA and resulted in a significant difference for test time, F(1,34) = 16.46, p < .001. Both sections scored better on applying the BREPs to everyday-life decisions after IRB training (M = 5.53, SD = 2.28) than in the pretest (M = 6.95, SD = 2.19). There was no main effect of course.

Discussion
The data demonstrate that a modified IRB protocol experience within a CURE is associated with increased understanding of the three BREPs and how to apply them to everyday-life experiences. This result is consistent with previous findings showing that such experiences were associated with greater appreciation of ethical research principles (Kallgren & Tauber, 1996) and issues of informed consent and participant risks (Segarra & Gomez, 2014). The current investigation demonstrated that a mentored IRB-like activity was related to increased knowledge of ethical research issues as proposed by Hubbard and Ritchie (1995) as well as Olszewski (2019). In addition, the modified IRB protocol procedure for program development was also related to increased knowledge of ethical principles for a nonmethods MF class. There are many reasons why using IRB protocols would yield these results. First, undergraduate research experiences have been related to greater discipline-specific knowledge generally (e.g., John & Creighton, 2011;Stanford et al., 2017), and responsible conduct and research ethics is one such area of discipline-specific knowledge. Thus, general benefits of increased disciplinespecific knowledge of undergraduate research might yield increased understanding of the BREPs. Yet, given that HIPs also show positive benefits for undergraduates in skills and knowledge, it could be the results are an effect of such practices rather than the use of the modified IRB protocols. Moreover, because a highly mentored procedure was used with faculty feedback giving rise to reflective changes in study design, these results could have been due to increases in faculty input and engagement on the student project, typical in such mentor relationships. Still, the modified IRB protocol follows procedures that may encourage greater knowledge and mentorship specifically because of the nature of the process itself. Students engage in active, experiential learning in which they must clearly communicate through writing a professional document justifying their decisions based on applying the BREPs. They then must consider adjustments to their study in light of specific feedback given by the instructor in a process very similar not only to IRB procedures for research but to practices in professional settings as well. This procedure, unlike traditional methods that focus on teaching what not to do by analyzing ethically problematic studies, focuses on teaching students what to do using a positive framework for critical analysis of ethical decisions. In addition, this procedure encourages the extended complex case-based ethical instruction that leads to better knowledge of research ethics (Watts, et al., 2017).
Though not in this study, faculty mentorship through an IRB-like protocol should improve not only knowledge of ethical principles but also the quality of the student research projects. Future research should investigate if the use of such protocols leads to better student projects, as has been suggested by research on CUREs. For example, Corwin et al. (2015) found benefits of such coursebased research projects for student perceptions on technical skills, analytical abilities, teamwork, and communication. Conceivably, these perceived benefits of undergraduate research would also include using IRB protocols to sharpen the project and ensure ethical procedures, leading to higher quality projects and better student outcomes. Furthermore, this mentored experience also corresponds to the types of professional communications and processes that students will encounter in their future careers. Engaging students in an IRB-like process to develop projects in the MF class did increase ethical knowledge of the BREPs. Since not all areas of research and many professions do not use IRB processes, modifying the process used in the MF class to include other ethical principles could lead not only to greater ethical knowledge but perhaps also to ethical professional behavior generally and better student projects overall.
There are, of course, limitations to these findings. The current study was quasiexperimental; it was not possible to randomize either which students would receive the modified IRB training or in what order that training would be given. Not only must caution be taken in concluding a causal relation between the modified IRB training and increased student knowledge, but students could have simply improved because they were getting increased instruction, regardless of the type of instruction. In fact, the pre-and posttest, though separated by 3 months, contained the same questions. Though students were unaware that there would be a posttest, previous exposure could have led to increased scores. Last, because the investigation was completed within a classroom environment, student motivation to excel could have impacted the results, much like a placebo effect. Moreover, the investigation did not track students over a long period of time. It could be that the increased knowledge fades over time rather than having a lasting impact. Nor is it possible to know from the results of applying the BREPs to everyday-life experiences if the increased knowledge will have an impact on students' actual ethical and moral behavior. Further research should extend beyond one class term to explore if the knowledge persists and affects actual behaviors. In fact, investigations of how these research experiences impact future professional behaviors would be beneficial.
There are also some practical issues of note. First, individual IRBs may differ in their procedures and interpretations of what constitutes research. Thus, many projects led by undergraduate students within courses might be deemed "not research" because they would not add to greater knowledge within the field; this is the case at my own institution. In such cases, instructors may need to think in more creative ways to introduce IRB-like procedures into instruction (see Olszewski, 2019), including the use of modified proposals, classroom review panels, blind peer reviews of proposals by faculty and other students, and others tactics. Moreover, a modified IRB protocol can be easily adapted for use in other, nonmethodology classes in which students propose programs or projects that still collect data even though the projects are not specifically research for basic knowledge but rather for internal use within a program or agency, as was the case for the MF class in this investigation. In addition, the IRB process could provoke frustration and stress in students who are on a timetable and have not experienced such a sometimes legalistic, negotiated process of decision making and professional correspondence. However, these types of decision-making processes are common in professional settings, and using the IRB process to mentor students may lead to students' better understanding of working with teams and other professional groups, giving them insight into how to effectively navigate professional dilemmas as well as a way to sharpen their professional demeanor and communication skills. Future research should explore if IRB-like procedures improve teamwork and communication.
Though there are some limitations to causal pathways as well as practical issues to consider, the results demonstrate that student knowledge increased with both in-depth traditional case-based instruction on the BREPs and then showed growth in knowledge after a modified IRB protocol training. Further research should explore how this knowledge might persist into professional settings both in terms of applying the BREPs and if this increased knowledge is associated with more professional ethical conduct and behavior as well as teamwork skills. These findings, however, do suggest that using the modified IRB protocol procedure described in this investigation and perhaps those suggested by others might be beneficial to the ethical conduct of research and perhaps students' future professional lives.