Can Undergraduate Research Participation Reduce the Equity Gap?

: Undergraduate research (UGR), one of several high-impact practices (HIPs) in education, can positively impact student retention and graduation rates. However, not all students take advantage of UGR opportunities, with fewer students from underrepresented minority groups, those with first-generation status, and students eligible for a Pell grant or federally subsidized loan. We obtained retention and graduation data from our Office of Institutional Research and Planning for all UGR participants for academic years 2009–2010 to 2016–2017. We specifically focused on data for UGR participants from underrepresented demographics that historically have lower retention and graduation rates than those of the overall student body. We created Sankey-like ribbon diagrams to analyze the characteristics of UGR participants, whether they participated in UGR for 1 year or longer, their class standings when they started UGR, retention rates for 1st-and 2nd-year students for the year following their UGR participation, and graduation rates for all participants. Our data show that irrespective of demographics, students who participated in UGR were significantly more likely to persist in college and graduate within 6 years compared to students who did not. Persistence and success in college may depend on students’ socioeconomic status, sense of belonging, and other factors. Assessing the impact of a single HIP, such as UGR, on retention and graduation rates, can, therefore, be complicated. However, our study indicates that UGR participation can significantly improve persistence and success for students traditionally considered “at-risk,” irrespective of their socioeconomic status, family background, or class standing.. This information can be important for campus leaders and other stakeholders interested in facilitating student success and reducing the equity gap by incorporating UGR in more students’ college experiences. We describe our analytical methods and discusses our findings. We also demonstrate the effectiveness of Sankey-like diagrams for visualizing and analyzing large programmatic data sets, and as a tool for communicating program impacts to a general audience.


Introduction
Undergraduate Research as a High-Impact Practice Kuh (2008) identified 10 (later modified to 11, Kuh, O'Donnell, & Schneider, 2017, Figure 1) highimpact practices (HIPs) in education that can promote student engagement, academic achievement, persistence, satisfaction, and attainment of desired academic outcomes.The effects of HIPs have been shown to be more pronounced on students from underserved backgrounds (e.g., Kuh Participants in the traditional URP are expected to develop their research proposals themselves with help and guidance from their mentors, as opposed to mostly acting as research assistants for their mentors.These participants can apply for research credits to count toward their academic degree program but are not paid for their work unless it is done during the summer months.They are also expected to present their research to a general audience on campus and at appropriate off-campus venues.In contrast, RAP participants are not expected to develop their own research projects but are paid to assist with the research agenda of their mentors.They are encouraged to present their work but are not required to do so.Both traditional URP and RAP students are expected to participate in research for two academic semesters after being accepted in the chosen program. The framework for our campus URP, therefore, allows us to offer several key HIPs, by giving students the opportunity to engage in exploring contested questions outside the classroom, participate in one-on-one mentorship, receive frequent constructive feedback, and publicly demonstrate their competence.The differences in the characteristics of students participating in the traditional URP and RAP are summarized in Table 1.A member of an underrepresented minority (URM) is defined on campus as a student who indicates a race/ethnicity of African American/Black, American Indian, Hispanic/Latino or Latina, or Southeast Asian, either alone or in combination with other races/ethnicities.Table 1 shows that the RAP has been more successful than the traditional URP in recruiting an ethnically diverse student population, with students from all URM groups being overrepresented, and also in engaging students in research early in their college career.Besides students from URM groups, a significant proportion (almost 41%) of the UW-W student body also identified as a first-generation student (defined as an undergraduate student whose parents have not earned a 4-year college/university degree), and/or 1.At what stage(s) of their college career do students from different demographics enroll in the traditional URP and/or RAP? 2. How do the 6-year graduation rates for students from underserved groups participating in UGR compare to the 6-year graduation rates for students from those specific groups in the overall student body? 3. How do the 6-year graduation rates for all students participating in UGR for 1 year and 2-or more years compare with the 6-year graduation rate for the overall student body? 4. What is the retention rate for students from different demographics after completing a year of UGR in their 1st or 2nd year of college compared to the 2nd-and 3rd-year retention rate for students from those demographics in the overall student body?

Method
Traditionally, Sankey diagrams are used in disciplines such as engineering and supply chain management, among others, to visualize energy and/or material flows (Sankey, 1898;Schmidt, 2008aSchmidt, , 2008b)).In these diagrams, the thickness of the flows indicates the relative proportions of the material flowing between categories.For this project, we used our URP/RAP participant data set to create Sankey-like diagrams using the online Ribbon tool developed by researchers at the University of California, Davis (2018;Molinaro, Steinwachs, Li, & Guzman-Alvarez, 2017).This tool has been used to visualize how students select or leave academic majors throughout their college career (e.g., Bradforth et al., 2015).We used these diagrams to visualize and analyze URP/RAP participation trends for different student populations during their college career (Figure 1).The two most important aspects of these types of diagrams are the segmented columns, labeled "student status" in Figure 1 but referred to as "time steps" by the creators of the Ribbon tool (Molinaro et al., 2017;University of California, Davis, 2018), and the "flows" joining different segments of those columns.The width of the flows is proportional to the number of elements (in this case, number of student participants) in the flow.Conventionally, the time steps represent academic years or semesters spanning the data set.However, for our project the time steps represented different pieces of student information, such as financial aid status, first-generation status, and class standing at the time students started participating in UGR, among others.The segments of each column are called "nodes" in traditional Sankey diagrams.On Ribbon diagrams they are referred to as "groups."Groups or nodes are zones where the flows originate, end, converge, or diverge.There has to be a minimum of two groups at either end of a Ribbon diagram.The width of a group is proportional to the number of students in that group.
The web-based Ribbon tool allows users to obtain the number of students in each group and the relative percentages of different student groups distributed across the studied population.This capacity allowed us to analyze and synthesize our data set and conveniently extract relevant information for different student demographics participating in UGR for this project.Examples of information obtained from the Ribbon tool are shown as text boxes on the Ribbon diagram (Figure 1).
We reformatted and reorganized the URP/RAP participant data set obtained from the UW-W Office of Institutional Research and Planning to be compatible with the Ribbon tool.For our analyses, each participant was identified by a unique computer-generated seven-digit identification number (CID).We defined the student status and related groups corresponding to each CID as shown in Table 2.

Results
In this section we briefly describe the information we gathered from the Ribbon tool regarding the overall patterns of UGR experiences for the following student demographics: (a) transfer students, (b) first-generation students, (c) students belonging to a URM group, (d) students receiving a Pell grant, and (e) students receiving a federally subsidized loan.Students belonging to these groups are traditionally considered "academically at-risk," and we focused our analyses on exploring how participating in the traditional URP or RAP might affect academic success for them.We used data for all students (N = 1,049) who participated in UGR during 2011-2017 (in either the traditional URP or RAP) unless otherwise specified.We have broken down the results according to the different constituent parts of our research question, listed above.How do the 6-year graduation rates for students from underserved groups participating in UGR compare to the 6-year graduation rates for students from those specific groups in the overall student body?How do the 6-year graduation rates for all students participating in UGR for 1 year and 2 or more years compare with the 6-year graduation rate for the overall student body?

At what stage(s) of their college career do students from different demographics enroll in the traditional
We calculated the 6-year graduation rates for UGR participants broken down for different student demographics using the Ribbon tool.For a comparison we calculated 5-year weighted averages of 6year graduation data for corresponding student demographics spanning academic years 2006-2007 to 2010-2011 for the student body using data provided by the UW-W Office of Institutional Research and Planning.We conducted the same analyses on 6-year graduation rates for all first-time students, full-time students, and those with 1 year and 2 or more years of UGR experience, irrespective of their demographics or socioeconomic status, and compared these with the 5-year weighted average of the 6-year graduation rate for the overall student body.
We conducted chi-square and Fisher's exact tests to see if students participating in UGR were statistically significantly more likely to graduate from college within 6 years than their peers not participating in UGR.The Fisher's exact test is more conservative than the chi-square test, and any significant difference between groups can be considered meaningful.The results are shown in Table 3.The 6-year graduation rates of students participating in UGR were statistically significantly higher than those of corresponding student populations in the overall student body.What is the retention rate for students from different demographics after completing 1 year of UGR in their 1st or 2nd year of college compared to the 2nd-and 3rd-year retention rates for students from those demographics in the overall student body?
The 2nd-and 3rd-year retention rates for students from the specified demographics are provided in Tables 4 and 5, respectively.The 2nd-year retention rates (Table 4) were calculated on 176 students (152 students participating in the RAP and 24 in the traditional URP) who joined the traditional URP or RAP as 1st-year students.Statistical analyses show that the 2nd-year retention rate of students participating in UGR was statistically significantly higher at the 95% confidence level than that of students from the same demographics in the overall student body, except for students receiving a federally subsidized loan.The 3rd-year retention rates (Table 5) were calculated on 239 students (135 students participating in the RAP and 104 in the traditional URP) during their sophomore year.Statistical analyses show that the 3rd-year retention rate of students participating in UGR was statistically significantly higher than that of students from the same demographics in the overall student body.These numbers exclude students who joined the program during the 2016-2017 academic year.Since very few transfer students joined the traditional URP or RAP as first-years and sophomores, Tables 4 and 5 do not include separate retention rates for transfer students.Instead, we compared 2nd-year retention data for all first-time, full-time students and 3rd-year retention data for all sophomores participating in UGR to the corresponding overall student body retention rates (5year weighted average).Retention data for specific student demographics from the overall student body were obtained from the UW-W Office of Institutional Research and Planning.We conducted chi-square and Fisher's exact tests to determine if students participating in the traditional URP or RAP during their 1st or 2nd year of college were statistically significantly more likely to be retained in the academic year following their UGR participation.These results for 1st-and 2nd-year students are also shown in Tables 4 and 5, respectively.

Discussion
Mentored UGR is well integrated in our campus culture.UGR participants can engage in mentored research for one or more years as an extra-or cocurricular activity.They receive one-on-one mentoring, opportunity to design and take ownership of their own projects (mostly traditional URP participants), or help faculty mentors or upper-level students with research (RAP participants).Traditional URP participants are also expected to present their research on campus during spring and/or fall Undergraduate Research Days and off campus at the National Conference on Undergraduate Research or statewide symposia for scholarly and creative activities.While a presentation is not expected of RAP participants, they are strongly encouraged to participate during on-campus UGR celebration events.UGR participants also have the opportunity to apply to the UW-W Summer Undergraduate Research Fellowship program, a highly competitive 10-week summer research program.
Being engaged in research has various tangible and intangible benefits for students.They learn to effectively communicate orally and in writing, analyze and synthesize their data to make valid conclusions, critically evaluate concepts from different perspectives, and other valuable skills.A survey by Hart Research Associates (2015) found that employers strongly endorsed an emphasis on applied learning in college and believed that working on applied learning projects would prepare students better for a career after graduation and improve their chances of being hired.Student researchers also learn how to work as part of a team, be respectful of others, receive constructive criticism, and deal with setbacks.Being part of a group/research team, hands-on experience, and being engaged in solving a real-world problem are among the benefits of mentored research, all of which can increase students' sense of belonging and provide a support network for them to persist in college (e.g., Strayhorn, 2019).
Both traditional URP and RAP students work closely with faculty and staff mentors and in many cases, also have upper-level students acting as near-peer mentors.We surveyed RAP students (Institutional Review Board Protocol Number: B14509018Q) in academic years 2014-2015 and 2016-2017 at the beginning of their research experience, after one semester of conducting research, and again at the end of their one academic year of RAP experience to gauge the progressive change in their self-perceptions of skills and knowledge gain (Bhattacharyya, Chan, & Waraczynski, 2018).We also gathered information on what beginning students identified as benefits of research besides learning to do research.Our results show that 28.1% of responders (68 participants over three cohorts) identified "network and support" from faculty and staff mentors and peers as one of the major benefits of conducting mentored research.
Positive impacts of mentoring on undergraduate STEM students, especially on those from minority backgrounds have been documented (e.g., Haeger & Fresquez, 2016).Students from traditionally underrepresented groups in STEM fields who participated in mentored UGR showed Journal of the Scholarship of Teaching and Learning, Vol.21, No. 1, April 2021.josotl.indiana.edusignificantly higher cumulative GPAs and similar graduation rates to those of matched peers.Furthermore, students participating in UGR for an extended period of time (longer than one academic semester or one summer) showed significantly higher gains in research skills and level of research independence.Our data show that students participating in UGR as part of either the traditional URP or the RAP were statistically significantly more likely at a 95% confidence level to be retained during the academic year immediately following their UGR experience, as well as graduate from college within 6 years, irrespective of their background or socioeconomic status, including students traditionally considered academically at-risk.On our campus, considerable equity gaps remain in retention and graduation rates between students from underserved demographics and majority students.UGR participation can potentially be a way to reduce this gap.This is especially relevant given the changing student demographics at UW-W. Figure 2 shows changes in the URM student population at UW-W over the last 10 years (2009)(2010)(2011)(2012)(2013)(2014)(2015)(2016)(2017)(2018)(2019).The data show an increase in the overall URM population from 9.2% in the 2009-2010 academic year to 14.5% in 2018-2019.Over the same time span, our Hispanic/Latino or Latina student population has increased from 2.7% to 7.40% of the overall student population.Increasing UGR participation for these students can be a way to help all students succeed.We should not, however, assume that UGR participation is the sole driver of student success.Students who self-select to participate in UGR are also more likely to participate in other HIPs, such as internships.Most traditional URP participants joined in their junior or senior year (  2009-10 2010-11 2011-12 2012-13 2013-14 2014-15 2015-16 2016-17 2017-18 2018-19 Percentage of Overall Student Body Academic Year URM Hispanic/Latino or Latina African-American/Black Two or more races therefore, they had already successfully navigated their way through their first 2 years of college when students are most at risk of dropping out.They also met the minimum GPA requirement for the traditional URP, indicating that they were academically in good standing and therefore more likely to graduate within 6 years.Traditional URP students were also more likely to have made a connection with a faculty mentor in their chosen discipline and to have developed an identity as a scholar and researcher. In contrast, RAP recruits beginning students who have not yet made a connection to campus or their discipline and pairs them with research mentors as paid research assistants to help faculty with their research agenda.While the data on student retention (Tables 4 and 5) demonstrate the impact of UGR participation on 2nd-and 3rd-year retention of beginning students, especially those traditionally considered to be academically at-risk, the RAP program is still relatively new, and we do not yet have the data to measure its sustained impact on student retention and graduation rates.Also, for logistical reasons, only a limited number of students can participate in RAP in any given semester, and therefore, more work still needs to be done to broaden UGR participation for beginning students.
Nonetheless, our data on the positive effects of UGR participation on students' academic outcomes, even if incomplete, are promising.Currently, many colleges and universities are working on strategies to boost retention and graduation rates of their students as part of their moral obligation as institutions of higher education.This undertaking also has financial consequences for public institutions in states with performance-based funding models, whereby allocation of state funding to public colleges and universities is at least partially dependent on student outcomes.UGR participation can be implemented as part of a comprehensive student success program to increase student persistence and degree completion rates, especially for those from underserved backgrounds.Broadening UGR participation obviously requires resources but it can also be considered an investment in improving student outcomes that can also yield financial returns.

Conclusion
Our data show that mentored UGR conducted over one or more academic years, where students are allowed to design and conduct their own research projects with help and guidance from faculty/staff mentors, can positively impact persistence and 6-year graduation rates for students, especially for those from demographics traditionally considered academically at-risk.Beginning students helping mentors with their research agendas are also significantly more likely to persist in college than their counterparts not participating in research.While UGR cannot be identified as the sole factor driving student persistence and success, it can provide essential tangible academic skills and intangible benefits, such as a sense of belonging to the discipline and the university, a support network of fellow researchers, one-on-one mentoring from faculty/staff, and the self-efficacy necessary for academic success, and ultimately it can reduce the equity gap.

Figure 1 .
Figure 1.Example of a visualization diagram created using the Ribbon tool.The diagram displays data for students who joined the traditional Undergraduate Research Program (URP) or the Research Apprenticeship Program (RAP) at the University of Wisconsin-Whitewater (UWW) within their 1st year of study; data collected 2011-2017 (N = 222).

Table 3 . Statistical analyses of 6-year graduation rates of students participating in UGR and graduation rates for corresponding student populations in the overall student body, academic years 2006-2007 to 2010-2011.
Journal of the Scholarship of Teaching and Learning, Vol.21, No. 1, April 2021.josotl.indiana.edu

Table 4 . Second-year retention rates of students participating in UGR versus the overall student body for selected demographics.
Note.RAP = Research Apprenticeship Program; UGR = undergraduate research; URM = underrepresented minority; URP = Undergraduate Research Program. a Significant at p < .05. b 0.1> p >0.05.

Table 5 . Third-year retention rates retention rates of students participating in UGR versus the overall student body for selected demographics.
Journal of the Scholarship of Teaching and Learning, Vol.21, No. 1, April 2021.josotl.indiana.edu