Students’ Sense of Belonging: The Development of a Predictive Retention Model

: Educational institutions increasingly recognize the role that student belonging plays in retention. Many studies in this area focus on helping students improve a sense of belonging before they matriculate or identifying belonging as a reason for their departure. This study measures students’ sense of belonging at key transition points during the first year and finds that social belonging and academic performance are both strong predictors of retention that are not necessarily correlated. These results suggest that a comprehensive, focused outreach protocol that encompasses both social and academic factors could have a positive impact on student persistence.

Soon after St. Cloud State University joined AASCU's Reimagining the First Year initiative, which encourages institutions to think and act boldly when promoting the success of new incoming students, we turned our focus to belonging, which we recognized as the foundation for other strategies we planned to implement to improve students' experience.For more than forty years, researchers have recognized the critical role that social and academic integration play in students' decisions to remain in college and persist through to graduation (Tinto, 1975;Tinto, 1993;Tinto, 1997;Berger & Braxton, 1998).Hurtado and Carter's (1997) sense of belonging measure focused on students' attachment to the campus community as a whole while other researchers focused on attachment to various external communities or other university contexts (Hoffman, Richmond, Morrow, & Salomone, 2002;Kember & Leung, 2004;Lee & Davis, 2000).Zea, Reisen, Beil and Kaplan (1997) showed that both academic and social integration experiences impacted student persistence in college.In a study of 512 first-year students, Beil et al. (1999) found that academic and social integration predicted students' institutional commitments, which in turn influenced their persistence in college after three years.Researchers also have identified distinctions in the way that a sense of belonging to a campus community can be promoted for members of different student populations, including first-generation students (Woosley & Shepler, 2011;Stephens, Hamedani, & Destin, 2014) and students of color (Hurtado & Carter, 1997;Steele 1997;Lee & Davis 2000;Lane 2016).More recently, Jorgenson, Farrell, Fudge, and Pritchard (2018) have shown the importance of engaging students in defining what holistic social connectedness looks like on campus.Interventions such as orientation experiences, first-year seminar courses, mentoring, and promoting more intentional engagement with campus activities have all been shown to improve students' sense of belonging, and also their persistence.
The work of Walton andCohen (2007, 2011), Walton andYeager (2011), andYeager et al. (2016) demonstrates that it is possible to improve students' sense of belonging before they matriculate, and that a short, online activity designed to normalize students' concerns about making the transition to university had a long-lasting, measurable impact on a variety of success measures, including retention.This impact was especially noticeable among students of color and first-generation college students, who represent a key focus of the Reimagining the First Year project.
These and other studies clearly demonstrate the association between belonging and student persistence and that intervention can have a positive impact.What we still hoped to find, however, was a reliable way to measure students' sense of belonging so that we could identify at-risk students and engage them in more focused, individualized outreach designed to improve their chances of persisting to Terms Two and Three.This measurement would need to begin early in students' first term, when they are in the process of deciding whether to stay or leave (Levitz & Noel, 1989;Woosley, 2003;Woosley & Miller, 2009).By providing this information to faculty and staff members, we would be able to conduct timely, focused, and meaningful outreach that could have an impact on a student's decision to remain enrolled.
Fortunately, our campus community was already comfortable using predictive measures to guide interventions.We currently equip advisors of first-year students with information derived from two predictive tools -the Quality Points Predicted (QPP) score, which forecasts students' GPA at the end of the first term, and a retention index -to guide and inform their outreach.Each of these measures, however, is based primarily on pre-matriculation academic factors, such as ACT score and high-school GPA, along with various demographic factors.While these metrics serve as useful tools for identifying at-risk students, we knew that they did not reveal the whole story.We would consistently lose significant numbers of students whose academic indicators suggested a high probability of retention at the same time we were retaining students whom our existing models suggested would be likely to leave.Since we know that belonging plays a critical role in student persistence, we set out to establish a new predictive index to supplement our existing ones, a model that would allow us to measure first-year students' sense of belonging early in their first term and therefore help us make struggling students more visible to people who could help (Berger & Braxton, 1998).We were encouraged by a study conducted by Woosley and Miller (2009) that demonstrated early assessment was indeed predictive of real challenges to persistence and not "merely temporary struggles during a normal college transition."Krumrei-Mancuso, Newton, Kim, & Wilcox (2013) have also recognized the predictive relevance of psychosocial factors that include involvement with college activity.
The tool we developed, which we refer to as a Sense of Belonging Index, adds a critical layer to our existing predictive modeling strategies to provide a more complex and complete picture of students' experience on campus.In addition to allowing us to identify and intervene with students who were struggling with a low sense of belonging, this index, when collected at key transition points, can also be used to measure the effectiveness of other student success initiatives and suggest ways to improve them as needed.Through that improvement, we hope to see increases in both student retention and completion rates at St. Cloud State University, particularly for students from traditionally under-represented backgrounds.

Developing a Sense of Belonging Index
The Sense of Belonging index we have developed includes two aspects of belonging: belonging to institution and belonging to major, which we refer to respectively as Social Belonging and Academic Belonging.The index measurements were initially developed from the results of a Mapworks survey given to new entering first-years who started in 2014 and 2015.The survey consisted of 218 multiple choice or short answer questions and was designed to gauge students' attitudes towards social, financial, and academic aspects of life as a new campus member.These 218 questions were divided josotl.indiana.eduDavis, Hanzsek-Brill, Petzold, and Robinson into Factors, each of which measured a different aspect.Table 1 contains the Mapworks factors that were considered for the project, along with a short description.Factors 1 and 11-20 were used to determine a baseline Social Belonging Index (SBI) and factors 7-10 and 21-22 were used to determine a baseline Academic Behaviors Index (which differs from the Academic Belonging Index, or ABI, which will be introduced later).Students answered questions on a scale from 1 to 7 and we averaged responses to determine each student's belonging indices.Indices were grouped using quartiles as cut scores to sort students into Low, Medium, and High belonging groups.

Table 2. Academic Behavior Groups versus Term 1 GPA
These results, which show an association between Academic Behavior Index, GPA, and retention, mirrored what we would have expected from our existing retention prediction model.The results from the Social Belonging Groups (Figure 2), however, revealed that social belonging played a more significant role in predicting persistence to Term Two and Three than we had anticipated (Woosley & Miller, 2009).While the persistence rate difference between the High and Low on the Academic Behaviors scale was 12% in Term 2 and 13% in Term 3, the difference between High and Low on the Social Belonging was significantly larger: 18% and 28%, respectively.These results indicate that Social Belonging had a much higher impact on retention than Academic Belonging (Tinto, 1993;Woolsey 2003).
We also note that GPA was not strongly correlated to SBI level (Table 3), which suggests that a social belonging measure might capture academically high-achieving students at risk for nonretention or allow us to change our strategy for outreach to academically low-achieving students who have a high belonging score and are therefore more likely to persist.

Table 3. Social Belonging Groups versus Term 1 GPA
Having demonstrated the importance of social belonging as a critical factor for determining retention, we sought to construct a more streamlined survey that would allow us to track students' sense of belonging at critical transition points throughout their time on campus.We also wanted to find an academic measure that moved away from self-reported classroom and study behavior, and instead focused on students' perceptions of their position and relative sense of belonging in a classroom setting (Steele & Aronson, 1995;Steele, 1997;Dweck 2008).
An analysis was conducted for each of the eleven factors that contributed to Social Belonging and their related questions to determine their correlation and significance to Term 3 retention.The results found that Factors 1, 11, 13, 15 and 16 had the strongest effect.Two corresponding questions to each of these five Factors were selected based on their strength in predicting Term 3 retention.A second Social Belonging Index analysis was run on these ten questions using the 2014 and 2015 Mapworks data, referred to in Figure 3 as New Index, to see if the retention rates for each belonging category aligned with the original index.

Figure 3. Retention Rates: Original Social Belonging Index and New Social Belonging Index
Given the close alignment of retention rates and Social Belonging categories for both the Original and the New Indices, we felt confident moving forward with the ten chosen questions reflecting the five Social Belonging factors for use in predicting retention.Questions were reworded to improve clarity and allow questions to stand on their own.Mapworks questions are in clusters, with groups of questions sharing common starting language.Our questions are designed to be answered independently, so students can answer the survey without scrolling on their device.
Additionally, ten new questions were developed for Academic Belonging based on the work of Dweck (2008).These new questions were split into two factors: Social Aspects within Major and Classroom Aspects within Major.Examples of these Academic Belonging questions are given below.

Social Aspects within Major
To what degree are you • hanging out with other students in your major?
• making friends with others in your major?
• satisfied with the social activities in your major?

Classroom Aspects within Major
In your classes, to what degree do you The ten questions produce similar results in both the original data from 2014-2015 and in the new Fall 2017 data.From here we needed to establish what belonging category cut-offs might look like for the data that include the new survey questions for Fall 2017 and how all of the questions correlate.The quartiles in Table 6 for all belonging questions, both social and academic, appear to be close to the quartiles in both the Mapworks and the Qualtrics results of the ten original questions.Further analysis (Figure 4) shows a high positive correlation between the ten original questions and these questions plus the new academic belonging questions, with an adjusted R-squared of 0.855, indicating that the new survey and its resulting index should produce results close to those found from the Mapworks data.

Results
SBI continues to show the students whose score places them into the Low Belonging category are at much higher risk for non-retention (Table 7).A new finding shows that non-response is also a risk category for non-retention.Except for the No Response category, GPA is similar across groups, which led us to identify what impact students who earned a zero (0) GPA in Term 1 would have on these groups (Table 8).While students with a zero GPA in Term 1 were part of each belonging category, they were largely clustered in the No Response category.This leads us to conclude that No Response is a higher risk of immediate drop out, in particular when paired with GPA.According to results shown in Table 9, Low belonging and No Response categories have lower retention for ABI, as was true with SBI.Further analysis was conducted by removing the students whose Term 1 GPA was zero (Table 10).As with the SBI, we see a significant difference in the No Response category once students with a Term 1 zero GPA are removed.Analysis also indicates that SBI and ABI are correlated to each other, but neither is correlated with GPA (Table 11).Since we have been using academic measures such as GPA in our retention models, we decided to compare the academic retention model to actual Term 2 retention in the light of the SBI belonging index categories (Table 12).While overall retention rates are essentially the same, actual retention by SBI belonging category indicates a large discrepancy, which suggests that we have been overlooking a key aspect of why students stay and what factors cause them to make that decision.The analysis above indicates our current prediction models for retention based on academic measures alone have been missing a key component: a sense of social belonging.We can break this overall theme into four key findings.
Finding #1: There are at least two identifiable categories of at-risk students: academic performance risk and social belonging risk.The academic performance risk can be predicted by the traditional prediction models using Fall GPA and demographics.The present study, however, shows that social belonging risk must also be assessed and included in retention predicition models.This risk appears to be well predicted using the new survey created to measure a student's Social Belonging Index.
Finding #2: Survey taking behavior is an indicator of retention.Students who do not take the survey are at higher risk for poor academic performance, and therefore Term 2 retention.
Finding #3: The two at-risk groups are stochastically independent.The study results indicate that belonging and academic performance are not correlated and therefore are not predictors of each other.Both must be considered when predicting retention.
Finding #4: ABI and SBI are correlated, but SBI is a better predictor of retention.

Conclusion
The information this survey provides allows us to use predictive measures beyond academic performance to assist first-year students in achieving their goals.A group of students, faculty, staff, and administrators is currently developing a new outreach protocol that takes social belonging into account to complement our existing efforts.Students whose challenges might otherwise have gone unnoticed can now be offered support that is tailored to their specific needs.This work has also prompted us to identify ways our campus can promote social and academic belonging more broadly, including pedagogy workshops, seminars, and a speaker series.Together, these efforts will continue to shape the way we reimagine the first year of college for our students.
Figures 1-2 and Tables 2-3 present the results from the 2014 and 2015 Mapworks surveys correlated with Term 2 and Term 3 persistence and GPA.

Figure 1 .
Figure 1.Academic Behavior Groups and Retentions Rates: Term 2 and Term 3

Figure 2 .
Figure 2. Social Belonging Groups and Retentions Rates: Term 2 and Term 3

Table 7 . Social Belonging Index Belonging Level Number of Students
Davis, Hanzsek-Brill, Petzold, and Robinson Journal of the Scholarship of Teaching and Learning, Vol. 19, No. 1, February 2019.josotl.indiana.edu