TESTING CROSS-CULTURAL MEASUREMENT INVARIANCE AND COMPARING INTERNATIONAL STUDENT ENGAGEMENT

Abstract

Research instruments are often designed in specific cultural contexts and most often for participants who are from dominant cultures. Responses from participants who are not from these types of cultures could present a threat to the validity of comparisons between students from differing cultural origins. This threat to validity of comparisons is called measurement bias (or measurement non-invariance), which can originate from sources that differ systematically across cultures.

This study employs Berry’s Ecological Framework (Berry, 2018) from the field of cross- cultural psychology as a basis for assessing how well certain constructs of student engagement, as conceptualized for and measured by the National Survey of Student Engagement (NSSE), relates to international undergraduate student experiences in US four-year institutions. The internal measurement validity for three NSSE engagement indicators (EI) is assessed between domestic and international students and between students from different world regions and nations. Maintaining measurement invariance allows for valid comparisons among different groupings on three engagement constructs.

The methods used in this study include Multi-group Confirmatory Factor Analysis (MGCFA) to assess measurement invariance and the Multiple Indicators Multiple Causes (MIMIC) model—with and without accounting for covariates—to compare levels of engagement between groups if invariant. Results indicate that one EI had poor model fit and was excluded from the study. The other two EIs, Student-Faculty Interaction and Collaborative Learning, maintained measurement invariance, indicating that these EIs function similarly across cultural groups.

In terms of comparing levels of engagement, on average: 1) international students report more interaction with faculty than domestic students; 2) Asian, Latin American, and Sub-Saharan African students report more interaction with faculty than North Americans; 3) Chinese, Indian, Mexican, and South Korean students report more interaction with faculty than US students; 4) Sub-Saharan African students report more collaboration with their peers than North Americans; 5) and Indian and South Korean students report more collaboration with their peers than US students. These findings offer evidence to higher education administrators that the Student-Faculty Interaction and Peer Collaboration EI’s are valid for international students. These methods and findings offer institutional researchers as well as higher education researchers insights into assessing cultural bias from survey data.