Improving the Comparability and Local Usefulness of International Assessments: A Look Back and A Way Forward

Abstract

Over time international large-scale assessments have grown in terms of number of studies, cycles, and participating countries, many of which are a heterogeneous mix of economies, languages, cultures, and geography. This heterogeneity has meaningful consequences for comparably measuring both achievement and non-achievement constructs, such as social and emotional skills. In this paper we propose one way to directly incorporate country-specific differences into the methods used to construct background scales. We use research that demonstrates data quality issues in international assessment and the degree to which these issues can impact inferences. Our proposed solution incorporates innovations that have been developed for achievement measures but have not been applied to background scales. We demonstrate this possible solution with PISA 2012 data.