Sensitivity of Achievement Estimation to Conditioning Model Misclassification

Loading...
Thumbnail Image
Date
2014-04
Journal Title
Journal ISSN
Volume Title
Publisher
Applied Measurement in Education, Taylor and Francis
Abstract
Large-scale assessment programs such as NAEP, TIMSS, and PISA use a sophisticated assessment administration design called matrix sampling that minimizing the testing burden on individual students. Under a matrix sampling approach, not every examinee is administered every item, which poses currently intractable challenges to estimating individual achievement. Instead, population achievement is estimated via a latent regression approach that uses item responses and a vector of examinee background information (e.g., gender and ethnicity). This vector of covariates, used in an imputation model (more commonly called a conditioning model), is assumed to be fully measured, without error. Using simulated data that follows typical large-scale assessment designs, this paper provides some evidence that departures from this assumption can have a meaningful impact on conditioning model parameter estimates, subpopulation achievement estimates, and under- or over-estimates of subpopulation differences. Findings from this study indicate that the severity of parameter estimate bias depends on the measurement error mechanism and the generating ability distribution. Policy implications and impediments to detecting the impact of measurement error are briefly discussed.
Description
Keywords
large-scale assessment, achievement estimation, measurement error, population modeling
Citation
DOI
Link(s) to data and video for this item
Relation
Rights
Type
Article