Equivalence reliability: How often is often?

Abstract

Without reliability, valid score interpretation is meaningless (Throndike & Throndike-Christ, 2010). Based on a similar study conducted earlier (Nelson Laird, Korkmaz, & Chen, 2008), this study focuses on assessing the equivalence reliability of the updated FSSE. In particular, the emphasis is on whether two parallel forms or different versions of survey items produce similar results (have equal means, variances, and errors, etc.). Survey researchers often wonder about the meaning of vague quantifiers such as “sometimes” or “often” as employed by surveys. These analyses examined a set of FSSE questions asked in two different ways, first with vague quantifiers and second with a quantifiable time allocation. If the two versions of items were essentially asking for the same information, we would expect much of the following to be true: each response option will have a distinct meaning (Often means something different than Sometimes, etc.), the intervals between response options would progressively increase in frequency from Never to Very often, and the intervals would be approximately equal (Very often means nine times per week, Often means six times per week, and Sometimes means three times per week).