Analyzing ordinal data with metric models: What could possibly go wrong?
Loading...
If you need an accessible version of this item, please email your request to iusw@iu.edu so that they may create one and provide it to you.
Date
2018-03-23
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Indiana University Workshop in Methods
Permanent Link
Abstract
We surveyed all articles in the Journal of Personality and Social Psychology, Psychological Science, and the Journal of Experimental Psychology: General that mentioned the term "Likert," and found that 100% of the articles that analyzed ordinal data did so using a metric model. We demonstrate that analyzing ordinal data as if they were metric can systematically lead to errors. We demonstrate false alarms (i.e., detecting an effect where none exists, Type~I errors) and failures to detect effects (i.e., loss of power, Type~II errors). We demonstrate systematic inversions of effects, for which treating ordinal data as metric indicates the opposite ordering of means than the true ordering of means. We show the same problems --- false alarms, misses, and inversions --- for interactions in factorial designs and for trend analyses in regression. We demonstrate that averaging across multiple ordinal measurements does not solve or even ameliorate these problems. We provide simple graphical explanations of why these mistakes occur. Moreover, we point out that there is no sure-fire way to detect these problems by treating the ordinal values as metric, and instead we advocate use of ordered-probit models (or similar) because they will better describe the data. Finally, although frequentist approaches to some ordered-probit models are available, we use Bayesian methods because of their flexibility in specifying models and their richness and accuracy in providing parameter estimates.
Description
John K. Kruschke is Provost Professor of Psychological and Brain Sciences, and Adjunct Professor of Statistics, at Indiana University in Bloomington, Indiana, USA. He is eight-time winner of Teaching Excellence Recognition Awards from Indiana University. He won the Troland Research Award from the National Academy of Sciences (USA), and the Remak Distinguished Scholar Award from Indiana University. He has been on the editorial boards of various scientific journals, including Psychological Review, the Journal of Experimental Psychology: General, and the Journal of Mathematical Psychology, among others.
(With co-author Torrin M. Liddell, Research and Statistics Analyst, Public Defender Commission, Indianapolis)
Keywords
Workshop in Methods, research methods, statistical methods, quantitative research methods
Citation
DOI
Link(s) to data and video for this item
Rights
Type
Presentation