Power Analysis Software for Educational Researchers

Abstract

Given the importance of statistical power analysis in quantitative research and the repeated emphasis on it by AERA/APA journals, we examined the reporting practice of power analysis by the quantitative studies published in 12 education/psychology journals between 2005 and 200910. It was surprising to uncover that less than 2% of the studies conducted prospective power analysis. Another 3.54% computed observed power, a practice not endorsed by the literature on power analysis. In this paper, we clarify these two types of power analysis and discuss functionalities of eight programs/packages (G*Power 3.1.3, PASS 11, SAS/STAT 9.3, Stata 12, SPSS 19, SPSS/Sample Power 3.0.1, Optimal Design Software 2.01, and MLPowSim 1.0 BETA) to encourage proper and planned power analysis. Based on our review, we recommend two programs (SPSS/Sample Power and G*Power) for general-purpose univariate/multivariate analyses, and one (Optimal Design Software) for hierarchical/multilevel modeling and meta-analysis. Recommendations are also made for reporting power analysis results and exploring additional software. The paper concludes with an examination of the role of statistical power in research and viable alternatives to hypothesis testing.