Transparent quality: Framing and building a psychometric portfolio

Abstract

High quality data and assessment instruments have become essential for institutional researchers to take a data-driven approach to informing decision-making and strategic planning. Instruments and the resulting data they collect can be studied for different aspects of validity and reliability as well as the procedures and standards used to reduce error, bias, and increase the rigor of the data. This presentation will focus on a framework for operationalizing and organizing a wide variety of studies to investigate data quality. Participants in this session will see how a large-scale quantitative survey project designed and created a psychometric portfolio with studies designed to make survey instrument and data quality transparent so that higher education leaders, researchers, and professionals can trust the results. Challenges and potential solutions, including thinking about strategies for conducting studies of data quality with limited time and resources will be discussed.