Using a Quasi-Experimental Design in Combination with Multivariate Analysis to Assess Student Learning

Article Sidebar

cover image
PDF
Published: Oct 24, 2018
DOI: https://doi.org/10.14434/josotl.v19i1.24474
Keywords:
learning versus performance, knowledge gain, direct measures, pretest/post-test assessment, examinations, group work

Main Article Content

Michael Delucchi
University of Hawaii

Abstract

College professors in the social sciences and professional studies have adopted numerous strategies for teaching undergraduate statistics, yet few researchers provide empirical evidence students’ learning actually increased because of the instructional innovation. Assessment of pedagogy is frequently subjective and based on comments from students or faculty. Consequently, evaluating the effectiveness of teaching activities on student learning in statistical analysis courses is warranted. This study employed a pretest-posttest design to measure student learning and then examined the relationship between student demographics, prior knowledge, and course characteristics on knowledge gained in undergraduate statistics. Data derived from 185 students enrolled in six different sections of a statistical analysis course taught over a seven-year period by the same instructor. Multiple regression analyses revealed age, age X gender (interaction effect), major, prior knowledge, examinations, and group projects all had statistically significant effects on how much students learned in the course. The results suggest faculty assess students’ prior knowledge at the beginning of the semester and use such data to inform both the content and delivery of statistical analysis. Moreover, before embracing a new pedagogy, faculty should establish empirically that learning is linked to the teaching innovation.

 





Downloads

Download data is not yet available.

Article Details

How to Cite
Delucchi, M. (2018). Using a Quasi-Experimental Design in Combination with Multivariate Analysis to Assess Student Learning. Journal of the Scholarship of Teaching and Learning, 19(2). https://doi.org/10.14434/josotl.v19i1.24474
Issue
Vol. 19 No. 2 (2019): Journal of the Scholarship of Teaching and Learning
Section
Articles
  1. Authors retain copyright and grant the Journal of the Scholarship of Teaching and Learning (JoSoTL) right of first publication with the work simultaneously licensed under a Creative Commons Attribution License, (CC-BY) 4.0 International, allowing others to share the work with proper acknowledgement and citation of the work's authorship and initial publication in the Journal of the Scholarship of Teaching and Learning.
     
  2. Authors are able to enter separate, additional contractual agreements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in the Journal of the Scholarship of Teaching and Learning.

  3. In pursuit of manuscripts of the highest quality, multiple opportunities for mentoring, and greater reach and citation of JoSoTL publications, JoSoTL encourages authors to share their drafts to seek feedback from relevant communities unless the manuscript is already under review or in the publication queue after being accepted. In other words, to be eligible for publication in JoSoTL, manuscripts should not be shared publicly (e.g., online), while under review (after being initially submitted, or after being revised and resubmitted for reconsideration), or upon notice of acceptance and before publication. Once published, authors are strongly encouraged to share the published version widely, with an acknowledgement of its initial publication in the Journal of the Scholarship of Teaching and Learning.
Author Biography

Michael Delucchi, University of Hawaii

Professor of Sociology

References

Arum, R. and Roksa, J. (2011). Academically adrift: Limited learning on college campuses. Chicago, IL: University of Chicago Press.

Astin, A. (1993). What matters in college? Four critical years revisited. San Francisco, CA: Jossey-Bass.

Auster, C. J. (2000). Probability sampling and inferential statistics: An interactive exercise using M&M’s. Teaching Sociology, 28(4), 379-385.

Baker, P. (1985). Does the sociology of teaching inform Teaching Sociology? Teaching Sociology, 12(3), 361-375.

Bandalos, D. L., Finney, S. J., & Geske, J. A. (2003). A model of statistics performance based on achievement goal theory. Journal of Educational Psychology, 95(3), 604-616.

Bjork, E. L., & Bjork, R. (2011). Making things hard on yourself, but in a good way: Creating desirable difficulties to enhance learning. In M.A. Gernsbacher, R.W. Pew, L.M. Hough, & J.R. Pomerantz (Eds.), Psychology and the real world: Essays illustrating contributions to society (pp. 56-64). New York, NY: Worth Publishers.

Blalock, H. M., Jr. (1987). Some general goals in teaching statistics. Teaching Sociology, 15(2), 164-172.

Borresen, R. C. (1990). Success in introductory statistics with small Groups. College Teaching, 38(1), 26-28.

Bridges, G. S., Gillmore, G. M., Pershing, J. L., & Bates, K. A. (1998). Teaching quantitative research methods: A quasi-experimental analysis. Teaching Sociology, 26(1), 14-24.

Bye, D., Pushkar, D., & Conway, M. (2007). Motivation, interest, and positive affect in traditional and nontraditional undergraduate students. Adult Education Quarterly, 57(2), 141-158.

Carney-Crompton, S., & Tan, J. (2002). Support systems, psychological functioning, and academic performance of nontraditional female students. Adult Education Quarterly, 52(2), 140-154.

Center for Postsecondary Research. (2015). Carnegie classification of institutions of higher education. Indiana University School of Education. Bloomington, IN. http://carnegieclassifications.iu.edu/index.php

Cerrito, P. B. (1999). Teaching statistical literacy. College Teaching, 47(1), 9-13.

Chin, J. (2002). Is there a scholarship of teaching and learning in Teaching Sociology? A look at papers from 1984 to 1999. Teaching Sociology, 30(1), 53-62.

DeCesare, M. (2007). Statistics anxiety among sociology majors: A first diagnosis and some treatment options. Teaching Sociology, 35(4), 360-367.

Delucchi, M. (2006). The efficacy of collaborative learning groups in an undergraduate statistics course. College Teaching, 54(2), 244-248.

Delucchi, M. (2007). Assessing the impact of group projects on examination performance in social statistics. Teaching in Higher Education, 12(4), 447-460.

Fischer, H. W., III. (1996). Teaching statistics from the user’s perspective. Teaching Sociology, 24(2), 225-230.

Fisher-Giorlando, M. (1992). Sampling in a suitcase: Multistage cluster sampling made easy. Teaching Sociology, 20(4), 285-287.

Forte, J. (1995). Teaching statistics without sadistics. Journal of Social Work Education, 31(2), 204-219.

Garfield, J., & Chance, B. (2000). Assessment in statistics education: Issues and challenges. Mathematical Thinking and Learning, 2(1/2), 99-125.

Gelles, R. J. (1980). Teaching sociology on teaching sociology. Teaching Sociology, 8(1), 3-20.

Healey, J. F. (2015). Statistics: A tool for social research. 10th Edition. Cengage Learning.

Helmericks, S. (1993). Collaborative testing in social statistics: Toward gemeinstat. Teaching Sociology, 21(3), 287-297.

Little, J. L., & Bjork, E. L. (2010). Multiple-choice testing can improve the retention of non-tested related information. In S. Ohisson & R. Catrabone (Eds.), Proceedings of the 32nd Annual Conference of the Cognitive Science Society (pp. 1535-1540). Austin, TX: Cognitive Science Society.

Little, J. L., & Bjork, E. L. (2011). Pretesting with multiple-choice questions facilitates learning. In L. Carlson, C. Holscher, and T. F. Shipley (Eds.), Proceedings of the 33rd Annual Conference of the Cognitive Science Society (pp. 294-296). Austin, TX: Cognitive Science Society.

Little, J. L., Bjork, E. L., Bjork, R. A., & Angello, G. (2011). Multiple-Choice tests exonerated, at least of some charges: Fostering test-induced learning and avoiding test-induced forgetting. Psychological Science, 18(11), 1337-1344.

Lomax, R. G., & Moosavi. S. A. (2002). Using humor to teach statistics: Must they be orthogonal? Understanding Statistics, 1(2), 113-130.

Lucal, B., Albers, C., with Ballantine, J., Burmeister-May, J., Chin, J., Dettmer, S., & Larson, S. (2003). Faculty assessment and the scholarship of teaching and learning: knowledge available/knowledge needed. Teaching Sociology, 31(2), 146-161.

Luce, C., & Kirnan, J. P. (2016). Using indirect vs. direct measures in the summative assessment of student learning in higher education. Journal of the Scholarship of Teaching and Learning, 16(4), 75-91. doi:10.14434/josotl.v16i4.193711.

Macheski, G. E., Buhrmann, J., Lowney, K. S., & Bush, M. E. (2008). Overcoming student disengagement and anxiety in theory, methods, and statistics courses by building a community of learners. Teaching Sociology, 36(1), 42-48.

Marson, S. M. (2007). Three empirical strategies for teaching statistics. Journal of Teaching in Social Work, 27(3/4), 199-213.

McKinney, K., & Graham-Buxton, M. (1993). The use of collaborative learning groups in the large class: Is it possible? Teaching Sociology, 21(4), 403-408.

Neuman, L. W. (1989). Which students learn the most, and why? A replication and extension of the Szafran pretest study. Teaching Sociology, 17(1), 19-27.

Perkins, D. V., & Saris, R. N. (2001). A ‘Jigsaw Classroom’ technique for undergraduate statistics courses. Teaching of Psychology, 28(2), 111-113.

Potter, A. M. (1995). Statistics for sociologists: Teaching techniques that work. Teaching Sociology, 23(3), 259-263.

Price, B. A., & Randall, C. H. (2008). Assessing learning outcomes in quantitative courses: Using embedded questions for direct assessment. Journal of Education for Business, 83(5), 288-294.

Rau, W., & Heyl, B. S. (1990). Humanizing the college classroom: Collaborative learning and social organization among students. Teaching Sociology, 18(2), 141-155.

Schacht, S. P., & Stewart, B. J. (1990). What’s funny about statistics? A technique for reducing student anxiety. Teaching Sociology, 18(1), 52-56.

Schacht, S. P., & Stewart, B. J. (1992). Interactive/User-Friendly gimmicks for teaching statistics. Teaching Sociology, 20(4), 329-332.

Schumm, W. R., Webb, F. J., Castelo, C. S., Akagi, C. G., Jensen, E. J., Ditto, R. M., Spencer-Carver, E., & Brown, B. F. (2002). Enhancing learning in statistics classes through the use of concrete historical examples: The space shuttle Challenger, Pearl Harbor, and the RMS Titanic. Teaching Sociology, 30(3), 361-375.

Schutz, P. A., Drogosz, L. M., White, V. E., & Distefano, C. (1998). Prior knowledge, attitude, and strategy use in an introduction to statistics course. Learning & Individual Differences, 10(4), 291308.

Smith, B. (2003). Using and evaluating resampling simulations in SPSS and EXCEL. Teaching Sociology, 31(3), 276-287.

Stork, D. (2003). Teaching statistics with study survey data: A pedagogical innovation in iupport of student learning. Journal of Education for Business, 78(6), 335-339.

Strangfeld, J. A. (2013). Promoting active learning: Student-Led data gathering in undergraduate statistics. Teaching Sociology, 41(2), 199-206.

Wagenaar, T. C. (2002). Outcomes assessment in sociology: Prevalence and impact. Teaching Sociology, 30(4), 403-413.

Weiss, G. L. (2002). The current status of assessment in sociology departments. Teaching Sociology, 30(4), 391-402.

Wilder, E. I. (2010). A qualitative assessment of efforts to integrate data analysis throughout the sociology curriculum: Feedback from students, faculty, and alumni. Teaching Sociology, 38(3), 226-246.

Wybraniec, J., & Wilmoth, J. (1999). Teaching students inferential statistics: A ‘Tail’ of three distributions. Teaching Sociology, 27(1), 74-80.

Yamarik, S. (2007). Does cooperative learning improve student learning outcomes? Journal of Economic Education, 38(Summer), 259-277.