Increasing student evaluation capacity through a collaborative community-based program evaluation teaching model

Main Article Content

Shauna K Carlisle
Jean M Kruzich


The evaluation literature reflects a long-standing interest in ways to provide practical hands-on training experience in evaluation courses. Concomitantly, some funders have shown rising expectations for increased accountability on the part of Community-Based organizations (CBOs), even though agencies often lack the associated funding and expertise required. This paper describes a training evaluation model, developed and refined over the past ten years, that partners student needs for real-world evaluation experience with CBOs needs for technical expertise and assistance in conducting evaluation activities that support program improvement. The process of developing and implementing the model is presented, and content analysis of ten years of student feedback is summarized.


Download data is not yet available.

Article Details

How to Cite
Carlisle, S. K., & Kruzich, J. M. (2013). Increasing student evaluation capacity through a collaborative community-based program evaluation teaching model. Journal of the Scholarship of Teaching and Learning, 13(4), 85–102. Retrieved from
Author Biographies

Shauna K Carlisle, University of Washington Bothell

School of Interdisciplinary Arts and Sciences

Assistant Professor

Jean M Kruzich, University of Washington Seattle

School of Social Work

Associate Professor


Anderson, L. W., & Krathwohl, D. R. (Eds.). (2001). A taxonomy for learning, teaching and assessing: A revision of Bloom's Taxonomy of educational objectives. New York : Longman.

Anderson, S. G. (2003). Engaging students in community-based research. A model for teaching social work research. Journal of Community Practice, 10(2), 71-87.

Benjamin, L. M. (2012). Nonprofit organizations and outcome measurement: From tracking program activities to focusing on frontline work. American Journal of Evaluation, 33(3), 431447. doi: 10.1177/1098214012440496

Carman, J. G. (2007). Evaluation practice among community-based organizations: Research into the reality. American Journal of Evaluation, 28(1), 60-75. doi: 10.1177/1098214006296245

Carman, J. G. (2010). The accountability movement: What’s wrong with this theory of change? Nonprofit and Voluntary Sector Quarterly, 39(2), 256-274.

Carman, J. G., & Fredericks, K. A. (2008). Nonprofits and evaluation: Empirical evidence from the field. New Directions for Evaluation, 119, 51-71.

Carman, J. G., & Fredericks, K. A. (2009). Evaluation capacity and nonprofit organizations: Is the glass half-empty or half-full? American Journal of Evaluation, 31(1), 84 -104. doi: 10.1177/1098214009352361

Christensen, R., & Ebrahim, A. (2006). How does accountability affect mission: The case of nonprofit serving immigrants and refugees. Nonprofit Management & Leadership, 17, 195-209.

Cohen J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37-46.

Dewey, J. D., Montrosse, B. E., Schroter, D. C., Sullins, C. D., & Mattox, J. R., II. (2008). Evaluator competencies: What’s taught versus what’s sought. American Journal of Evaluation, 29(3), 268-287. doi: 10.1177/1098214008321152

Dewey, J. D., Montrosse, B. E., Schroter, D. C., Sullins, C. D., & Mattox, J. R., II. (2008). Evaluator competencies: What’s taught versus what’s sought. American Journal of Evaluation, 29(3), 268-287. doi: 10.1177/1098214008321152

Ebrahim, A. (2005). Accountability myopia: Losing sight of organizational learning. Nonprofit and Voluntary Sector Quarterly, 34(1), 56-87.

Fierro, L. A., & Christie, C. A. (2011). Understanding evaluation training in schools and programs of public health. American Journal of Evaluation, 32(3), 448-468. doi: 10.1177/1098214010393721

Gredler, M. E., & Johnson, R. L. (2001). Lessons learned from the directed evaluation experience. American Journal of Evaluation, 22(1), 97-104. doi: 10.1177/109821400102200110

Hall, M. H., Phillips, S. D., Meillat, C., & Pickering, D. (2003). Assessing performance: Evaluation practices & perspectives in Canada’s voluntary sector. Evaluation Research Project (VSERP) Centre for Voluntary Sector Research and Development: Carleton University, Ottawa, Ontario.

Hatry, H.P. Wholey, J.S. & Newcomer, K.E. (2010). Evaluation challenges, issues, and trends. In Wholey, J. S., Hatry, H. P., & Newcomer, K. E. (Eds.). 3rd ed. Handbook of practical program evaluation (pp. 668-679). San Francisco, CA: Jossey-Bass.

Human Subjects Protection, 46 C.F.R., part 46.101, 2009 Available at Accessed October 7, 2012.

Jacobson, M., & Goheen, A. (2006). Engaging students in research: A participatory BSW program evaluation. The Journal of Baccalaureate Social Work, 86-104.

Kelly, M. A., & Kaczynski, D. (2008). Teaching evaluation from an experiential framework: Connecting theory and organizational development with grant making. American Journal of Evaluation, 29(4), 547-554. doi: 10.1177/1098214008324181

King, J. A., Cousins, J. B., & Whitmore, E. (2007). Making sense of participatory evaluation: Framing participatory evaluation. New Directions for Evaluation, (114), 83-105. doi: 10.1002/ev.226

Knowlton, L. W., & Phillips, C. C. (2009). The logic model guidebook: Better strategies for great results. Thousand Oaks, CA: Sage publications Inc.

Lee, J., LeBaron Wallace, T., & Alkin, M. (2007). Using problem-based learning to train evaluators. American Journal of Evaluation, 28(4), 536-545. doi: 10.1177/1098214007305613

Lombard, M., Snyder-Duch, J., & Bracken, C. C. (2002). Content analysis in mass communication: Assessment and reporting of inter-coder reliability. Human Communication Research, 28, 587-604.

Lundahl, B. W. (2008). Teaching research methodology through active learning. Journal of Teaching in Social Work, 28(1), 273-288.

Margolis, L. H., Stevens, R., Laraia, B., Ammerman, A., Harlan, C., Dodds, J., Pollard, M. (2000). Educating students for community-based partnerships. Journal of Community Practice, 7(4), 21 - 34.

Mattessich, P. W., Murray-Close, M., & Monsey, B. R. (2001). Collaboration: What makes it work (2nd ed.). St. Paul, MN: Amherst H. Wilder Foundation.

McLaughlin, J. A., & Jordan, G. B. (1999). Logic models: A tool for telling your programs performance story. Evaluation and Program Planning, 22(1), 65-72. doi:

Murphy, D. M., & Mitchell, R. (2007). Building evaluation capacity in North Carolina's nonprofit sector: A survey report. Raleigh, NC: Institute for Nonprofits.

Patton, M.Q. (2008). What is utilization-focused evaluation? Utilization-focused evaluation. 4th edition (pp. 35-57). Newbury Park, CA: Sage.

Preskill, H., Zuckerman, B., & Matthews, B. (2003). An exploratory study of process use: Findings and implications for future research. American Journal of Evaluation, 24(4), 423-442. doi: 10.1177/109821400302400402

Simons, L., & Cleary, B. (2006). The influence of service learning on student’s personal and social development. College Teaching, 54(4), 307-319.

Snibbe, A. C. (2006). Drowning in data. Stanford Social Innovation Review, (4), 3, 39-45.

Taut, S. M., & Alkin, M. C. (2003). Program staff perceptions of barriers to evaluation implementation. American Journal of Evaluation, 24(2), 213-226. doi: 10.1177/109821400302400205

Trevisan, M. S. (2002). Enhancing practical evaluation training through long-term evaluation projects. American Journal of Evaluation, 23(1), 81-92. doi: 10.1177/109821400202300110

Trevisan, M. S. (2004). Practical training in evaluation: A review of the literature. American Journal of Evaluation, 25(2), 255-272. doi: 10.1177/109821400402500212

Wholey, J. S., Hatry, H. P., & Newcomer, K. E. (Eds.). (2010). Handbook of practical program evaluation. 3rd ed. San Francisco, CA: Jossey-Bass.

W.K. Kellogg Foundation (2004). Logic model development guide: Using logic models to bring together planning, evaluation, and action. Battle Creek, Michigan.