Increasing student evaluation capacity through a collaborative community-based program evaluation teaching model
Main Article Content
Abstract
The evaluation literature reflects a long-standing interest in ways to provide practical hands-on training experience in evaluation courses. Concomitantly, some funders have shown rising expectations for increased accountability on the part of Community-Based organizations (CBOs), even though agencies often lack the associated funding and expertise required. This paper describes a training evaluation model, developed and refined over the past ten years, that partners student needs for real-world evaluation experience with CBOs needs for technical expertise and assistance in conducting evaluation activities that support program improvement. The process of developing and implementing the model is presented, and content analysis of ten years of student feedback is summarized.
Downloads
Article Details
- Authors retain copyright and grant the Journal of the Scholarship of Teaching and Learning (JoSoTL) right of first publication with the work simultaneously licensed under a Creative Commons Attribution License, (CC-BY) 4.0 International, allowing others to share the work with proper acknowledgement and citation of the work's authorship and initial publication in the Journal of the Scholarship of Teaching and Learning.
- Authors are able to enter separate, additional contractual agreements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in the Journal of the Scholarship of Teaching and Learning.
- In pursuit of manuscripts of the highest quality, multiple opportunities for mentoring, and greater reach and citation of JoSoTL publications, JoSoTL encourages authors to share their drafts to seek feedback from relevant communities unless the manuscript is already under review or in the publication queue after being accepted. In other words, to be eligible for publication in JoSoTL, manuscripts should not be shared publicly (e.g., online), while under review (after being initially submitted, or after being revised and resubmitted for reconsideration), or upon notice of acceptance and before publication. Once published, authors are strongly encouraged to share the published version widely, with an acknowledgement of its initial publication in the Journal of the Scholarship of Teaching and Learning.
References
Anderson, L. W., & Krathwohl, D. R. (Eds.). (2001). A taxonomy for learning, teaching and assessing: A revision of Bloom's Taxonomy of educational objectives. New York : Longman.
Anderson, S. G. (2003). Engaging students in community-based research. A model for teaching social work research. Journal of Community Practice, 10(2), 71-87.
Benjamin, L. M. (2012). Nonprofit organizations and outcome measurement: From tracking program activities to focusing on frontline work. American Journal of Evaluation, 33(3), 431447. doi: 10.1177/1098214012440496
Carman, J. G. (2007). Evaluation practice among community-based organizations: Research into the reality. American Journal of Evaluation, 28(1), 60-75. doi: 10.1177/1098214006296245
Carman, J. G. (2010). The accountability movement: What’s wrong with this theory of change? Nonprofit and Voluntary Sector Quarterly, 39(2), 256-274.
Carman, J. G., & Fredericks, K. A. (2008). Nonprofits and evaluation: Empirical evidence from the field. New Directions for Evaluation, 119, 51-71.
Carman, J. G., & Fredericks, K. A. (2009). Evaluation capacity and nonprofit organizations: Is the glass half-empty or half-full? American Journal of Evaluation, 31(1), 84 -104. doi: 10.1177/1098214009352361
Christensen, R., & Ebrahim, A. (2006). How does accountability affect mission: The case of nonprofit serving immigrants and refugees. Nonprofit Management & Leadership, 17, 195-209.
Cohen J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37-46.
Dewey, J. D., Montrosse, B. E., Schroter, D. C., Sullins, C. D., & Mattox, J. R., II. (2008). Evaluator competencies: What’s taught versus what’s sought. American Journal of Evaluation, 29(3), 268-287. doi: 10.1177/1098214008321152
Dewey, J. D., Montrosse, B. E., Schroter, D. C., Sullins, C. D., & Mattox, J. R., II. (2008). Evaluator competencies: What’s taught versus what’s sought. American Journal of Evaluation, 29(3), 268-287. doi: 10.1177/1098214008321152
Ebrahim, A. (2005). Accountability myopia: Losing sight of organizational learning. Nonprofit and Voluntary Sector Quarterly, 34(1), 56-87.
Fierro, L. A., & Christie, C. A. (2011). Understanding evaluation training in schools and programs of public health. American Journal of Evaluation, 32(3), 448-468. doi: 10.1177/1098214010393721
Gredler, M. E., & Johnson, R. L. (2001). Lessons learned from the directed evaluation experience. American Journal of Evaluation, 22(1), 97-104. doi: 10.1177/109821400102200110
Hall, M. H., Phillips, S. D., Meillat, C., & Pickering, D. (2003). Assessing performance: Evaluation practices & perspectives in Canada’s voluntary sector. Evaluation Research Project (VSERP) Centre for Voluntary Sector Research and Development: Carleton University, Ottawa, Ontario.
Hatry, H.P. Wholey, J.S. & Newcomer, K.E. (2010). Evaluation challenges, issues, and trends. In Wholey, J. S., Hatry, H. P., & Newcomer, K. E. (Eds.). 3rd ed. Handbook of practical program evaluation (pp. 668-679). San Francisco, CA: Jossey-Bass.
Human Subjects Protection, 46 C.F.R., part 46.101, 2009 Available at http://www.hhs.gov/ohrp/humansubjects/guidance/45cfr46.html#46.101 Accessed October 7, 2012.
Jacobson, M., & Goheen, A. (2006). Engaging students in research: A participatory BSW program evaluation. The Journal of Baccalaureate Social Work, 86-104.
Kelly, M. A., & Kaczynski, D. (2008). Teaching evaluation from an experiential framework: Connecting theory and organizational development with grant making. American Journal of Evaluation, 29(4), 547-554. doi: 10.1177/1098214008324181
King, J. A., Cousins, J. B., & Whitmore, E. (2007). Making sense of participatory evaluation: Framing participatory evaluation. New Directions for Evaluation, (114), 83-105. doi: 10.1002/ev.226
Knowlton, L. W., & Phillips, C. C. (2009). The logic model guidebook: Better strategies for great results. Thousand Oaks, CA: Sage publications Inc.
Lee, J., LeBaron Wallace, T., & Alkin, M. (2007). Using problem-based learning to train evaluators. American Journal of Evaluation, 28(4), 536-545. doi: 10.1177/1098214007305613
Lombard, M., Snyder-Duch, J., & Bracken, C. C. (2002). Content analysis in mass communication: Assessment and reporting of inter-coder reliability. Human Communication Research, 28, 587-604.
Lundahl, B. W. (2008). Teaching research methodology through active learning. Journal of Teaching in Social Work, 28(1), 273-288.
Margolis, L. H., Stevens, R., Laraia, B., Ammerman, A., Harlan, C., Dodds, J., Pollard, M. (2000). Educating students for community-based partnerships. Journal of Community Practice, 7(4), 21 - 34.
Mattessich, P. W., Murray-Close, M., & Monsey, B. R. (2001). Collaboration: What makes it work (2nd ed.). St. Paul, MN: Amherst H. Wilder Foundation.
McLaughlin, J. A., & Jordan, G. B. (1999). Logic models: A tool for telling your programs performance story. Evaluation and Program Planning, 22(1), 65-72. doi: http://dx.doi.org/10.1016/S0149-7189(98)00042-1
Murphy, D. M., & Mitchell, R. (2007). Building evaluation capacity in North Carolina's nonprofit sector: A survey report. Raleigh, NC: Institute for Nonprofits.
Patton, M.Q. (2008). What is utilization-focused evaluation? Utilization-focused evaluation. 4th edition (pp. 35-57). Newbury Park, CA: Sage.
Preskill, H., Zuckerman, B., & Matthews, B. (2003). An exploratory study of process use: Findings and implications for future research. American Journal of Evaluation, 24(4), 423-442. doi: 10.1177/109821400302400402
Simons, L., & Cleary, B. (2006). The influence of service learning on student’s personal and social development. College Teaching, 54(4), 307-319.
Snibbe, A. C. (2006). Drowning in data. Stanford Social Innovation Review, (4), 3, 39-45.
Taut, S. M., & Alkin, M. C. (2003). Program staff perceptions of barriers to evaluation implementation. American Journal of Evaluation, 24(2), 213-226. doi: 10.1177/109821400302400205
Trevisan, M. S. (2002). Enhancing practical evaluation training through long-term evaluation projects. American Journal of Evaluation, 23(1), 81-92. doi: 10.1177/109821400202300110
Trevisan, M. S. (2004). Practical training in evaluation: A review of the literature. American Journal of Evaluation, 25(2), 255-272. doi: 10.1177/109821400402500212
Wholey, J. S., Hatry, H. P., & Newcomer, K. E. (Eds.). (2010). Handbook of practical program evaluation. 3rd ed. San Francisco, CA: Jossey-Bass.
W.K. Kellogg Foundation (2004). Logic model development guide: Using logic models to bring together planning, evaluation, and action. Battle Creek, Michigan.