A Multilevel Analysis of Problem-Based Learning Design Characteristics
Main Article Content
Abstract
The increasing use of experience-centered approaches like problem-based learning (PBL) by learning and development practitioners and management educators has raised interest in how to design, implement and evaluate PBL in that field. Of particular interest is how to evaluate the relative impact of design characteristics that exist at the individual and team levels of analysis. This study proposes and tests a multilevel model of PBL design characteristics. Participant perceptions of PBL design characteristics are used to examine PBL reactions and perceived learning outcomes. Findings affirm the importance of problem design characteristics and effective team facilitation, while raising new questions about team-level characteristics such as goal orientation diversity.
Article Details
1. Publication and Promotion: In consideration of the Publisher’s agreement to publish the Work, Author hereby grants and assigns to Publisher the non-exclusive right to print, publish, reproduce, or distribute the Work throughout the world in all means of expression by any method now known or hereafter developed, including electronic format, and to market or sell the Work orany part of it as Publisher sees fit. Author further grants Publisher the right to use Author’s name in association with the Work inpublished form and in advertising and promotional materials
2. Copyright: Copyright of the Work remains in Author’s name.
3. Prior Publication and Attribution: Author agrees not to publish the Work in print form prior to publication of the Work by the Publisher. Author agrees to cite, by author, title, and publisher, the original Interdisciplinary Journal of Problem-based Learning publication when publishing the Work elsewhere
4. Author Representations: The Author represents and warrants that the Work:
(a) is the Author’s original Work and that Author has full power to enter into this Agreement;
(b) does not infringe the copyright or property of another;
(c) contains no material which is obscene, libelous, defamatory or previously published, in whole or in part.
Author shall indemnify and hold Publisher harmless against loss of expenses arising from breach of any such warranties.
5. Licensing and Reuse: Reuse of the published Work will be governed by a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0; http://creativecommons.org/licenses/ by-nc/4.0/). This license lets others remix, tweak, and build upon the Work non-commercially; although new works must acknowledge the original Interdisciplinary Journal of Problem-based Learning publication and be non-commercial, they do not have to be licensed on the same terms.
References
Araz, G., & Sungur, S. (2007). The interplay between cognitive and motivational variables in a problem-based learning environment. Learning and Individual Differences, 17(4), 291–297. http://dx.doi.org/10.1016/j.lindif.2007.04.003
Armstrong, S. J., & Mahmud, A. (2008). Experiential learning and the acquisition of managerial tacit knowledge. Academy of Management Learning and Education, 7(2), 189– 208. http://dx.doi.org/10.5465/AMLE.2008.32712617
Barrows, H. S., & Tamblyn, R. M. (1980). Problem-based learning: An approach to medical education. New York, NY: Springer.
Barrows, H. S., & Mitchell, D. L. M. (1975). An innovative course in undergraduate neuroscience: Experiment in problem-based learning with “problem boxes.” British Journal of Medical Education, 9(4), 223–230. http://dx.doi.org/10.1111/j.1365-2923.1975.tb01930.x
Benjamin, B., & O’Reilly, C. (2011). Becoming a leader: Early career challenges faced by MBA graduates. Academy of Management Learning and Education, 10(3), 452–472. http://dx.doi.org/10.5465/amle.2011.0002
Bigelow, J. D. (2004). Using problem-based learning to develop skills in solving unstructured problems. Journal of Management Education, 28(5), 591–609. http://dx.doi.org/10.1177/1052562903257310
Breugst, N., Patzelt, H., Shepherd, D. A., & Aguinis, H. (2012). Relationship conflict improves team performance assessment accuracy: Evidence from a multilevel study. Academy of Management Learning and Education, 11(2), 187–206. http://dx.doi.org/10.5465/amle.2011.0032
Brownell, J., & Jameson, D. A. (2004). Problem-based learning in graduate management education: An integrative model and interdisciplinary application. Journal of Management Education, 28(5), 558–577. http://dx.doi.org/10.1177/1052562904266073
Button, S. B., Mathieu, J. E., & Zajac, D. M. (1996). Goal orientation in organizational research: A conceptual and empirical foundation. Organizational Behavior and Human Decision Processes, 67(1), 26–48. http://dx.doi.org/10.1006/obhd.1996.0063
Chaharbaghi, K., & Cox, R. (1995). Problem-based learning: Potential and implementation issues. British Journal of Management, 6(4), 249–256. http://dx.doi.org/10.1111/j.1467-8551.1995.tb00098.x
DeGeest, D., & Brown, K. G. (2011). The role of goal orientation in leadership development. Human Resource Development Quarterly, 22(2), 157–175. http://dx.doi.org/10.1002/hrdq.20072
DeRue, D. S., & Wellman, N. (2009). Developing leaders via experience: The role of developmental challenge, learning orientation, and feedback availability. Journal of Applied Psychology, 94(4), 859–875. http://dx.doi.org/10.1037/a0015317
Dierdorff, E. C., & Ellington, J. K. (2012). Members matter in team training: Multilevel and longitudinal relationships between goal orientation, self-regulation, and team outcomes. Personnel Psychology, 65(3), 661–703. http://dx.doi.org/10.1111/j.1744-6570.2012.01255.x
Dolmans, D. H. J. M., & Schmidt, H. G. (2006). What do we know about cognitive and motivational effects of small group tutorials in problem-based learning? Advances in Health Sciences Education, 11(4), 321–336. http://dx.doi.org/10.1007/s10459-006-9012-8
Dweck, C. S. (1986). Motivational processes affecting learning. American Psychologist, 41(10), 1040–1048. http://dx.doi.org/10.1037/0003-066X.41.10.1040
Dweck, C. S., & Leggett, E. L. (1988). A social-cognitive approach to motivation and personality. Psychological Review, 95(2), 256–273. http://dx.doi.org/10.1037/0033-295X.95.2.256
Garavan, T. N., McGuire, D., & O’Donnell, D. (2004). Exploring human resource development: A levels of analysis approach. Human Resource Development Review, 3(4), 417–441. http://dx.doi.org/10.1177/1534484304271669
Gijselaers, W. H., & Schmidt, H. G. (1990). Towards a causal model of student learning within the context of a problem- based curriculum. In Z. M. Nooman, H. G. Schmidt, & E. S. Ezzat (Eds.), Innovation in medical education: An evaluation of its present status (pp. 95–114). New York, NY: Springer.
Goltz, S. M., Hietapelto, A. B., Reinsch, R. W., & Tyrell, S. K. (2008). Teaching teamwork and problem solving concurrently. Journal of Management Education, 32(5), 541–562. http://dx.doi.org/10.1177/1052562907310739
Hallinger, P., & Lu, J. (2011). Assessing the instructional effectiveness of problem-based management education in Thailand: A longitudinal evaluation. Management Learning, 42(3), 279–299. http://dx.doi.org/10.1177/1350507610388596
Harman, H. H. (1976). Modern factor analysis (3rd ed.). Chicago, IL: University of Chicago Press.
Hmelo, C. E., & Lin, X. (2000). Becoming self-directed learners: Strategy development in problem-based learning. In D. H. Evensen & C. E. Hmelo (Eds.), Problem-based learning: A research perspective on learning interactions (pp. 227–250). Mahwah, NJ: Lawrence Erlbaum Associates.
Hmelo-Silver, C. E. (2004). Problem-based learning: What and how do students learn? Educational Psychology Review, 16(3), 235–266. http://dx.doi.org/10.1023/B:EDPR.0000034022.16470.f3
Hmelo-Silver, C. E., & Barrows, H. S. (2008). Facilitating collaborative knowledge building. Cognition and Instruction, 26(1), 48–94. http://dx.doi.org/10.1080/07370000701798495
Hmelo-Silver, C. E., Duncan, R. G., & Chinn, C. A. (2007). Scaffolding and achievement in problem-based and inquiry learning: A response to Kirschner, Sweller, and Clark (2006). Educational Psychologist, 42(2), 99–107. http://dx.doi.org/10.1080/00461520701263368
Hofmann, D. A. (1997). An overview of the logic and rationale of hierarchical linear models. Journal of Management, 23(6), 723–744. http://dx.doi.org/10.1177/014920639702300602
House, R., Rousseau, D. M., & Thomas-Hunt, M. (1995). The meso paradigm: A framework for the integration of micro and macro organizational behavior. Research in Organizational Behavior, 17, 71–115.
Hung, W. (2006). The 3C3R model: A conceptual framework for designing problems in PBL. Interdisciplinary Journal of Problem-based Learning, 1(1), 55–77. http://dx.doi.org/10.7771/1541-5015.1006
Hung, W. (2009). The 9-step problem design process for problem-based learning: Application of the 3C3R model. Educational Research Review, 4(2), 118–141. http://dx.doi.org/10.1016/j.edurev.2008.12.001
Hung, W. (2011). Theory to reality: A few issues in implementing problem-based learning. Educational Technology Research and Development, 59(4), 529–552. http://dx.doi.org/10.1007/s11423-011-9198-1
Hung, W., Jonassen, D. H., & Liu, R. (2007). Problem-based learning. In J. M. Spector, M. D. Merrill, J. van Merrien- boer, & M. P. Driscoll (Eds.), Handbook of research on educational communications and technology (Vol. 1) (pp. 485–506). New York, NY: Lawrence Erlbaum Associates.
Innes, R. B. (2006). What can learning science contribute to our understanding of the effectiveness of problem-based learning groups? Journal of Management Education, 30(6), 751–764. http://dx.doi.org/10.1177/1052562906287966
James, L. R., Demaree, R. G., & Wolf, G. (1984). Estimating within-group interrater reliability with and without response bias. Journal of Applied Psychology, 69(1), 85–98. http://dx.doi.org/10.1037/0021-9010.69.1.85
James, L. R., Demaree, R. G., & Wolf, G. (1993). rwg: An assessment of within-group interrater agreement. Journal of Applied Psychology, 78(2), 306–309. http://dx.doi.org/10.1037/0021-9010.78.2.306
Kek, M. Y. C. A., & Huijser, H. (2011). The power of problem- based learning in developing critical thinking skills: Preparing students for tomorrow’s digital futures in today’s classrooms. Higher Education Research and Development, 30(3), 329–341. http://dx.doi.org/10.1080/07294360.2010.501074
Kolb, A. Y., & Kolb, D. A. (2005). Learning styles and learning spaces: Enhancing experiential learning in higher education. Academy of Management Learning and Education, 4(2), 193–212. http://dx.doi.org/10.5465/AMLE.2005.17268566
Lindell, M. K., Brandt, C. J., & Whitney, D. J. (1999). A revised index of interrater agreement for multi-item ratings of a single target. Applied Psychological Measurement, 23(2), 127–135. http://dx.doi.org/10.1177/01466219922031257
Liu, D., & Fu, P.-P. (2011). Motivating protégés’ personal learning in teams: A multilevel investigation of autonomy support and autonomy orientation. Journal of Applied Psychology, 96(6), 1195–1208. http://dx.doi.org/10.1037/a0024716
Loyens, S. M. M., Kirschner, P. A., & Paas, F. (2012). Problem-based learning. In K. R. Harris, S. Graham, T. Urdan, A. G. Bus, S. Major, & H. L. Swanson (Eds.), APA educational psychology handbook (Vol.3): Application to learning and teaching (pp. 403–425). Washington, D.C.: American Psychological Association. http://dx.doi.org/10.1037/13275-016
Loyens, S. M. M., Magda, J., & Rikers, R. M. J. P. (2008). Self-directed learning in problem-based learning and its relationships with self-regulated learning. Educational Psychology Review, 20(4), 411–427. http://dx.doi.org/10.1007/s10648-008-9082-7
Mamede, S., Schmidt, H. G., & Norman, G. R. (2006). Innovations in problem-based learning: What can we learn from recent studies? Advances in Health Sciences Education, 11(4), 403–422. http://dx.doi.org/10.1007/s10459-006-9018-2
Marquardt, M. J. (1999). Action learning in action: Transforming problems and people for world-class organizational learning. Palo Alto, CA: Davies-Black Publishing.
Miflin, B. M., Campbell, C. B., & Price, D. A. (2000). A conceptual framework to guide the development of self-directed, lifelong learning in problem-based medical curricula. Medical Education, 34(4), 299–306. http://dx.doi.org/10.1046/j.1365-2923.2000.00564.x
Molleman, E., Nauta, A., & Buunk, B. P. (2007). Social comparison-based thoughts in groups: Their associations with interpersonal trust and learning outcomes. Journal of Applied Social Psychology, 37(6), 1163–1180. http://dx.doi.org/10.1111/j.1559-1816.2007.00207.x
Neufeld, V. R., & Barrows, H. S. (1974). The “McMaster Philosophy”: An approach to medical education. Journal of Medical Education, 49(11), 1040–1050.
Newman, M. (2006). Fitness for purpose evaluation in problem based learning should consider the requirements for establishing descriptive causation. Advances in Health Sciences Education, 11(4), 391–402. http://dx.doi.org/10.1007/s10459-006-9017-3
Pease, M. A., & Kuhn, D. (2011). Experimental analysis of the effective components of problem-based learning. Science Education, 95(1), 57–86. http://dx.doi.org/10.1002/sce.20412
Phan, H. P. (2009). Exploring students’ reflective thinking practice, deep processing strategies, effort, and achievement goal orientations. Educational Psychology, 29(3), 297–313. http://dx.doi.org/10.1080/01443410902877988
Pieterse, A. N., van Knippenberg, D., & van Ginkel, W. P. (2011). Diversity in goal orientation, team reflexivity, and team performance. Organizational Behavior and Human Decision Processes, 114(2), 153–164. http://dx.doi.org/10.1016/j.obhdp.2010.11.003
Poortvliet, P. M., Janssen, O., van Yperen, N. W., & van de Vliert, E. (2007). Achievement goals and interpersonal behavior: How mastery and performance goals shape information exchange. Personality and Social Psychology Bulletin, 33(10), 1435–1447. http://dx.doi.org/10.1177/0146167207305536
Raelin, J. A. (1997). Individual and situational precursors of successful action learning. Journal of Management Education, 21(3), 368–394. http://dx.doi.org/10.1177/105256299702100308
Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Thousand Oaks, CA: Sage Publications, Inc.
Reiser, B. J. (2004). Scaffolding complex learning: The mechanisms of structuring and problematizing student work. Journal of the Learning Sciences, 13(3), 273–304. http://dx.doi.org/10.1207/s15327809jls1303_2
Rotgans, J. I., & Schmidt, H. G. (2011). Cognitive engagement in the problem-based learning classroom. Advances in Health Sciences Education, 16(4), 465–479. http://dx.doi.org/10.1007/s10459-011-9272-9
Schmidt, H. G. (1983). Problem-based learning: Rationale and description. Medical Education, 17(1), 11–16. http://dx.doi.org/10.1111/j.1365-2923.1983.tb01086.x
Schmidt, H. G., & Moust, J. H. C. (2000). Factors affecting small-group learning: A review of the research. In D. H. Evensen & C. E. Hmelo (Eds.), Problem-based learning: A research perspective on learning interactions (pp. 19–52). Mahwah, NJ: Lawrence Erlbaum.
Schmidt, H. G., Rotgans, J. I., & Yew, E. H. J. (2011). The process of problem-based learning: What works and why. Medical Education, 45(8), 792–806. http://dx.doi.org/10.1111/j.1365-2923.2011.04035.x
Schmidt, H. G., van der Molen, H. T., te Winkel, W. W. R., & Wijnen, W. H. F. W. (2009). Constructivist, problem-based learning does work: A meta-analysis of curricular comparisons involving a single medical school. Educational Psychologist, 44(4), 227–249. http://dx.doi.org/10.1080/00461520903213592
Seibert, S. E., Silver, S. R., & Randolph, W. A. (2004). Taking empowerment to the next level: A multiple-level model of empowerment, performance and, satisfaction. Academy of Management Journal, 47(3), 332–349. http://dx.doi.org/10.2307/20159585
Senko, C., Hulleman, C. S., & Harackiewicz, J. M. (2011). Achievement goal theory at the crossroads: Old controversies, current challenges, and new directions. Educational Psychologist, 46(1), 26–47. http://dx.doi.org/10.1080/00461520.2011.538646
Sherwood, A. L. (2004). Problem-based learning in management education: A framework for designing context. Journal of Management Education, 28(5), 536–557. http://dx.doi.org/10.1177/1052562904265773
Sitzmann, T., Ely, K., Brown, K. G., & Bauer, K. N. (2010). Self-assessment of knowledge: A cognitive learning or affective measure? Academy of Management Learning and Education, 9(2), 169–191. http://dx.doi.org/10.5465/AMLE.2010.51428542
Smith, G. F. (2005). Problem-based learning: Can it improve managerial thinking? Journal of Management Education, 29(2), 357–378. http://dx.doi.org/10.1177/1052562904269642
Sockalingam, N., Rotgans, J. I., & Schmidt, H. G. (2011). The relationships between problem characteristics, achievement-related behaviors, and academic achievement in problem-based learning. Advances in Health Sciences Education, 16(4), 481–490. http://dx.doi.org/10.1007/s10459-010-9270-3
Sockalingam, N., & Schmidt, H. G. (2011). Characteristics of problems for problem-based learning: The students’ perspective. Interdisciplinary Journal of Problem-based Learning, 5(1), 6–33. http://dx.doi.org/10.7771/1541-5015.1135
Svinicki, M. D. (2007). Moving beyond “it worked”: The ongoing evolution of research on problem-based learning in medical education. Educational Psychology Review, 19(1), 49–61. http://dx.doi.org/10.1007/s10648-006-9040-1
van Berkel, H. J. M., & Schmidt, H. G. (2000). Motivation to commit oneself as a determinant of achievement in problem-based learning. Higher Education, 40(2), 231–242. http://dx.doi.org/10.1023/A:1004022116365
van den Hurk, M. M., Dolmans, D. H. J. M, Wolfhagen, I. H. A. P., & van der Vleuten, C. P. M. (2001). Testing a causal model for learning in a problem-based curriculum. Advances in Health Sciences Education, 6(2), 141–149. http://dx.doi.org/10.1023/A:1011402507476
van Knippenberg, D., & Schippers, M. C. (2007). Work group diversity. Annual Review of Psychology, 58(1), 515– 541. http://dx.doi.org/10.1146/annurev.psych.58.110405.085546
Walker, A., & Leary, H. (2009). A problem based learning meta analysis: Differences across problem types, implementation types, disciplines, and assessment levels. Interdisciplinary Journal of Problem-based Learning, 3(1), 12–43. http://dx.doi.org/10.7771/1541-5015.1061
Wexley, K. N., & Baldwin, T. T. (1986). Posttraining strategies for facilitating positive transfer: An empirical exploration. Academy of Management Journal, 29(3), 503–520. http://dx.doi.org/10.2307/256221
Wirkala, C., & Kuhn, D. (2011). Problem-based learning in K–12 education: Is it effective and how does it achieve its effects? American Educational Research Journal, 48(5), 1157–1186. http://dx.doi.org/10.3102/0002831211419491