Linking Essential Learning Outcomes and Interprofessional Collaborative Practice Competency in Health Science Undergraduates

Assessing student learning outcomes and determining achievement of the Interprofessional Collaborative Practice (IPCEP) Core Competency of Values/Ethics in a generic pre-professional Bachelor of Science in Health Science (BSHS) program is challenging. A course level Student Learning Outcome (SLO) is: “....articulate the impact of personal values and professional ethics in healthcare decision-making”. A program level terminal learning outcome is to “....critically discuss the interface of values/ethics on health outcomes”. One university level Essential Learning Outcome (ELOs) that all baccalaureate students are expected to achieve by graduation is Ethical Reasoning. This was equivalent to the IPCEP Values/Ethics core competency. This paper describes a strategy to simultaneously measure the Values/Ethics competency at course, program, and university levels. A narrative analysis (n=94) using required ethical decision making BSHS student papers was conducted to determine achievement of SLOs/ELOs and the IPCEP Values/Ethics core competency. Eleven items in the grading criteria were linked to outcome criteria for university ELO competency. A point value was assigned to each of these items using a scoring rubric indicating level of achievement. Results indicated that most students were at the Skilled level for the majority of students, and demonstrated adequate achievement of university, program, and course level learning outcomes as well as achievement of the IPCEP Values/Ethics core competency.

The purpose of this project was to link university level Essential Learning Outcomes (ELOs) related to Ethical Reasoning with Program Learning Outcomes(PLOs) and Student Learning Outcomes (SLOs) and the Interprofessional Collaborative Practice Core Competency of Values and Ethics in an introductory baccalaureate level health science course.Assessment of a course level assignment using a rubric was used to determine mastery.The course is part of the Bachelor of Health Science (BSHS) Program in the School of Health sciences in a mid-Atlantic public liberal arts college with an enrollment of 8570 undergraduate and graduate students.

Background
Outcome based models of education are centuries old.One example is the apprentice-ship method common in the Middle Ages.Outcome-Based Education "means clearly focusing and organizing every-thing in an educational system around what is essential for all students to be able to do successfully at the end of their learning experiences.This means starting with a clear picture of what is important for students to be able to do, then organizing curriculum, instruction, and assessment to make sure this learning ultimately happen" (Spady, 1994, p. 12).Several conceptual models of instruction focus on ELOs and student competence (Cydis, et al., 2015).Malan (2000) described characteristics of the outcome based education.It is needs based, and outcome driven.It has a design down approach.It specifies outcomes and levels of outcomes.Malan (200) also stated that criterion based assessment of mastery or competencies is integral to outcomes based education.The focus is shifted from teaching to learning.The frame work is holistic in its outcomes focus, meaning that achievement of outcomes is not enough: they must be built upon continually.
While the idea of outcome based learning is not new, the current trend toward instituting college and university-wide ELOs that provide an overriding structure aimed at providing a comprehensive, cohesive education across disciplines was influenced by the American Association of Colleges and Universities (AAC&U).The AAC&U launched the Liberal Education & America's Promise (LEAP) program in 2005 to respond to the demands of the twenty first century for more college educated, engaged, and informed citizens through liberal education (AACU, n.d.).The LEAP initiative is organized around a group of ELOs that are thought to be best developed through a liberal education (AAC&U, 2007).LEAP developed Valid Assessment of Learning in Undergraduate Education (VALUE) rubrics with which to assess these ELOs on a campus wide basis.
Our university developed their own ELOs based on AAC&U recommendations and drawn from teaching experience of our faculty.ELOs to be achieved during student experience at the university were designed to help all students "focus on the intellectual and marketable talents needed to prepare for personal and professional success in the 21st century" (Stockton University, n.d.).Several conceptual models of instruction focus on ELOs and student competence (Cydis, et al., 2015).Cydis, et al. (2015) described the integration and assessment of ELOs.They conducted mixed methods research and developed a model of ELO integration based on data collected at our university.They identified several components of ELO integration that converged to support student competence.These included the following concepts: Connection -involves students making associations among ELOs and understanding the process of change over time; Awareness -emphasizes the importance of metacognition regarding learning how to learn which in turn, supports lifelong learning; Facilitation-involves faculty facilitating ELO acquisition by clearly identifying learning outcomes in coursework and during advising to develop student metacognitive awareness about the criteria for competence and to foster life-long learning; Intention -supports learning through articulating learning activities that offer opportunities to develop specific ELO competencies in syllabi and course aims; Utility -integration of specific ELO concepts and competencies within the course, among other courses, and in the real world to provide a sense that these competencies and concepts are meaningful; Reflection -providing opportunities to intentionally reflect upon their learning experiences along with meaningful learning experiences and specific evaluation criteria helps students to build awareness and learn more about their specific competencies.This model is useful for integrating ELOs into "courses, assignments, and co-curricular activities" (Cydis,et al.,p. 47).The current project illustrates how university level ELOs can be linked to program level outcomes (PLOs), and course outcomes (SLOs).Using a specific learning opportunity in the form of a multifaceted student learning experience operationalizes the concepts of ELO integration identified by Cydis and colleagues (2015).
ELOs can "provide direction to faculty in designing courses, but to understand whether competency frameworks are affecting student learning outcomes, there must be some way to assess whether students can demonstrate the competencies" (Klein-Collins, 2012, p. 30).LEAP developed rubrics for assessment and several competency based college and university programs are approaching competency assessment through the use of rubrics (Klein-Collins, 2012)."A rubric is typically an evaluation tool or set of guidelines used to promote the consistent application of learning expectations, learning objectives, or learning standards in the classroom, or to measure their attainment against a consistent set of criteria" (Hidden Curriculum, 2014, para 1).Rubrics provide transparency and consistency in evaluation, clearly define expectations for student achievement before learning experiences occur, and allow for more specific, detailed feedback to students regarding their level of achievement.Rubrics also provide an opportunity for students to reflect upon and evaluate their own learning (Klein-Collins, 2012;Hidden Curriculum, 2014).Rubrics typically include the educational purpose of the assignment and links to appropriate concepts or learning objectives.They also include specific criteria that will be evaluated to demonstrate competence or achievement.Perhaps the most important component of a rubric is explication of specific quality standards that the instructor or evaluator will use in scoring, grading or evaluating the student's performance."For example, if the evaluation scale is scored "on a scale of 1 to 4, the rubric may detail what students need to do or demonstrate to earn a 1, 2, 3, or 4. Other rubrics will use descriptive language-does not meet, partially meets, meets, or exceeds the standard, for example-instead of a numerical score" (Hidden Curriculum, 2014, para 2).

Linking Levels of Learning Outcomes and Competencies
The university level ELO defines Ethical Reasoning as: "the recognition of an ethical dimension in a situation, that leads to critical examination of the potential consequences of your (in)actions based on consideration of alternate viewpoints and multiple principles and their implications".Students are expected to gain knowledge in personal and social responsibility, including Ethical Reasoning throughout their college education, at successively higher levels.
The In the university level ELO of Ethical Reasoning, one criterion was specific to ethical reasoning in dealing with moral dilemmas.This ELO linked to specific program and course outcomes as follows (see figure 1): ELO: Demonstrate ethical reasoning when faced with moral dilemmas in a variety of real-world contexts PLO: Critically discuss the interface of value/ethics on health outcomes SLO: Articulate the impact of personal values and professional ethics in healthcare decisionmaking.
Figure 1: Linking Essential Learning Outcomes

Course Assignment
In the second of two introductory BSHS program courses, one of the course student learning objectives (SLOs ) is to articulate the impact of personal values and professional ethics in healthcare decision-making.A learning activity designed to help meet this objective involved a multifaceted case study assignment.Students were given a case study that required a multidisciplinary healthcare team to make an ethical decision.Students formed groups of no more than four members.A variety of cases were provided from which each group chose one.Students in each group assumed the role of a specific member of the healthcare team, such as a physician, nurse, occupational therapist, or family member.They had to look up the codes of ethics of the professional they represented and any other pertinent legal or ethical standards.In class, several ethical decision making models were presented, and student groups were required to use one to work through the ethical issues and come to a decision.Using the case study, they explored the relationship of values and ethics to making a healthcare decision.Students had to identify their personal values as well as those of the profession they represented and explore any conflict between the two.They also had to identify and resolve any conflicts within the group in order to come up with a resolution that satisfied all members.Each group created a presentation that was given to the class detailing the ethical dilemma and considerations, the decision they made, why they thought it was the best decision, and the process by which they arrived at the resolution.Each student wrote an individual scholarly paper with a reflective component based on this experience.

Assessment Procedure
Review of the ethical case study assignment criteria revealed that they linked quite clearly with the levels of competency in the Stockton ELO.After linking university ELOs, program PLOs, and course SLO's, we chose to analyze the student case based ethical reasoning papers to evaluate the competency level at which students were operating by the end of the course.Student papers were received from all but one instructor and represented 3 of 5 sections of the course.Approx.160 students enrolled in HLTH 1102 during the time of this but 70 were inadvertently discarded by an instructor of 2 sections of the course.A total of 94 student papers were included in the analysis.This represented 59% of the total semester enrollment in the course, and 3 of 5 sections taught.University IRB approval was obtained prior to analysis.
Papers were analyzed by a faculty member who did not know the students or teach the course.Papers were analyzed after the semester was over and did not affect or reflect student grades in the course or the assignment.The analysis was used to help determine how well students met the ELOs, PLOs, and SLOs related to Ethical Reasoning and the IPCEP Core Competency of Values and Ethics.A scoring rubric was developed based on the assignment guidelines which mirrored the college level ELO.See Figure 2 for the university competency levels of the Ethical Reasoning ELO.
Five items in "Introduction" section guidelines of the student paper were clearly indicative of the "Aware" level of competency according to criteria in Stockton's Ethical decision making ELO.Each of these items were scored as either 0-absent or 1-present.1. Provides a brief summary of the facts of the case, including identifying the health problem(s), case needs, and key players.2. Clearly identify the ethical dilemma(s) in the case 3. Present your personal moral positions in relation to the ethical dilemma(s).4. Present the ethical position of the profession you are representing in the IP team.Make sure to research and cite the Code of Ethics or your assumed professional role.You can also cite articles discussing your assumed profession's position in this issue.5. Describe divergent moral (e.g., from patient, family) and/or ethical (i.e., professionals) positions in the IP group.If applicable, discuss the role and moral position in the group of patient, family, and friends.

Figure2: Stockton University Essential Learning Outcomes: Ethical Reasoning
There were 3 items in the Ethical Decision Making section of the paper guideline that linked to the ELO.These were analogous to the "Competent" level in the ELO.These were rated from 0 to 2 with 0 indicating not met, 1 indicating partially met, and 2 indicating completely met.1. Describe completely at least two (2) recommendations discussed in the group, including their pros and cons, and anticipated outcomes.2. Describe how you & your team came to a decision between options considered in #2. 3. Describe the agreed upon course of action and why it was the best option (support the reasons you give why it was the best option) Three items in the reflection section of the paper were linked to the "Skilled" level of the university ELO and were scored from 0 to 3, with 0 indicating not met, 1 indicating minimally met, 2 indicating partially met and 3 indicating completely met.1. Reflect and discuss agreements and/or disagreements between your personal moral position and the ethics of the profession you were representing (intrapersonal conflict).2. Reflect and discuss agreements and/or disagreements between you (personal morals and professional ethical position) and those of other group members (interpersonal conflict).3. Assess how well you feel you understood you own moral position in relation to the ethical dilemma.Assess how well you feel you understood the ethical position of the profession you were representing.Which one you understood better?How your understanding of each affected your performance in the group.
A final item from the paper was linked to the Knowledge Transfer criterion of the ELO, representing the highest level of learning.It was scored from 0 to 3 as in the previous section.1. Summarize what this learning experience was like for you personally and in your future professional role.For example, how this experience may impact you if you experience a similar situation in your personal life and in your future professional role.Separate subscale totals were calculated for each of the 3 subscales representing Aware, Competent, and Skilled level criteria, plus one for Knowledge Transfer.

Assessment Results
Using the narrative scoring rubric scores, means for each of the subscales were calculated.Relative scores were computed as percentage of the maximum possible score achieved.The mean score for the Aware level subscale was 4.67 (93.40%) out of a possible 5 points.The Competent subscale mean was 5.48 (91.33%) of a possible 6 points, and the Skilled subscale mean was 7.79 (86.55%) with 9 points as the maximum achievable score.The overall score was computed by adding the subscale scores for a maximum of 20 points.The overall mean score was 17.94 (89.70%).Results are displayed in Table 1.Awareness, Reflection, and Utility were demonstrated through the requirements of the assignment that students identify and reflect upon their own values and ethics in relation to those of others, reflect upon the process by which a decision was reached, and apply learning by role playing in a case study example similar to what may be encountered in the real world.The assignment itself facilitated mastery.The concept of Connection is illustrated by linking ELOs.PLOs, and SLOs, and in the intentional weaving of the competency of Values and Ethics throughout the BSHS curriculum as a course and program outcome.
Care must be taken when appraising the results reported here since narrative analysis of student papers using a rubric is not a precise quantification.The rubric used in scoring is not nor was it intended to be a research instrument with precise psychometric qualities.It does provide an authentic assessment of student achievement of learning outcomes that can be used in course and program assessment related to Values/ Ethics.It was an appropriate choice for this project since it was used to measure the attainment of learning objectives against a consistent set of criteria (Hidden Curriculum, 2014).
This strategy could be combined with a standardized objective measure of Values/ Ethics in a mixed methods design.This strategy has potential for use in assessing a variety of Student Learning Outcomes if closely linked with course, program, & college outcomes.
Core Competencies of Interprofessional Collaborative Practice (Interprofessional Education Collaborative Expert Panel [IPCEP], 2011) serve as the theoretical framework of the BSHS program curriculum.One of these competencies is Values and Ethics.The BSHS Program Mission is to provide a quality, contemporary, pre-health professional education for students who desire careers in the healthcare system.The program curriculum consists of eight core course plus Reed, Garcia, Slusser, Konowitz, and Yep Journal of the Scholarship of Teaching and Learning, Vol.17, No. 1, February 2017.josotl.indiana.edu18 several cognates and electives depending on the individual student's educational goals and focus.There are 3 concentrations that prepare students for specific graduate programs such as physical therapy, plus a general concentration that prepares students for direct entry into the healthcare system or to pursue a variety of graduate programs.Current enrollment in the BSHS program is 1111 undergraduate students.The BSHS program is the largest program in the university.One of the program outcomes (based on the IPCEP core competency of Values and Ethics) was that students critically discuss the interface of value/ethics on health outcomes.

Table 1
Narrative Analysis Rubric ScoresResults indicated that the majority of students achieved desired course and program outcomes related to Ethical Decision Making.The course level objective related to the core competency of Values and Ethics was fully met.Most students achieved the skilled level of the University ELO mid-way through program at the end of the second Introduction to Health Science course.Role playing and reflection were successful strategies in helping to achieve desires learning outcomes.Using a rubric to score student's competency when analyzing papers on ethical decision making provided an accurate and authentic evaluation of student learning that can be used in determining how well university level, program level, and course level learning outcomes were achieved.Linking program and course level learning outcomes to university level ELOs was essential in the process.Results were as expected given students' level of progression in the program and the university.The university ELO that students "Possess the ability to transfer the above skill to multiple areas such as personal, professional, and societal" was only partially met.This represents the highest level of learning.It is not possible to accurately measure this ELO in pre-professional students who do not have supervised clinical experiences Given students' level in program, program goals related to values and ethics adequately achieved.These students have one more core program course in which Values and Ethics are included as an objective.Evidence from this assessment project suggests that students should fully achieve program outcomes in Values and Ethics by completion of the final core course.Our results suggest that most BSHS students should fully meet criteria for "Skilled" level in Ethical Decision Making according to college level ELO criteria by graduation.Linking the university, program, and course level ELOs illustrates the concepts of Intention and Facilitation in Cydis and colleagues' (2015) model of ELO integration.The concepts of Reed, Garcia, Slusser, Konowitz, and Yep Journal of the Scholarship of Teaching and Learning, Vol.17, No. 1, February 2017.josotl.indiana.edu22