Assessing interdisciplinary learning and student activism in a Water issues course

In response to a request from a campus student organization, faculty from three fields came together to develop and teach an integrated interdisciplinary course on water issues and social activism. This course, “Water as Life, Death, and Power,” brought together issues from the fields of anthropology, biology and chemistry to explore water rights, access to clean water, and water treatment methods. Students enrolled in the course developed interdisciplinary projects related to a variety of local and global water issues to present real-world solutions at a university-wide student research showcase. This article reports the assessment outcomes of the course, measuring changes in both interdisciplinary learning and levels of student activism.


Background
Universities Allied for Essential Medicines (UAEM) is a coalition of undergraduate, graduate and professional students at academic institutions worldwide dedicated to providing global access to affordable medicines.The student group at Central Michigan University (CMU) indicated that they are interested in undergraduate courses that combined interdisciplinary teaching with solving real world problems, combining theory with activism.Three CMU UAEM faculty advisors took up the challenge to develop such a course: Stephen Juris (Biology); Anja Mueller (Chemistry); and Cathy Willermet (Anthropology).We decided to develop a course that would bridge all three disciplines around a complex problem and encourage both interdisciplinary thinking and activism in our students.

Interdisciplinary Teaching and Learning
The students requested an interdisciplinary course as a result of their involvement with UAEM, personal and professional interests, and because they understood that the complex problems their generation will have to solve would require people from different disciplines to work together and come up with a complex solution.In addition to the advantages of interdisciplinary learning identified by the students, researchers (e.g., Begg and Vaughan, 2011;

Interdisciplinary Course Goals and Objectives
Student learning outcomes (SLO) are sometimes challenging to assess effectively in an interdisciplinary freshmen course; for students to gain truly interdisciplinary understanding to a point that they can apply it to solve an interdisciplinary problem in a group setting, they have to first gain several skills, such as working in a group, and then synthesize and combine information from different disciplines.Since this class is designed for freshmen/sophomores, it has to be assumed that these skills need to be taught during the class as well.Thus, assignments and grading rubrics needed to consider how students will demonstrate their attainment of the SLOs related to interdisciplinary understanding in not only the final product, but the various steps that lead to this outcome.
In addition to the SLO focused on water issues, we identified two additional overarching goals for the course: 1) developing interdisciplinary thinking rather than focusing specifically on content; and 2) encouraging students to engage in actively solving current, real-world problems in an interdisciplinary way.(See the Master Course Syllabus, Appendix 1).We considered collaborative learning to be an essential goal to allow students to see how real-world complex problems can be solved in real-life.
We hypothesized that: 1) students would increase their knowledge about water and waterrelated issues, such as water chemistry, water-borne pathogens, and global access to clean water; 2) students would increase their desired level of social activism; and 3) students would increase their interdisciplinary thinking.These hypotheses guided our assessment efforts, as described below.

Research Design and Methods
Our research design included two separate assessment strategies: a pre-post survey to address hypotheses one and two, and an interdisciplinary project to address hypothesis three.
We obtained Internal Review Board (IRB) approval (CMU 377609-2) to collect student data assessing whether students increased their competency in interdisciplinary thinking, as well as increased their knowledge of activism and human rights.The Internal Review Board approval extended to administration of a pre-and post-course survey and application of a rubric to specific group-assigned course activities to assess interdisciplinary thinking.On the first day of class, we invited interested students to join us in a research study that would help assess how well they learned about water issues, their level of activism, and degree of interdisciplinary thinking.Students received a manila envelope that contained two copies of the consent form, a bubblesheet response form, and two surveys, the research survey and a similar-looking alternate survey.If students wished to participate, they signed a consent form and completed the research survey; if not, they completed the alternate.Both surveys and bubble sheet were returned to the envelope.One author, Eron Drake, acted as the project's "honest broker."She assigned each student a randomly generated three-digit code and kept the key of student names and keys in a secure, locked location.The instructional team does not know which students participated in the study; students received the same number of points for completing either survey.
There were 29 students that registered and completed the course.Of the 29 students registered for the course that ultimately completed the course, 12 were male and 17 were female.These students registered for the course in one of three disciplines (anthropology, biology, and chemistry); 15 students registered under the anthropology designator, 13 students registered under the biology designator, and 1 registered under the chemistry designator.Twenty-eight students completed the pre-and post-test associated with the research project.
Interdisciplinary thinking can be difficult to assess through objective means such as multiple-choice exams.Rather, interdisciplinary thinking can be better assessed through projects, essays, and discussion.To that end, we assigned a semester-long group project for which students chose a water-related problem and developed an interdisciplinary solution and a strategy for implementation.The proposed solution had to include perspectives from anthropology, biology, and chemistry.We decided to break up the interdisciplinary project development process into several steps; we needed to start groups out with a solid, disciplinary foundation for their project, before we could start them on the steps to integrate that information by bridging the concepts, integrating them into a complex discussion and finally into an interdisciplinary solution to the problem (Mansilla & Duraisingh, 2007).In this manner, the students could practice and improve their interdisciplinary understanding and implementation strategy.
We started with a group contract, to make sure that all students understood their role in the group and could solve problems within the groups more easily.One of the authors (Eron Drake) prepared the students for group work and group contract by presenting them with information about group formation, group roles, and group expectations, and giving them examples for group contracts.Students next completed a problem statement so that the groups had to decide early what exactly to work on.This problem statement needed to include how the three disciplines would be part of the solution.Students were also taught how to search for materials for their project in the library.The material was mostly disciplinary and part of the disciplinary grounding for the project.
Groups were then asked to complete a concept map to develop the connections between the different fields in relationship to their specific problem solution.Building this concept map allowed the students to bridge the different concepts into a first step towards interdisciplinary understanding.The next step was a short, persuasive pitch and an abstract to make sure the groups stayed focused and provide them a means to practice how to present their work.The final project was a poster presentation of their final complex solution strategy for a complex water problem at a campus-wide event.The final project included an interdisciplinary discussion of the problem, as well as the integrative solution the students came up with.At the same time students had to present and pitch their solution to the "general public" as any activist would have to do.We met with the groups at each stage to give them maximum feedback and opportunity for questions.
We were concerned that a heightened interest in assessing interdisciplinarity would bias us to see it more often than students were in fact presenting it.To reduce this bias, we employed a grading rubric for any assignments that required subjective assessment (see below and Appendix 2).The assessment of the group projects for interdisciplinary understanding was developed according to the steps in student learning (Mansilla & Duraisingh, 2007).The first step in this process is to have an effectively presented disciplinary argument (Disciplinary Grounding).To assess disciplinary grounding, we modified our assessment using the Universal Intellectual Standard developed by Drs.Paul and Elder from the Center for Critical Thinking (Elder & Paul, 2013).When we graded the interdisciplinary assignments, each faculty evaluated students for this section based on their discipline.We based the interdisciplinary part of the rubric on Mansilla and Duraisingh's snapshots of interdisciplinary integration (Mansilla & Duraisingh, 2007).We used integrative summary, conceptual bridging, and complex explanation as the three consecutive steps of interdisciplinary understanding in our rubric.
The group project in this class asked specifically for a solution of a water-related problem.Therefore we needed an additional part for our rubric assessing the pragmatic solution the students proposed.We based the evaluation on Six Sigma, which was invented by a Motorola researcher and is used in industrial project evaluations (Motorola University, 1994).Our guest lecturer, Keith Helferich, presented the basis of Six Sigma to our students as several steps that have to be completed for a successful project: Define (plan), measure (do), analyze (review performance, identify opportunities, root causes, and effects), improve (prioritize actions to enhance performance), and control (implement and establish future assessment program).We wrote the solution assessment on these five steps.The full rubric can be found in Appendix 2.
As mentioned above, students would have to learn all of the steps outlined in the Interdisciplinary Assessment Rubric (Appendix 2) during the class, which takes careful planning and the allotment of time-on-task to enhance student learning.Instruction must be scaffolded to allow for students to develop and practice higher-level cognitive skills associated with interdisciplinary learning.Therefore, we decided to use the rubric in the evaluation of many of the project assignments, but we weighed the three parts (disciplinary grounding, interdisciplinary reasoning, and pragmatic solution) differently throughout the semester.We first weighed disciplinary grounding more heavily than interdisciplinary reasoning, next weighed them equally, and at the end weighed disciplinary grounding least and the pragmatic solution most.Using this redistribution of weighting, we accounted for the increasing interdisciplinary understanding throughout this course.
Two specific assignments provided an excellent opportunity to evaluate student gains in interdisciplinary thinking.The first was the concept map that each group drew at the beginning of their project, to describe how the disciplines would interact in their proposed research.The second was the final presentation that described their problem and proposed solution.We assessed the interdisciplinary understanding at these two stages of their group projects using our interdisciplinary rubric.All instructors of the course separately utilized this rubric to grade every assignment.We then averaged the grades over all instructors to finalize the assignment grade.When disciplinary grounding needed to be established, each of the instructors with expertise in the questioned disciplinary grounding provided guidance on grading criteria.

Goal 1: Increased Knowledge of Water-Related Issues
The student pre-post survey contained questions designed to measure overall student factual knowledge of water-related issues.Students were asked nine questions to assess their overall knowledge of water-related material.The source of the questions was the course textbook.We used the textbook mostly as a reference, with content provided from the lecture materials and supplemental readings.Therefore these questions were not a direct measure of specific fact retention.
To analyze whether factual answers improved over the semester, data were analyzed statistically in R (version 3.0.1)(R, 2013) using a generalized linear model with a binomial error distribution.Calculated probability values were deemed significant with a = 0.05 using a sequential Bonferroni adjustment for each question.

Goal 2: Increased Student Interest in Social Activism
The student pre-post survey also contained questions designed to measure a change in student familiarity with water-related issues and their interest in activism.This evaluation included questions about the student's awareness of water issues, to assess the student's personality, the student's interest in volunteering within the university, and about the civic engagement and social awareness of the student.
These questions were posed on a five-point Likert scale.On questions with a Likert scale, increasing values might mean a decrease or increase of improvement, depending how the question was asked.For the statistical analysis, all Likert scales were adjusted so that increasing values meant improvement.Student responses were categorized into three different groups, students who agreed with a statement (i.e.answered 4 or 5 on the Likert scale), students who disagreed with a statement (i.e.answered 1 or 2 on the Likert scale), or students who were neutral about a statement (i.e.answered 3 on the Likert scale).Student responses were paired pre-post, and changes in student opinion in a positive (disagree/neutral stance pre-test to agree stance post-test) or negative (agree stance pre-test to disagree/neutral stance post-test) were analyzed using a McNemar's test and calculating chi-squared.A p-value was obtained using one degree of freedom, and a value of p < 0.05 was interpreted as a significant change comparing pre-and post-test data.

Goal 3: Increased Interdisciplinary Thinking
To measure if the interdisciplinary reasoning of the students increased from an earlier assignment (concept map) to the final assignment (poster presentation), we compared student performance on the interdisciplinary sections of the rubric as applied to these two assignments (See Appendix 2).For each of the assignments, assigned points and weights differed due to the nature of the individual assignment.For example, points were given for staying within a 3minute time limit for the persuasive pitch; the final presentation included a self-and peerassessment.We needed to remove the effects of these points on the assignment grade, to isolate points related to interdisciplinary understanding and problem solving.To exclude the effect of all other rubric sections and other points that were included in the grade, it was assumed that the students obtained full points for everything but the interdisciplinary section of the rubric.These points were averaged across all groups.We then compared the remaining points assigned exclusively for interdisciplinary reasoning.This technique should, if anything, underestimate the students' performance on interdisciplinary learning.Since the data were organized in this way, we did not perform a statistical analysis, but rather calculated the mean group performance on this measure between the two assignments.

Results and Discussion
The following discusses the results obtained in exploration of our hypotheses: 1) students would increase their knowledge about water and water-related issues, such as water chemistry, water-borne pathogens, and global access to clean water; 2) students would increase their desired level of social activism; and 3) students would increase their interdisciplinary thinking.

Goal 1: Increased Knowledge of Water-Related Issues
The data reported (Table 1) indicate that students gained a deeper overall knowledge of water-related material, although the overall increase is not significant and performance on three of the questions decreased.There was a statistically significant improvement on both the question connected to water required to make one calorie of food (21% increase in correct responses during post-test compared to the pre-test) and the question connected to which food type takes the most water to produce one kilogram of food (18% increase in correct responses during post-test compared to the pre-test).There was decline on questions related to water access, which saw a 7% decline in correct answers; however this decrease was not statistically significant.

Goal 2: Increased Student Interest in Social Activism
The student pre-post survey contained questions designed to measure overall student interest in water-related issues and degree of student activism.Students were asked 48 questions to assess their agreement with statements connected to clean water access and degree of willingness to play a role in university and/or community activism.Questions were rated on a five-point Likert scale, for which one indicates "strong disagreement" and five indicates "strong agreement".Students were pooled into one of two categories depending on their answers to the questions -1) students who answered 1-3 for a question (i.e.disagreed/neutral with the question), and 2) students who answered 4 or 5 for a question (i.e. agreed with the question).The data reported (Table 2) represent percent changes in each of these two categories when comparing pre-test and post-test answers, where a negative change was scored when a student answered 4 or 5 on the pre-test and 1-3 on the post-test, and a positive change was scored when a student answered 1-3 on the pre-test and 4 or 5 on the post-test.Results indicate that students gained a deeper appreciation and understanding of water-related issues.Of note was a statistically significant greater agreement with statements concerning the potential for a water crisis in America (39.3% increase in students agreeing post-test) and a statistically significant stronger agreement on the negative impact of bottled water on the world's water supply (35.7% increase in students agreeing post-test).Furthermore, there was a statistically significant stronger agreement in students when asked whether social problems directly affect the quality of life in their community (32.1% increase in students agreeing post-test) Interestingly, data collected indicate that students changed their potential for civic engagement, and this change seemed to be a refocusing of student energy away from university organizations towards community involvement.There was a trend in students disagreeing with questions asking about plans to become involved in university organizations (although not statistically significant) with a concurrent increase in the importance of civic engagement issues.However, this may be an inadvertent artifact of the question text, which asks, "During this term, to what degree do you intend to…" As a pre-term question, it asks the student what he/she might do in the near future.However, as a post-term question, students might answer in the negative either as the term is over and they don't intend to do it in the next few days, or since they know they didn't in fact do it this term.The wording of this question might not capture student intent in the future.
Every question concerning civic engagement issues showed an increase in agreement with the statements posed, with ten showing statistically significant increases: participating in a community action program (17.9% positive increase post-test, p = 0.0253); helping promote racial understanding (25% positive increase post-test, p = 0.0082); influencing social values (25% positive increase post-test, p = 0.0082); finding a career that directly benefits others (25% positive increase post-test, p = 0.0082); giving some income to those in need (28.6% positive increase post-test, p = 0.0196); becoming a community leader (25% positive increase post-test, p = 0.0339); working toward equal opportunity for all people (17.9% positive increase post-test, p = 0.0253); viewing social issues from multiple perspectives (25% positive increase post-test, p = 0.0082); developing a meaningful philosophy of life (25% positive increase post-test, p = 0.0339); and developing leadership abilities in others (25% positive increase post-test, p = 0.0339).There were correlative increases in other questions including participating in programs to help clean up the environment (25% positive increase post-test), serving the community (17.9% positive increase post-test) and participation in voting (28.6% positive increase post-test), although these increases were not statistically significant.It is interesting to speculate that a shift from focus on involvement in student organizations to involvement in community organizations may be due to a redefined student view on the ability to affect change within the community more directly depending on the organization with which they are involved.The increased dedication of students becoming involved in a community issue was apparent based on the dedication students had of their group projects that they developed throughout the semester (connected to Goal 3 below).Several groups continued to seek outcomes of their projects after the semester had ended and had developed plans to further promote their project agenda through the formal submission of proposals or letters to their corresponding agencies/affected communities in order to affect change.

Goal 3: Increased Interdisciplinary Thinking
The students had free choice of which problem they wanted to solve as collaborative groups.The groups came up with a list of problems and solutions spanning issues at local, regional, and international levels: • Development of a time-release version of an existing anti-worming drug for schistosomiasis in Uganda, along with educational call-and-response children's song on how to avoid getting sick; • A plastic water bottle deposit to promote recycling and tap water usage; • Installation of composting toilets at CMU to reduce water consumption; • Community education on hydrologic fracturing to understand water contamination; • Modification of city green-lawn ordinances to reduce local water contamination through chemical runoff; • Analysis of strategies to connect Iowa farmers to government programs to promote bioswale buffer zones along the Mississippi River, to reduce downriver dead zones; • Proposal to Mayoral Office in Copacabana, Bolivia to design totora reed beds that clean wastewater before it enters Lake Titicaca; • Investigation of water disinfection techniques using solar UV radiation (SODIS) in plastic bottles in Uganda.
For the concept map, 32% of total points were available for interdisciplinary learning based on the rubric in Appendix 2. Students were graded on the concept maps based on development of their solution to a global problem and whether their solution contained the three disciplines associated with the course (anthropology, biology, chemistry) and was sustainable.For the final project, 45% of the points were assigned to interdisciplinary learning.The student groups' performance improved from an average of 40.1% of the total available points for interdisciplinary learning on the concept map to 71.6% of the total available points for the final project.The standard deviation decreased between the two assignments, from 22% to 11.8%, which suggests that student groups as a whole performed more consistently on their final projects.All groups but one experienced a large improvement in performance on the interdisciplinary rubric; the remaining group (Group 5) was the highest performing group, doing very well on both assignments.

Discussion
Besides teaching anthropological, biological, and chemical facts about water, there were two overarching goals for the course that we assessed: 1) developing interdisciplinary thinking rather than focusing specifically on content; and 2) encouraging students to engage in actively solving current, real-world problems in an interdisciplinary way.Real world-problem solving often occurs in groups, combining different strengths and backgrounds.We wanted to mimic that; at the same time, benefiting the learning of diverse students in collaborative assignments and projects is also recognized as a high impact practice (Kuh, 2008).We encouraged the students to reach out to NGOs and other community groups in the process; this community connection, as well as the service rendered with the project, is considered a high-impact teaching process as well.
As discussed above, the fact-based questions might not have been an accurate measure for students' content knowledge.Upon reflections, questions aligned with the content covered would better reflect content learning.The end-of-term Student Opinion Survey comments for each faculty did indicate that factual learning took place.Examples for the question "What are some specific things your instructor does that help you learn in this course?"are "Helped fill in knowledge for "non-chemists;" "Powerpoints -helped me learn Chemistry I had never understood before;" "There were always good examples and explanations on subjects covered for the Anthro portion of the class;" "Helped me understand biology that I had never learned before;" "For the non-bio student he explained things well so they were easy to understand."On the students' social activism, the awareness about critical water problems increased significantly.Interestingly, the willingness to participate socially in the university decreased (although this may be an artifact of the question wording, as discussed above).On the other hand, becoming active in the community at large increased significantly; in fact, it was the largest change measured.Complete: Includes all disciplinary information needed; lacking none of its parts or aspects thorough, whole.
All disciplinary information needed is presented.
Most of the disciplinary information needed is presented.
Only some of the disciplinary information needed is presented.
None of the disciplinary information needed is presented.

Interdisciplinary Reasoning
Integrative Summary: All disciplinary arguments are distilled into a coherent summary with an overall meaning or result.
All disciplinary information has been included in the summary in a logical manner.
2 disciplines are favored over the 3 rd .
1 discipline is favored over all other disciplines.
No integrative summary is attempted.

Conceptual Bridging:
A particular concept, instrument, skill is used in a variety of concepts resulting in a deeper understanding of the tool itself.
The topic is investigated from the viewpoint of all disciplines, leading to deeper understanding of the topic.
2 disciplines are favored over the 3 rd .
1 discipline is favored over all other disciplines.
No deeper understanding has been achieved.

Complex Explanation:
The The problem was not solved in an interdisciplinary manner or did not include 6σ processes.

Table 1
Student familiarity with material questions related to water (N=28)

Table 2
Student Familiarity with Clean Water Issues and Willingness to Participate in Social Activism (N=28) "Please

Table 3
Comparison of Student Group Means for Interdisciplinary Learning Portion of Group Projects, as Percentage of Total Interdisciplinary Learning Points Possible (N=8) Mueller, A., Juris, S. J., Willermet, C., Drake, E., Upadhaya, S., & Chhetri, P.Journal of the Scholarship of Teaching and Learning, Vol.14, No. 2, May 2014.
Journal of the Scholarship of Teaching and Learning, Vol.14, No. 2, May 2014.josotl.indiana.edu