Learning to Learn: Lessons from a Collaboration

E-learning has become one of the primary ways to deliver education around the globe. Research is keeping pace with the use of various techno-aids as educators evaluate how to effectively use these aids in an ever-changing e-classroom. Adding to this body of work, and in assessing the effectiveness of techno-tools, this study evaluates meaningful and deliberative exchanges of online discussions towards building an inclusive online classroom. Unknown to each participant were the gender, race, ethnicity, religious beliefs, course level, and mode of instruction of the other students in the study. These unknowns are important in determining how civically engaged participants are in their discussions with each other. Are they creating dialogue and being reflective irrespective of differing instruction types or levels? A secondary focus of this study, is to provide suggestions in constructing purposefully created online elearning communities. This project’s outcomes have important implications in the everdemanding need to design effective online communities.

Online teaching and learning technologies are becoming mainstream as universities and educators reach out to students across domestic and international boundaries.Much of the previous work in this area has centered on hybrid2 and face-to-face classes (Roscoe 2012) and online discussions complementing in-class usage (Pollock, Hamann, & Wilson 2005, Clawson 2013,).While past research points to growing evidence that online discussions are highly effective means of engagement in political science courses (Hamann, Pollock, Wilson, 2009;Clawson, Deen, & Oxley 2002), we know very little about how students perform in online collaborative projects across levels of courses and instruction type online.Comparisons such as these are important concerns as institutions grow and face challenges to develop and technologically innovate their course offerings.This significance is greatest for educators who want to cultivate and promote reflective student interactions globally across any course level or mode of instruction in comparable fields.
Having been involved in eight semesters of these specifically created online collaborations, I examine one such semester of student participation that used asynchronous discussion forums involving students in three universities across three states and time zones.The levels of the courses and the type of instruction differed across the collaboration.Two of the courses were 3000-level courses.The type of instruction in one class was by online delivery, and the other was a face-toface course.The third course was a 2000-level, or lower-division, course with face-to-face instruction.
The purpose of the study was to determine if students are reflective participants despite the differences in instructional type and course level, and if that participation is consistently reflective over the entire term.Reflectivity of student responses to instructor questions were analyzed, not based on their frequency of posts, but for their thoughtful and deliberative comments.Given the ever-demanding need to offer online courses, a secondary purpose is to offer suggestions for creating robust academic collaborations.These are increasingly important as more instructors and universities offer online courses, and these collaborative academic experiences can be shared across comparable disciplines.

Benefits of Online Discussions
Offering structured online discussions provides a number of pedagogical benefits.First, online discussions provide for greater diversity than face-to-face classes (Van Vechten et al. 2013).The nature of online discussions allows the flexibility of connecting students across diverse demographic and cultural environments.Unknown to the student was the gender, race, ethnicity, religion, course level or modes of instruction of the others in the study, yet membership in these highly diverse groups greatly challenges viewpoints and develops an awareness of alternative perspectives which can lead to developing a more reflective understanding of collective problems, a deeper appreciation of minority rights, and greater empathy for others (Guttman 2000).Exposure to and experience with diversity can help students develop skills to handle and resolve disagreements arising from conflicting points of view (Zuniga, Vasques-Scalera, Sevig & Nagda 1997;Gurin, Nagda & Lopez 2004).Second, online discussions continue providing educational opportunities for students while travelling or military deployment, thus reducing interruptions of educational experiences.Third, online discussions encourage critical reflection and dialogue concerning current and theoretical issues in a space and time that is comfortable and familiar to the student.
Because universities and instructors seek to offer online courses that are academically challenging, I focused on whether the reflectiveness of students' posts vary due to the differences in type of instruction, level of course, and over course length.Researching the variability in these differences is crucial.Comparing students' learning through online discussions across course levels and modes of instruction for an entire semester offers a longitudinal view of online discussions and not necessarily as an addition to a course.Given that the demographic characteristics of students in online classes are not readily known, they are more likely to be concerned with the quality of the discussion posts and responses.
Hypotheses: In terms of the level of course, H1: I hypothesized that student reflectiveness in online discussion forums that were a part of the upper-level face-to-face course would be on par with those in lower-division face-to-face courses over the length of the semester.Second, in terms of the instruction of course, H2: I hypothesized, similarly, that student reflectiveness in online discussion forums that were a part of the upper-division online course would be on par with a comparable upper-division face-to-face course over the length of the semester.Instruction level or type of course should not vary in reflectivity as online discussions do offer robust means of learning without regard to either.Third, over the length of the term, H3: I hypothesized that greater student reflectiveness occurs in posts to the professor and response to their peers earlier in the semester, versus semester-end as students leave projects and papers in their classes typically until they are due.Thus, I would expect a slight drop in reflectivity of posts and responses by semester-

Chadha
Journal of the Scholarship of Teaching and Learning, Vol. 17, No. 3, July 2017. josotl.indiana.edu 36 end.Towards this secondary research focus, I examine variations in student reflectivity in their posts and responses.
In addition to measuring student reflectiveness to questions asked by the instructors, I analyzed whether the type of question allowed for variability in the students' responses.Two types of questions were asked: one question focused on current events and one focused on theoretical issues.Fourth in terms of Question Type, H4: I hypothesized that students would perform with reflectivity across any type of question the instructor asked, and more so to current events questions than to theoretical questions.This is likely because students are more attuned to issues of the day and are more comfortable discussing these via other social media, as opposed to the theoretical questions.
Deliberating, pondering, reflecting on ideas and viewpoints, and engaging in discussions with others are important foci in discussion forums as they link learning, developing a sense of efficacy, and civic engagement.When students seek new information, explain or justify their positions, and hold others accountable for their own views, they engage in an active learning process (Bloom 1956;Bender 2003, Van Vechten et al, 2013).Discussions with their instructor or with their peers in the classroom or out in the hallway, further the knowledge students amass and increases their level of engagement as is evident in their participating in discussions beyond the class requirements online.Discussions in online forums may perform similar duties as face-to-face discussions (Hall 1993).These forums are known to aid in higher-order reasoning where peers engage each other in discussions of ideas and positions whether they are online or in classrooms.
As online learning "matures" and educators and universities push for greater online presence, my secondary focus is to provide recommendations, based on length of experience with this collaboration and statistically significant findings, on how to build reflective online communities applicable to online, face-to-face, or upper-or lower-division courses across time zones domestically and globally.The type or level of course would not undermine thoughtful student discussions and interactions with one another when they critically challenged and tested their own and each other's ideas, extended and revisited discussions, understood diverse perspectives, held each other accountable for their views, and for refining their positions.

Comparability across Courses
Before the start of the semester, three instructors who had prior experiences with these collaborations, agreed to collaborate on an invitation-only web-project in courses based on similarities in the university-required course objectives that would engage students in discussions about American politics.The students who participated in the program were enrolled in these American Politics courses "virtually" linked by a collaborative project across three different states and time zones. 3nstructors agreed to distribute common standardized instructions as well as assign a course grade as part of their syllabus requirements. 4Students were required to post responses to the same number of instructor question and respond to the same number of students' posts in order to build continuous dialogue, further discussions, and maintain a discussion-oriented online community.Professors monitored online student conversations for signs that students were abiding by general rules of respect and civility, but refrained from actually participating or extending the discussion of the forums, reminding students of these ground rules when necessary.Each class was a university requirement at a liberal arts institution.Students were enrolled in the same class type, an American Politics course, at each of the three campuses, a descriptive summary of comparisons are provided in table 1. 5  As Table 1 shows, all three courses were undergraduate courses, two of the courses classified as 3000-level or upper-division courses; one offered fully online and the other as faceto-face instruction.The third course was classified as a 2000-level or lower division face-to-face course.The instructors had agreed to the same common course objectives and goals for the courses themselves, each centered on lectures in classes with a similar video-taped lecture for the online class, lending similarities across levels of classes.All of the professors refrained from discussing the instructor questions in class.Instructor assigned course grades ranged from 10-15%.Females outnumbered males on the site, at 68% female to 32% male.The group was racially and ethnically diverse as well, with whites comprising 25% of students, African Americans at 44%, Latino Americans at 21%, and 10% indicated, "other" as their category.Data across gender, race, and other characteristics showed no significant difference in participation across their posts and responses.Instructors rotated responsibility for posing questions weekly across a variety of contemporary and enduring issues in American Politics.The total number of required posts and responses was exactly the same for each instructor as noted in table 1.The array of in-class activities 4 4 Instructors agreed that all students would be required to post and respond to the same minimum number of questions that would be a minimum of 8 posts and 8 responses.And with a minimum word requirement, that of 75 words for posts and responses.As students had to respond to a minimum of 8 posts and responses in a typical 14 week term, they could miss or continue their postings.Posting by students indicate that despite break schedules students continued posting and responding.FERPA issues were met and are further addressed in the conclusions section.

Chadha
Journal of the Scholarship of Teaching and Learning, Vol.17, No. 3, July 2017.josotl.indiana.edu38 was the same for all participating classes, further maintaining the similarities across the collaboration.Learning goals were outlined in respective syllabi, and included developing a better understanding of other points of view, deepening (students') sense of identity as members of a political community, improving their communication, research, analytical, and critical thinking skills through short writing assignments, including those online (Van Vechten et al, 2013).
While not a course requirement, several students initiated their own questions, furthering a sense of community.Student questions did not duplicate the instructor questions, students initiated seventy-nine different questions.Students responded to and revisited ninety percent of them throughout the semester.

Methods
To measure the four hypotheses, a mixed methods approach was used.Content analysis of online discussions that were a part of the students' course requirements was first conducted.Next, a repeated measures anova to test for statistical significance of scores over the length of the term was used.The professors administered an anonymous, online survey during the first week of class with a follow-up survey at the end of the semester.
The Dependent variable: The Reflectivity Index.
To measure reflectivity of posts and responses, the total number of postings per student (example: student X posted six times a day, five days in a row) as a measure toward increased learning was not used but rather a measurement of thoughtful understanding and contribution to a post or response.The dependent variable was the reflectiveness index coined and used in the published work of Van Vechten et.al, 2013, a composite measure that measures critical reflectiveness and deliberation that takes place in online interactive discussions.It measures how reflective and/or deliberate the students were; whether they were thoughtful in their posts and responses; whether they tied in a classroom text or idea, or referenced or cited an external web link or book; whether they asked a question that required further discussion; and whether the lengths of their posts or responses indicated a thoughtful, deliberative discussion.In total, ten instructor-initiateddiscussion questions were coded6 and analyzed, yielding a total of over 500 posts and responses.Each post and response was coded across six variables comprising this index: Reflectivity Index = reflective/deliberative + civic roles + referred to class or text + provided media link + posed an honest question + length of post (+1 for short which met the required minimum of 75 words, +2 for medium, +3 for long).

Findings
Content analysis on postings to measure reflectivity scores was performed across all three courses.Instructor questions were either theoretical in nature or were centered on current events.When using an anova with repeated measures with a greenhouse-geisser correction, the mean scores across modes of instruction, level of course, and length of term were statistically significantly different (p = 0.001) as noted in table 2 and proving hypothesis 1 and 2. A one-way betweengroups analysis of variance was conducted to explore the impact of level of course and mode of  3. Descriptive statistics showed that over the entire semester and across the three classes, students in upper-level face-to-face course had higher means than the upper-level online class; and students in the upper-level online class had higher means that the lower-level face-to-face class.However the one way anova results confirm statistical significance for level of reflectivity across all these classes.To appreciate reflectivity over the course of the semester, I created five time-periods by averaging the first two question reflectivity scores into "timeperiod1" and so on for a total of five time periods.The first and last two weeks of class responses were not used, since two of the three universities started classes the same day, the third institution started and ended a week later.As students had to respond to a minimum of 8 posts and responses over the term, they could miss a posting or continue their postings.Despite break schedules, indications are that students continued posting and responding.Table 4 displays the mean and standard deviation (SD) scores of reflectiveness using a repeated measures anova with a Greenhouse-geisser correction testing for significance of the content analysis of reflectivity scores over the course of the created time periods during the semester.They were statistically significantly different (p = 0.001) for all three courses across the five time periods.
Since the anovas have an overall significant difference in means, I examined the pairwise comparisons using the Bonferroni post hocs.The Bonferroni post hoc tests showed that reflectivity scores increased consistently through every time period 1-5 across all classes, modes of delivery, and levels of courses (p < .000).Although the means scores in time periods 4 and 5 are lower than 2 and 3 as noted in table 3, the evidence supports hypothesis 1 that end of semester pressures would produce a drop in the mean scores, as other course level expectations reach their conclusions.In addition to the student posts and responses in the collaborations, a review of the questions asked by the instructor towards student responses is examined next.I had hypothesized H4, that across theoretical versus current event questions 7 students would perform with similar reflectivity across question type and especially those that were current event questions.Evaluation of reflectivity scores using tukeys' post-hoc significance of test differences in mean scores shown in table 5 support my final hypothesis.Current event questions were significant at the .001level and theoretical question responses significant at the .05level.This is a crucial finding, as universities and educators are concerned with creating a thoughtful and deliberative academic web space.

Lessons from the collaboration: Building a reflective, deliberative classroom
Secondary to these findings I provide suggestions based on my length of experience 8 with these collaborations and these significant findings for building reflective academic communities, ones that can be achieved cross country and globally.
7 Sample instructor-initiated theoretical and current event questions and student-initiated questions are the appendix 3 and 4.
8 This collaboration was initiated by one professor who recruited interested faculty from the APSA Political Science Education listserv, and the initial three professors developed online activities and the site during fall 2008, recreating the site every following semester to date, with changes in institutions participating.Over the next several years, more professors joined the project after hearing about it at the annual APSA Teaching and Learning conferences.FERPA guidelines were followed at each campus and each participating faculty member filed human subject consent forms on their own campuses as well.Students were told about the collaborations and its implications for research.The students had to sign a consent form for the collaboration knowing its implications for research.If they chose not to, an alternative  Foremost, follow human subjects/IRB rules at each participating campus.Often times this is a length process and one undertaken several months prior to semester start.In all these collaborations, students were told about the collaboration several times, and then asked to sign consent forms or if they chose not to participate, an alternative project was provided.Parental consent may be needed if the student is under eighteen years of age.
Consider using both synchronous and asynchronous approaches.For synchronist, consider scheduling online town hall meetings on various topics.Having visual confirmation of like-minded students discussing and deliberating across states and zones makes these questions identifiable and personal.Also consider asynchronous approaches having students interview one another on the issues and then write papers on the interview.Involving the student on the site by asking them to create their own questions for their peers.With personal involvement, students visit and revisit questions furthering discussion as they develop personal stakes in the question they pose.The use of synchronous and asynchronous approaches build a sense of community.
Consider asking current event questions as students have ready access to information and are attuned to social media sounding out differences in opinion among friends before posting.Discussing the general topic prior to asking the questions would develop more reflective peerinteraction online.
The time and investment in building a challenging academic online site does result in committed and engaged students.An important reminder to note is that none of the students in any of the classes knew that their counterparts were in different class types or levels than they were.This study provides statistically significant support that a purposefully created robust academic web space can provide engaged and reflective citizenry across any mode of instruction or level of course.Each student was responding to others based on the quality of posts irrespective of who had posted them, although some had greater knowledge or skills and provided more thoughtful and deliberative posts.
Post-semester student evaluations of the site were additionally supportive of online collaborations.Table 6 compares students' use of website features across a spectrum of choices, whether the questions initiated by the instructor, questions initiated by the student themselves or the articles and links posed on the site were useful.In every instance, online students found these to be more helpful.project was assigned to them.Students then had to ask permission to join the site, and after the instructor verified their signed consent, they were allowed to join the site.Any student 18yrs or under, had to have parental consent in order to join.
Articles or Links posted to the site (Current event Questions) Mostly unhelpful for my understanding of American Politics.89 53 90 *Calculated as a percentage.
It is noteworthy that the student-generated questions were as useful to both online-and face-to-face students.When students are involved or have a personal investment in the site they do revisit and learn from the site.Interestingly, questions that incorporated articles or links which current event questions were most helpful and statistical significance in post-hoc reflectivity scores were achieved for the same (p< .01)as well as for the theoretical questions at p <.05.In current event questions, I find that student posts and responses referred to classroom ideas, class texts, or discussed civic roles versus questions that were theoretical or speculative.Online and upper-division face-to-face students reported that they found the articles and/or links posted with the questions to be useful 89% of the time versus the 53% of self-reports by lower-division students.
Responses about their experience with the site suggest that both online and face-to-face students felt that their experiences on the collaborative site were highly positive, and learning occurred in this collaboration.The students' perspective is key in understanding the learning that occurs in interactions with the instructor as well as with their peers as highlighted in their own words having to do with their interactions with others, and learning from each other and developing informed perspectives. 9These are important findings, ones that focus on pedagogical benefits of designing a robust academic online community.
This study confirms that e-learning in combination with collaborations enhance the students' educational experience and facilitate communication with other instructors and students and with the global community.Moving from classroom learning to e-learning can effectively succeed when universities provide technical support to instructors and students.
This study is an important step towards better understanding the experiences of e-learners.Future research would benefit from a study to see how widely the e-learning experiences found in this study can be applied across other comparable courses and student populations globally.Other studies can compare courses taught with a similar e-learning collaboration with one taught faceto-face or in a hybrid format without the intervention of the online collaboration.The continued research of peer-student interactions can also add to the knowledge of building mindful e-learning communities.Findings in this study are of greater importance as more universities consider offering online courses but are unsure of the academic effectiveness of discussion forums, or how to design quality e-learning activities in an online class in comparison to a face-to-face course or an upper or lower-division courses.What is clear is that online spaces have great potential to encourage critical thinking and interactions among students.

Appendix 1. NING Guidelines for Use and Instruction (Sample Instructions given to all students)
In an attempt to broaden our discussion of American politics, you are required to join the American Politics website.The site's networking platform will allow you to dialogue with other college students who are also tracking American political developments through their classes.Our activities and discussions will encompass a wide range of topics, including the current 05 level.a. R Squared = .388(Adjusted R Squared = .371)b.R Squared = .255(Adjusted R Squared = .233)c.R Squared = .078(Adjusted R Squared = .051)d.R Squared = .131(Adjusted R Squared = .105)e. R Squared = .071(Adjusted R Squared = .044)f.R Squared = .104(Adjusted R Squared = .078)g.R Squared = .466(Adjusted R Squared = .450)h.R Squared = .400(Adjusted R Squared = .383)

Chadha
Journal of the Scholarship ofTeaching and Learning, Vol.17, No. 3, July 2017.josotl.indiana.edu39instructiononthereflectiveness score.Anova results comparing mean scores on the peer evaluations according to level of course and mode of instruction are shown in Table2below.There was a statistically significant difference in scores based on level of course, (p < .001),as well as mode of instruction, (p< .001).

Table 3 : Mean and Standard Deviation (SD) scores of reflectiveness by Upper-Level online, Upper-Level Face-to-Face, and Lower-Level Face-to-Face course
ChadhaJournal of the Scholarship of Teaching and Learning, Vol.17, No. 3, July 2017.josotl.indiana.edu40