Switching From Face-to-Face to Online Instruction Midsemester: Implications for Student Learning

: The 2019 coronavirus disease pandemic disrupted higher education during spring 2020 by forcing all face-to-face classes to unexpectedly transition to online learning. To better understand how switching to remote learning affected students and the factors that impacted their ability to successfully complete classes, we asked 168 undergraduate students in three different psychology classes (six sections total) in the last week of the semester about their experiences from before and after the switch. Students reported some decreased access to technology, changes in work responsibilities, some amount of physical illness, and the need to care for others who were physically ill. Notably, students consistently reported increased stress and decreased ability to focus. Students varied in how much they prioritized classes after the switch, which predicted their performance in the class, measured by exam grades, overall grades, and completion of attendance before and after the switch. Importantly, survey respondents significantly differed from 173 nonrespondents in their class performance, which suggests that results from voluntary surveys may capture a limited perspective and possibly underestimate the detrimental effects of the shift to online instruction. Implications for planning for future online classes in a global pandemic are discussed.

A national survey of undergraduate students during the pandemic provides additional evidence that the unexpected shift to remote learning created challenges for students (Means & Neisler, 2020). While 51% of students reported being "very satisfied" with their courses before the switch to online instruction, only 19% remained "very satisfied" after the switch, with 40% reporting they were "somewhat satisfied" and 27% stating they were "somewhat dissatisfied." While many students said they were "very satisfied" (37%) or "somewhat satisfied" (39%) with their instructor's preparation to teach online, their assessment of their learning overall was less positive, with only 17% being "very satisfied," and a greater proportion being "somewhat dissatisfied" (27%) or "very" dissatisfied (15%; Means & Neisler, 2020).
Students indicated that during the transition to online teaching, they found it difficult to stay engaged with both their peers and instructors, and most felt the online learning experience was not as effective as face-to-face instruction (Hansen, Janik, Rauch, Marsiglio, & Keith, 2020;Means & Neisler, 2020). Students reported multiple problems including lacking opportunities to collaborate with others (Means & Neisler, 2020), difficulty paying attention, staying focused, and staying motivated (Hansen et al., 2020;Means & Neisler, 2020), and not feeling included (Means & Neisler, 2020).
The transition to online learning had effects on not only the academic success of students but also their mental health. Several studies explored the impact of the COVID-19 pandemic on the mental health of college students, a population already at risk for mental health concerns (Patsali et al., 2020;Z. Wang et al., 2020). One survey study of U.S. college students conducted in May 2020 found that more than 48% of participants showed moderate-to-severe levels of depression, with as many as 18% having had thoughts of self-harm or suicide in the prior 2 weeks (X. Wang et al., 2020). Over 38% of participants showed moderate-to-severe levels of anxiety. These rates are considerably higher than prevalence rates for depression and anxiety in college students found in research conducted prepandemic (see, e.g., Eisenberg, Hunt, & Speer, 2013;Ibrahim, Kelly, Adams, & Glazebrook, 2013;Kronfol et al., 2018). A majority of students (71.3%) also reported increased levels of stress (X. Wang et al., 2020). The most frequently cited contributor to increased stress was academics, followed by uncertainty regarding the pandemic, health concerns, finances, living/work environment, and finally social life/isolation.
A longitudinal study exploring the psychological impact of the pandemic on college students indicated that the risk for mental health issues may actually be higher and more of a concern in students without a prior history of mental health issues than those with preexisting mental health concerns (Hamza, Ewing, Heath, & Goldstein, 2020). Hamza et al. (2020) reported that social distancing may have had more of an impact on students who were not used to feeling isolated or being alone.
These findings echo research conducted during other natural disasters. Davis, Grills-Taquechel, and Ollendick (2010) explored the effect of Hurricane Katrina on undergraduates, revealing the psychological impact natural disasters can have on college students. Students displaced from their universities in New Orleans as a result of the hurricane were more likely to experience symptoms of distress, posttraumatic stress disorder, and depression compared to students at another Louisiana university who were not displaced. While there may be some similarities between the impact of natural disasters and that of a health pandemic, including financial difficulties, loss of resources, loss of community of peers, and disruption in social networks, it is important to be cautious when making comparisons. College students who were displaced because of Hurricane Katrina were reintegrated into another university and most of the reported stressors were related to property damage and housing (Davis et al., 2010). Nevertheless, the impact of natural disasters on students may include mental health as well as academic outcomes, and the unique challenges of a global pandemic in particular are important to explore to mitigate negative impacts on students in case of future waves.
The current study explored the academic and psychological impacts of the unexpected shift to remote learning in spring 2020 on undergraduate students enrolled in three different psychology classes at a large urban university. Students were asked about impacts on their personal, work, and academic lives. Responses to survey items provided a descriptive view of the overall impact of the transition to online learning for students. In addition, responses were used as predictors of class performance pre-and posttransition to online learning. Furthermore, the authors were able to conduct unique analyses comparing survey respondents and nonrespondents. Many surveys have explored only students' perceptions of their learning and the personal impacts of quarantine during COVID-19 (e.g., Aguilera-Hermida, in press;Hansen et al., 2020), leaving open the possibility that a portion of students may not be represented because they choose not to respond to such surveys. In the present study, the survey was administered as a class activity. Thus, the authors, who served as the course instructors, had access to grade data for both survey respondents and nonrespondents. This enabled comparisons of class performance between these two groups, providing insight on the portion of students who may not be represented in many reports on the impacts of the COVID-19 pandemic because they do not participate in voluntary surveys.

Participants
Data were collected at the end of the spring 2020 semester from students at Indiana University-Purdue University Indianapolis, a large Midwestern urban public university, enrolled in a 100-level introductory psychology course with two regular sections and one honors section, a 200-level ethics and diversity course with two sections, and a 300-level research methods course with one section (see Table 1). All sections were taught by the authors in a face-to-face format prior to the transition to online learning. To comply with Institutional Review Board protocols, respondents did not receive course credit or any other incentive for completing the survey. Survey response rates in each class ranged from 29.6 to 68.8% with a mean response rate of 50.7% (see Table 1). The survey sample consisted of 168 students (Mage = 20.5 years, SD = 3.87) who were mostly women (78.6% women, 19.6% men, and 1.2% nonbinary). Reported races were White (64%), Black (11.9%), Hispanic/Latino (8.3%), Asian or Pacific Islander (6%), Native American (< 1%), and other or more than one racial category (8.4%). The nonsurvey sample, that is, nonrespondents, consisted of 173 students. No demographic information could be collected from this group. Note. Instructor A is the first author and Instructor B is the second author.

Materials
Survey items. Students responded to a survey addressing how switching to remote learning affected them and what factors impacted their ability to successfully complete classes during the spring 2020 semester (see Appendix 1 for all survey items). Participants accessed the survey through the online learning management system (Canvas) and included questions about how their school, work, and personal life were different "before spring break" in comparison to when classes were held on campus "after spring break," when all classes were converted to online instruction. All items were developed by the authors.
More specifically, the questions examined student technology access, work, living situation, school life and study habits, and mental and physical health. For the technology access questions, students rated on a 4-point Likert scale how much they agreed with statements about having reliable access to devices such as a computer, tablet, or phone and reliable access to the internet. For the work items, students answered questions about the number of hours worked per week before versus after spring break, if those hours changed multiple times, whether work was in person or from home after spring break, and how work responsibilities changed after spring break. For the living situation questions, students answered where they were living (on campus in dorms, off campus with family vs. roommates vs. alone) before versus after spring break, how many people they lived with before versus after spring break, and how their personal/at home responsibilities changed after spring break. For the school life and study habits questions, students answered how long they spent studying for the specific class before versus after spring break and how their school responsibilities changed after spring break. For the mental and physical health questions, students answered if they or someone they lived with had fallen ill after spring break, with either COVID-19 or a non-COVID-19 illness, and if they were responsible for the care of someone who had fallen ill. Students were also asked about their stress, ability to focus, and prioritization of classes after spring break; these items served as proxies for mental health and are similar to survey items used in other studies exploring mental health in college students during the pandemic (see, e.g., Son et al., 2020;. Students were also asked about their age, gender, and race, and finally, an open-ended response item was provided for any other information respondents wanted to share. Class performance measures. In addition to responses on the survey, performance in the class was measured by looking at overall grades, exam grades, and attendance. Overall grades included students' average grade on assignments before spring break, average grade on assignments after spring break, and change in overall grade (after spring break minus before spring break).
Exam grades included average grade on exams before spring break, average grade on exams after spring break, and change in exam grades (after spring break minus before spring break). Because every class included an opportunity to drop the lowest 2 exam grades, zeroes were not included in the calculation for the average exam grades either before or after the break.
Attendance measures included proportion of attendance before spring break, proportion of completed "attendance" prompts after spring break, and change in attendance (after spring break minus before spring break). Because attendance at synchronous class meetings after break was not required, "attendance" was assessed through short writing prompts that were due by midnight on the usual class day. Both before and after spring break, attendance was worth 4 points per day, or around a little less than 0.5% of the total grade per activity.

Procedure
At the university where data were collected, all classes transitioned from regular face-to-face instruction to entirely online after an extended, 2-week spring break in March 2020. In the last week of class for the semester, at the end of April, the authors posted an announcement in each class recruiting participants for a survey, administered through Canvas, concerning how the switch to remote learning had affected them that semester. The instructions in the announcement were similar to the informed consent they saw in the survey's instructions. Students were told that the survey was optional, without incentive, should take less than 10 min to complete, and that they had 1 week to complete it. They were also told that the survey was not anonymous. This would allow us to tie their responses to their performance in the class, though analyses were conducted on anonymized data. After reading the instructions, students could click on a button to open the ungraded survey and respond. The survey was available for 1 week and closed before the semester officially ended. Class performance data for all students and survey data from respondents were downloaded from Canvas and anonymized by removing names and usernames prior to analyses.

Results
Three types of analyses provided insight into how the switch to online instruction affected students: the descriptive results from the survey, the prediction of performance in the class based on some responses to the survey, and the comparison of performance in the class from survey respondents and students who did not respond to the survey.

Survey Results
The responses to the current survey were somewhat consistent with a campus-wide survey administered by the university (Hansen et al., 2020). Students generally had reliable access to both devices and the internet before spring break, but some students shifted from "strongly" agreeing that they had reliable access to "slightly" agreeing after spring break. This shift was biggest with regards to access to the internet, where 88% of respondents "strongly" agreed to having reliable access before spring break, but only 56% responded that way after spring break.
Students' work life was disrupted by the COVID-19 pandemic, which coincided with the shift to online instruction after spring break. While 42% of respondents either did not work or did not have their work hours change after the break, 24% had their hours change once, and 34% had their hours change multiple times: increasing multiple times (8.3%), decreasing multiple times (7.8%), or both increasing and decreasing multiple times (18%). At the time, the state (Indiana) was under a lockdown order where only essential businesses were allowed to have in-person workers. Of the students who responded that they worked after spring break, 3 times as many had to go to work in person than were able to work from home. Overall, work responsibilities either "increased a lot" or "decreased a lot" for students (see Table 2). In contrast, personal and school responsibilities mostly increased for students, with nearly half of students indicating school responsibilities "increased a lot" (see Table 2). Most students moved in with their families after the transition. While 47% of students lived with their families before spring break, 81% lived with them after.
Some students did experience physical illness (30% fell ill after spring break), though only a small proportion of them either knew or believed it to be COVID-19 specifically (3%). Because testing was difficult to obtain at the time, we included both students who suspected it was COVID-19 and those who had tested positive in the same response option. A similar proportion of students' family members were ill (30%), but with a slightly higher number with confirmed or suspected COVID-19 (8%). In addition, 18% of students were responsible for the care of someone who was sick during this time period.
Most students did experience detrimental effects on their stress level and ability to focus, the proxies for mental health (see Table 3). For example, 65% of students said their stress level "increased a lot," and 57% of students said their ability to focus "decreased a lot." This pattern is consistent with results from a campus-wide survey from the spring (Hansen et al., 2020). Despite the impacts on stress and ability to focus, students varied widely in terms of how much they claimed to prioritize classes (see Table 3). While many students (48%) stated that they decreased their prioritization of classes after spring break, 30% increased their prioritization of classes, including 11% who indicated that they increased their prioritization "a lot." This diversity in prioritization is striking considering that a large percentage of students perceived their school responsibilities to "increase a lot" after the shift to online learning. This suggests that while some students were able to (or chose to) directly engage with the additional school responsibilities, many students did not.
The increased stress and overall detrimental effect on mental health was also evident in the open-ended responses. Students described difficulty in being able to focus on school work, coping with family or personal illness, and generally experiencing increased personal stress. Below are a sample of student comments 1 : • "It is harder to do online classes at home, I prefer campus so much better because of all of the resources and being at my dorm." • "Around the last week of March I came down with symptoms of Covid-19 .... I didn't get the results back till a week later that I was positive. About a day after I got tested my mom who have Lupus and other health issues came down with symptoms and also tested positive. It was difficult to get any motivation to do any thing while sick I could barely get out of bed." • "I am a caretaker for my mom who has multiple sclerosis. she is considered extremely high risk for COVID-19 so we've had to drastically change our lifestyle due to the pandemic." • "I've always suffered with anxiety, however the amount of it that I am feeling now after spring break is significantly more noticeable and affects me at a much greater degree than it did before spring break." • "Stress and anxiety have been through the roof for me. … My well oiled machine of school/work balance is non-existent and I feel like I wake up, work, do school work, and go to sleep. I leave my room to get food and go to the bathroom and whatnot but beyond that, I don't feel like I have the time. I try to take days off work to prioritize my mental health but I only have so much PTO and flexibility. School just doesn't feel as important right now, or doable. Motivation is at an all time low and so is focus."

Prediction of Class Performance From Survey Responses
To answer our initial questions of how students' performance in class was affected, we conducted simple regression analyses using the mental health predictors of perceived change in stress level, ability to focus, and prioritization of classes on the difference scores for overall grades, exam grades, and proportion of completed attendance. Stress level failed to predict any of the outcomes, and only ability to focus significantly predicted the change in exam grades, F(1, 165) = 10.14, p = .002, R 2 = .05; b = 2.12, 95% confidence interval (CI) [0.81, 3.44]. Students who agreed more strongly that they had less ability to focus after spring break also had less improvement in exam scores. While stress level and ability to focus predicted very little, prioritization of classes predicted all three outcomes: change in overall grade, F(1, 166) = 5.02, p = .03, R 2 = .02; b = 0.03, 95% CI [0.003, 0.05]; change in exam grade, F(1, 166) = 9.43, p = .002, R 2 = .05; b = 1.60, 95% CI [0.57, 2.63]; and proportion of completed attendance, F(1, 166) = 9.31, p = .003, R 2 = .05; b = 0.03, 95% CI [0.01, 0.06]. As students increased prioritization of class, they tended to do better after spring break (in comparison to before) on their overall grade, exam grade, and attendance completion.

Comparison of Survey Respondents and Nonrespondents
Because we had access to performance in the class for both respondents and nonrespondents, we had a unique ability to look at a population notoriously difficult to study: those who do not participate in surveys. There were some striking differences between these two groups, which were roughly equal in size (168 respondents and 173 nonrespondents). In particular, as seen in Table 4, respondents and nonrespondents differed significantly on overall grades, exam grades, proportion of completed attendance, and final posted grade, both before and after spring break; respondents scored significantly higher than nonrespondents on all of these measures. This indicates that the survey respondents were likely different from nonrespondents beyond just the decision to participate in the survey. Note. Time indicates before or after spring break 2020. CI = Confidence interval.
In addition, respondents and nonrespondents differed significantly in how the shift to online instruction after spring break affected their exam average and completion of attendance. Looking at exam average using a two-way mixed-effects analysis of variance (ANOVA), we found a significant effect of responding on exam average, F(1, 339) = 14.86, p < .001, ηp 2 = .04. There was no significant main effect of time (before vs. after spring break) on exam average, F(1, 339) = 2.55, p = .11, ηp 2 = .01. However, there was a significant interaction effect of responding and time on exam average, F(1, 339) = 5.10, p = .03, ηp 2 = .02. Students who did not respond to the survey performed significantly worse on exams after spring break, t(172) = 2.18, p = .03, Cohen's d = 0.17.
There were similar effects on completion of attendance. Using a two-way mixed effects ANOVA, we found a significant effect of responding on attendance, F(1, 339) = 30.67, p < .001, ηp 2 = .83. In contrast with the exam average, there was a significant effect of time on attendance, F(1, 339) = 7.57, p = .006, ηp 2 = .22. But most interestingly, again, there was a significant interaction effect of responding and time on attendance, F(1, 339) = 22.55, p < .001, ηp 2 = .06. Students who did not respond to the survey completed significantly less attendance after spring break, t(172) = 4.49, p < .001, Cohen's d = 0.34.
In total, it seems likely that respondents were qualitatively different from nonrespondents: perhaps they were more able to cope with the switch to online instruction (as seen in the interaction effects for both exam average and completion of attendance) or perhaps they were more conscientious students to begin with (as seen in the differences in overall grades, exam grades, and attendance even before the break). However, the conclusions from the respondents to the survey indicate clear effects on mental health and some effects on performance in class, and in conjunction with those potential differences, they suggest that the detrimental effects of the switch from face-to-face to online instruction and the disrupted semester are likely underestimated in the overall student population when measured only by survey respondents.

Discussion
Results from the current study indicate that students' lives were impacted in various ways during the spring 2020 semester, reflecting students' multiple roles. For example, 75% of respondents who reported that they worked after spring break were considered essential workers and had to go to work in person. The majority of students who worked also had to deal with changes in work hours and schedules. While most respondents (70%) did not fall ill with either COVID-19 or another illness, 18.5% were responsible for the care of another person after the switch to online learning. Students commented that they became the designated grocery shopper, maid, and caretaker for others. Others reported either having or living with individuals with a compromised immune system and the additional stress that this brought.
Prior research indicated that undergraduate students constitute a vulnerable population when it comes to the effects of lockdowns during the COVID-19 pandemic and are at risk for impacts on mental health, including depression and anxiety (Hamza et al., 2020;Patsali et al., 2020;Son et al., 2020;Z. Wang et al., 2020). In the current study, the majority of respondents reported that their stress level increased and their ability to focus decreased during the pandemic. These findings coincide with results from a larger scale survey conducted at the same university that also found that the majority of students reported increases in stress and anxiety during the pandemic as well as learning challenges due to difficulty studying and distractions at home (Hansen et al., 2020). Other studies have also reported that students found the online learning experience during the early portion of the pandemic to be unpleasant and that their level of interaction with other students and faculty decreased as did their motivation (Aguilera-Hermida, in press;Means & Neisler, 2020). This could have contributed to both the mental health responses and effects on class performance in the current study.
Most published studies in this area are based completely on self-reported data and consequently any analyses of the impact of attitude toward online learning and the emotional impact of the COVID-19 pandemic on actual class performance are based on students' own reporting of grades (Aguilera-Hermida, in press;Aucejo, French, Araya, & Zafar, 2020;Hansen et al., 2020;Means & Neisler, 2020). In this study, we were able to compare survey responses to multiple class assessments. While stress did not predict change in class performance, this may be because nearly all students reported at least some increase in stress level. Students who reported that their ability to focus "decreased a lot" were more likely to have lower exam scores after spring break. Students varied on how much they were able to prioritize their classes after break and this factor predicted changes in exam grade, attendance participation, and overall grade. Those who increased their prioritization of classes performed better after break compared with those who reported that their prioritization of classwork decreased.
These findings are mirrored in other studies that found significant heterogeneity in survey respondents. Gonzalez et al. (2020) reported that students in their sample were able to study more continuously and efficiently during the pandemic and that grades improved after the move to online learning. However, a survey of college students at Arizona State University during the pandemic found that while one quarter of students increased their study time by more than 4 hr per week after the transition to online learning, another quarter decreased their study time by more than 5 hr per week (Aucejo et al., 2020). Aucejo et al. (2020) reported that these differences often coincided with socioeconomic differences.
The current study provides a unique contribution to the literature as we were able to explore differences between survey respondents and nonrespondents, revealing that the impacts of the lockdown and sudden switch to online learning may be underestimated in the available literature. Nonrespondents were more likely to have lower exam grades and lower overall grades both before and after spring break. While this might simply indicate that respondents are stronger students in general, by looking at change in performance before and after break we can see that the pandemic, and the subsequent switch to online learning, may have had a disproportionate effect on nonrespondents. While exam grades and participation in attendance activities did not change after spring break for survey respondents, participation in these assessments was significantly lower for nonrespondents after break.

Limitations
Several limitations should be mentioned when considering the generalizability of the current study. First, all students were enrolled in psychology courses and several were likely majors in psychology, which might distinguish this sample from the general undergraduate population. However, three class sections were from introductory psychology courses, which are generally taken by students from a variety of majors and often representative of the general undergraduate population, and two class sections were from a course that qualifies for a general education cultural understanding requirement. While the sample was largely women, this is also typical of both survey respondents in general and psychology courses specifically. Given the higher performance observed for survey respondents compared to nonrespondents, it is important to point out that a portion of the sample was from an honors section, and nearly 70% of students in that section participated in the survey. However, this section made up only about 6% of the full sample so the impact on the data is limited.
The current study is descriptive and retrospective in nature and thus one must be cautious in inferring causality. As the survey was completed during finals week, it is possible that students overestimated their levels of stress and underestimated their ability to focus. Future research during the prolonged period of online learning during the 2020-2021 year should explore the extended impacts of the COVID-19 pandemic and whether students have begun to adjust to the current situation.

Conclusions and Implications for Future Online Classes in a Pandemic
Understanding the challenges faced by students during the initial COVID-19 lockdown period can help faculty and institutions know how best to support students during the 2020-2021 academic year and beyond. While people may be getting used to life during the pandemic, many programs are still relying on online learning, and students are still dealing with the personal, academic, and economic impacts of the health crisis. Data from this study indicate that students are under a great deal of stress, and finding ways to help them prioritize their courses might help them improve their class performance. Class performance from nonrespondents indicates that the most vulnerable students might be those we have studied the least. This research can help institutions plan during future waves of the pandemic or during any other disaster that may require a sudden transition to online learning.