Closing the Gap: First Year Success in College Mathematics at an HBCU

At our Historically-Black University, about 89% of first-year students place into developmental mathematics, negatively impacting retention and degree completion. In 2012, an NSF-funded learning enrichment project began offering the introductory and developmental mathematics courses on-line over the summer to incoming science, technology, engineering and mathematics (STEM) majors at no cost to students. Passing rates for the summer on-line classes were around 80%, and students in the on-line classes scored equivalently on the common departmental final exams as students taking the classes in the traditional format. For students who passed the on-line classes, their performance in the following classes (College Algebra and Trigonometry) at least equaled that of students who progressed to those courses by taking the traditional series of in-person courses. Three years of data show that students who started college with an on-line mathematics course in a summer bridge program had a higher first year GPA, a better first year retention rate and earned significantly more credits in their first year than the overall population of STEM students. While not solely attributable to the on-line classes, these results suggest that offering introductory mathematics courses on-line as part of a freshman bridge program is an effective, scalable intervention to increase the academic success of students who enter college under-prepared in mathematics. The positive results are particularly exciting since the students in our project were 87% minority.

to enroll in at least one remedial course (Ross & Kena, 2012) and only 44% of FTFY students entering college in 2013 met the ACT benchmarks for college-readiness in mathematics (ACT, 2013).
Students from underrepresented groups such as racial and ethnic minorities, low-income students, and those from urban or rural high schools are more likely to enter college unprepared, and thus struggle to succeed (Perna, 2005;Rose & Betts, 2001). Lack of preparation, particularly in mathematics, has the greatest impact on students majoring in STEM, as these programs generally require that students take at least one semester of calculus, and Calculus I is the required freshman mathematics class for STEM majors at many schools (Bahr, 2010). The declining number of students prepared to succeed in a college level calculus class in their freshman year reduces the pool of students likely to graduate with a STEM degree in four years (Kreysa, 2006).
Students' lack of college-readiness has a major impact on their success, as students who enroll in remedial classes are far more likely to drop out, and the lower the initial placement, the less likely a student is to obtain a degree (Bailey, Jeong, & Cho, 2010). However, whether large numbers of students benefit from taking remedial courses is not clear. On the positive side, a study of 28,000 traditional-age college freshmen in Ohio found that students who took one or more remedial classes were more likely to persist in college than students with similar backgrounds who were not in remediation (Bettinger & Long, 2009). A study of a large number of community college students found no difference in long-term academic outcomes between students who "remediated successfully" in mathematics -that is, passed a college level mathematics class after first taking a remedial class-versus students who passed a college-level math class without taking the remedial class first (Bahr, 2008). However in this study, three out of four remedial math students did not remediate successfully, and the outcomes for those students were disastrous, with 81.5% neither completing a credential nor transferring to a four year school (Bahr, 2008).
Some powerful quantitative studies of the benefit of remediation have used a regressiondiscontinuity design to compare the outcomes of remediation for students who fall just above and just below placement test cut-offs, however these studies show mixed results. Some have found that carefully designed placement procedures result in good outcomes for the students just below placement cut-offs who took remedial mathematics classes (Calcagno & Long, 2008), while other studies show that remedial classes have no effect on the likelihood of obtaining a degree, even when they are associated with a short-term increase in retention (Melguizo, Bos, & Prather, 2013;Scott-Clayton & Rodrigues, 2012). Similarly, a study of a large sample of college students in Texas found no evidence that assignment to remedial mathematics courses provided any benefit for the academic outcomes of students (Martorell & McFarlin, 2010), however, a study using longitudinal data from Tennessee gives a more nuanced view (Boatman & Long, 2010). The state's multi-tiered system places students into one of four levels of pre-college mathematics and three levels of precollege English courses, therefore allowing differentiation of the effect of placement based on the level of preparation of the student. The results of the analysis suggest that remedial and developmental courses have different impacts depending on the level of student preparation. For students on the margins of needing remediation the effects tended to be negative, however, for students with the greatest levels of under-preparation, the negative effects of remediation were much smaller, and in some cases, particularly for pre-college writing courses, students who took remedial courses performed better academically than their peers (Boatman & Long, 2010). Overall, the lack of clear positive results for current approaches to developmental mathematics education clearly shows the need for new and better approaches to remediating college students' mathematics unpreparedness (Doyle, 2012).
Our Historically-Black institution has an undergraduate student population that is approximately 80% African-American. Based on the results of Accuplacer TM placement exams, about 89% of FTFY students place into a no-credit, pre-college mathematics course, "Introduction to Algebra". Passing this course is required for students to advance into credit-bearing mathematics courses, which are prerequisites for introductory courses for almost all STEM majors. In spite of the high stakes for students, since 2010, pass rates for Introduction to Algebra have ranged from a low of 54% to a high of 62%. With nearly the entire first year class required to take pre-college math as their first "college" mathematics course, and a significant fraction taking the course multiple times before earning a passing grade, "Introduction to Algebra" represents a huge burden for both the University and its students. Moreover, it is not clear that the course has much impact on student success, as the passing rate for College Algebra, the credit bearing mathematics course that students take after they pass Introduction to Algebra, rarely exceeds 60% and most semesters is below 50%.
In order to address this problem, in 2012, the Science and Mathematics Initiative for Learning Enrichment (SMILE) project began piloting a program in which incoming students accepted to STEM majors at DSU could take "Introduction to Algebra" as an online course during the summer as part of a summer bridge program. In 2013 and 2014 the program was expanded to include an on-line version of College Algebra, and most students were placed into the on-line College Algebra regardless of their Accuplacer TM scores. Here we describe the outcomes for the first three years of the pilot program.

Description of the program
The SMILE project was targeted at STEM majors, which, on this campus, are students majoring in Agricultural Science (environmental science, plant science, natural resources or pre-veterinary medicine), Biology, Chemistry, Computer Science, Food Science, Information Technology, Mathematics, Movement Science, or Physics. See Table 1 for a breakdown of the majors of the students in the program.
In the summer of 2012 freshman STEM majors who attended one of the first two New Student Orientations (that took place in June) and were placed into Introduction to Algebra were given the option to take the course on-line during Summer Session II. The 35 students who enrolled in the course were divided into two sections with 2 different instructors. The students had seven weeks to complete the on-line material, and then came to campus for a one-week "STEM Summer Training Camp" just before the start of the fall semester where they took the common departmental final exam for Introduction to Algebra on campus.
In the summer of 2013, STEM students who attended one of the new student orientations in June could sign up to take either Introduction to Algebra or College Algebra on-line over the summer. Ten students placed directly into College Algebra and enrolled in the on-line course. An additional 12 students who had placed into Introduction to Algebra were "promoted" to College Algebra, either because they had a mathematics SAT score over 500, or they had taken trigonometry, precalculus or calculus in high school and gotten an A or B. As an aside, it is quite common for students at our institution to have gotten a good grade in precalculus or calculus in high school and still be placed into Introduction to Algebra by Accuplacer TM , this may be due to grade-inflation, or weak courses in high school or it is possible that Accuplacer TM -based placement may not reflect the students' true mathematics ability. Twenty-five students who had placed into Introduction to Algebra were left at that level and enrolled in the course on-line over the summer.
For both on-line mathematics classes, the students took the common final exam for the courses in person, on campus during the STEM Summer Training camp the week before classes started. The same instructor taught both on-line classes. This instructor has a lot of experience teaching both online mathematics courses and hybrid courses that blended extensive on-line exercises with traditional classroom work.
In 2014, 42 STEM students enrolled in one of the on-line mathematics classes. All 42 students had initially tested into Introduction to Algebra. Again, based on a math SAT over 500 or an A or B in a high school trigonometry, precalculus or calculus class, 27 were promoted to College Algebra. The remaining 15 were placed into Introduction to Algebra. The instructor for both courses was the same as for the summer of 2013, and once again, students in the on-line courses took the common final exam for the classes in person, on campus during the STEM Training Camp. In all three years, students enrolled in the on-line classes had access to peer tutors who were available to answer questions via email and by cell phone during "office hours" from 8:00 pm -11:00 pm, Sunday through Thursday.
In summer 2013, five students did not complete the class and did not matriculate to the institution, while in summer 2014, two students did not complete the course and did not matriculate. Students who did not matriculate to the University were excluded from this analysis. Table 1 shows the break-down of students in each SMILE cohort by major, while Table 2 provides information about student demographics and entering qualifications. While, in general, the differences are small, students who participated in the SMILE program and took the summer on-line mathematics courses were a little more likely to be female, minority and be out-of-state compared to the overall STEM population. They also tended to have slightly higher average high school GPAs than the overall STEM population.

Results for summer mathematics students
For all three years of this pilot program the performance of the students in the on-line summer courses matched or exceeded the performance of similar students taking the same classes in the fall. The passing rates for our on-line Introduction to Algebra classes were 80.8% in 2012, 75% in 2013 and 60% in 2014. In contrast, the average passing rate over three years for 1,499 firstyear students taking Introduction to Algebra as their first mathematics course in the fall was 58.2%.
The decline in passing rates in the 2013 and 2014 cohorts most likely results from our policy of "promoting" into College Algebra students who were placed into Introduction to Algebra by Accuplacer TM , but whose SAT or high school transcript suggested better mathematics preparation.
The 2012 Introduction to Algebra course included all students in the SMILE program, while in 2013 and 2014 only the students with the weakest mathematics preparation took the Introduction to Algebra course. The trend also represents some natural variation in students (who self-select for the SMILE program) as in the summer 2015 of the Introduction to Algebra course with the same instructor 12 out of 15 students (80%) passed the course.
For College Algebra, the passing rates for the summer online classes were 86% in 2013 and 77.8% in 2014, while in contrast, the average passing rate over two years for the 169 first-year Harrington, Lloyd, Smolinski,  students who took College Algebra in the fall was just 66.3%. The students in the on-line classes seemed to learn the material as well as students taking the same classes in person during the fall semester, as their scores on the mathematics department's common final exams for the courses equaled or exceeded the performance of students who took the courses in the traditional format. Figure 1 shows the average mean and median of scores on the common final exams for the summer Introduction to Algebra and College Algebra classes over the three years. These compare with a mean exam score for the fall College Algebra course sections of 49 + 20.6% with a median score of 52%. For the Introduction to Algebra course, the overall mean score on the final exam for all fall sections was 47 + 21.3% while the median was 48%. For 2013 we separated the exam scores for the summer College Algebra class for students whose Accuplacer TM scores placed them directly into College Algebra from the scores of students who placed into the developmental course, but were advanced to College Algebra and saw no difference between them.
The distribution of grades for the summer on-line classes is shown in Figure 2. During the first year, a substantial number of students did not complete the course, however, subsequent Bars are the means + standard deviation and median scores. In 2013 the "Low plcmnt" is students whose scores on Accuplacer TM put them in Introduction to Algebra, but were advanced to College Algebra. "High plcmnt" are students whose Accuplacer TM scores placed them directly into College Algebra. The differences in mean scores were not significantly different across the years or between the low and high placement groups. (One way ANOVA F(69) = 0.919, p = 0.404 for Intro to Algebra exam scores and F(46) = 0.249, p = 0.78 for College Algebra exam scores). iterations included more IT support for the students at the start of the course and increased availability of peer tutoring, and that probably helped reduce the number of students who did not complete the course.
Overall the students were very positive about their experiences with the summer classes. As shown in Figure 3, in 2014 most students in both classes rated the courses "excellent" or "very good" overall, as well as for ease of access to course material, ease of using the Blackboard system, ease of understanding the material, and the level of detail in the coverage of the topics. Surprisingly, students in the Introduction to Algebra course were a little more likely to feel that the class met their expectations for a college course, although for both groups, that was the question with the largest range of answers. The students were uniformly glad that they took the class and would recommend it to others. Most were clear that they had completed the course independently, but acknowledged that their parents played a role in their decision to take the course. Results from satisfaction surveys were essentially the same in summer 2012 and summer 2013.
An important question for us was how the students who took the on-line courses fared in the college mathematics courses that they took on campus the following semester. Most students who passed the Introduction to Algebra Course went on to take College Algebra in the fall, and most students who passed the College Algebra course went on to take Trigonometry. As shown in Figure 4, students who took the on-line summer classes (SMILE students) performed at least as In the Intro to Algebra course students can receive grades of only "satisfactory" (gray) or "unsatisfactory" (sand pattern), while students in College Algebra can receive the full range of grades. Harrington, Lloyd, Smolinski, and Shahin Journal of the Scholarship of Teaching and Learning, Vol. 16, No. 5, October 2016. josotl.indiana.edu 99 well in their following mathematics course as the other students in the class on the common final exam and in the course grade. A one-way ANOVA did not show significant differences at the p < 0.05 level for scores on the common final exams for the fall classes.
The College Algebra results were [F(769) = 2.051, p = 0.108 ] and for Trignonometry, [F(266) =1.382 p = 0.249] indicating no significant difference in the performance of students who took the previous course as on on-line or in-person course. Similarly, although there is a trend in the passing rates so that students who took the on-line classes appear to do better in the following classes than students who took the earlier classes in person, none of the differences are significant (Chi Square test). College Algebra and Trigonometry from Fall 2014 come the closest (Chi square statistic =1.9007; p = 0.168 for Trigonometry and Chi square statistic = 2.263, p = 0.132 for College Algebra), but the differences are not significant.   Figure 5 compares the grade distribution of the SMILE students in their summer class with the distribution of grades those students received in the follow-on classes in the fall. Students who did not pass their summer on-line mathematics classes have the option to take the same course again in the fall. In 2013, one of the three students who had failed the College Algebra course in the summer took the same course again in the fall and received an A.
The major goal of the NSF-funded SMILE project is to increase the freshmen to sophomore year retention rate of STEM students, and then their graduation rate, so we were very interested in the impact of the summer online classes on our students' overall academic performance. At the beginning of their sophomore year the SMILE students were assessed for how many had been retained in their major, what their overall GPA was and the number of credit hours that they had accumulated. We compared those numbers to the overall population of STEM freshmen. In addition to the on-line classes, the SMILE program includes a STEM Training Camp where STEM first-year students come to campus a few days before the start of academic year for cohort formation and skill-building in mathematics and English. SMILE students also receive intensive Figure 5: Summer to fall course grade distributions show most students who pass the summer on-line course also pass the following course For each cohort the left column shows the distribution of grades in the summer course and the right column shows the distribution of grades for the next mathematics course that the students took in the fall. In the Intro to Algebra course students can receive grades of only "satisfactory" (black) or "unsatisfactory" (gray), while students in College Algebra can receive the full range of grades.
peer mentoring in their first year. These activities began with the fall 2010 cohort, while the online summer classes did not begin until the fall 2012 cohort.
As shown in Figure 6, in general, students in the SMILE program had slightly higher first year GPAs, accumulated a higher number of credits, and were retained in their major better than their STEM peers. The positive effects of the SMILE program were much larger for the cohorts that took an on-line summer class (Fall 2012(Fall , 2013(Fall and 2014, particularly when that class was College Algebra. A oneway ANOVA showed that the differences in cumulative GPA are close, but do not quite reach significance [F(9,1152) = 2.290, p = 0.015], however a one-way ANOVA did show significant differences at the p < 0.

Discussion
A major issue for institutions of higher education, particularly those institutions who serve students who are from under-represented groups, is how to deal with students' lack of preparation for college-level mathematics. Developmental courses are a significant drain on the resources of both institutions and students, and the outcomes, in general, are not positive. At this institution, a new approach was piloted to prepare STEM students for college level mathematics -an approach that is affordable both for our students and the University, and has the potential to serve a large number of students. Having first-year STEM students take their first introductory mathematics course online during the summer before coming to campus accelerated their entry into trigonometry and calculus as well as the introductory courses for STEM majors. At this institution, STEM majors who begin their first semester in the developmental "Introduction to Algebra" course are not able to take the standard first-year curriculum for their majors and frequently end up a year behind in their academic program. The Introduction to Algebra class carries no credits, thereby reducing the number of credits that students can accumulate during their first year, making it less likely that they will start their second year as sophomores (which requires 30 credit hours). As highlighted in Table 3, a significant impact of the summer on-line mathematics classes is that they enable most students who take them to start their academic program in a much better place to complete the first year of their curriculum on schedule. It is also clear that most students who are placed into Introduction to Algebra by the Accuplacer TM exam are perfectly capable of successfully completing College Algebra. This likely reflects the fact that an adaptive test like the Accuplacer TM magnifies the effect of mistakes early in the test, perhaps too easily sending students down a track where it is not possible for them to get a score that will place them in the higher level math class.

Table 3: Change in course taking pattern of SMILE students
For the students who enrolled in the on-line summer math classes, a big impact is the difference in the mathematics course that they take in their first semester on campus. For each academic year, the first column represents the number of our SMILE students in each fall mathematics course based only on their placement scores. The second column for each year is the number of SMILE students who took each class after taking a summer on-line math class. Our students who take the on-line mathematics classes over the summer have the same or better outcomes as students taking the courses in the traditional format, both in the on-line class itself, and in their following mathematics courses. Our outcomes are somewhat contrary to what has been reported earlier for under-represented and under-prepared students taking on-line classes (Kaupp, 2012;Xu & Jaggars, 2014). The behavior of the students taking our on-line mathematics classes provides some insight into why our results may be better than is typical for students taking on-line classes. On the average, students in the summer classes do more assignments and appear to be more engaged than students taking those courses on-line during the academic year. Over three years, 76.5% of the 53 students who took the summer Introduction to Algebra classes completed 80% or more of course assignments, while in the spring 2015 on-line course (taught by the same instructor), only 68.6% of the 70 students completed 80% or more of the course assignments. The average passing rate was 71% for the summer courses versus 61.4% for the academic year class. The behavior of students in the on-line College Algebra courses was even better, with close to 85% of 49 students in the on-line courses completing more than 80% of the course assignments. Because the students are still at home they are insulated from the distractions of university peer culture, and may still be under the watchful eyes of their parents. At the same time, they are eager for the next phase of their life to begin, so doing homework for a college class may have an appeal that it will never have again. The students' largely self-directed effort pays off for them in success in the on-line course itself, and in a more solid foundation that increases their success in the following courses. This is particularly true for College Algebra, as students moving on from the on-line class performed much better in trigonometry than the other students in the class ( Figure 5), most of whom were taking the course as sophomores having spent their first year taking Introduction to Algebra and College Algebra.

Fall
Over the long term, students in our NSF-funded SMILE program did better than other STEM students in three major outcomes of the freshman year: retention in the major, cumulative GPA, and total number of hours earned. While that trend partially reflects the peer mentoring and other support elements of the SMILE program, the results from the Fall 2012 and Fall 2013 cohort compared to the Fall 2010 and 2011 cohorts clearly show the value added for students who take the summer on-line classes. While not a silver bullet for the problem of students underpreparedness in mathematics, our program is successful for most students who participate, is cost-effective for both students and the institution, and can be scaled to serve large numbers of students. Moreover, the student participants in our pilot project were 87% minority and a majority of them were first-generation college students, indicating that this approach is a viable model for vulnerable and at-risk populations.