The effects of implementing recitation activities on success rates in a college calculus course 1

: This study investigates the effects of using different types of recitation sessions with a large-enrollment section of a college calculus course on student achievement, success rates, and first-year retention. Over a period of six years, three different types of recitation sessions were implemented into the large enrollment section of a calculus course. During the fall semesters, the results on the departmental final examination, the DFW rates, and the one-year retention rates of students as STEM majors were examined by the type of recitation session used. The three types of recitation sessions studied were: (1) optional mentoring sessions at the Math Assistance Center conducted by undergraduate students (peer mentors), (2) required mentoring sessions conducted by graduate students, and (3) required VGNA (Verbal, Graphical or Geometric, Numeric, and Algebraic) Concept activities, which were also coupled with mentoring sessions conducted by graduate students. The success of the students in the large enrollment section of the course, which included one of the three different types of recitation sessions, was compared to the success of students in the small enrollment sections of the course (enrollments less than 50 students). The results of this study demonstrate methods of raising student success rates in large enrollment (lecture-format) courses.


Introduction
In order to retain and graduate more STEM majors on campus, this study investigated the effects of using different types of recitation sessions with a large-enrollment lecture-format first semester college calculus course on student achievement (as measured by the departmental final examination), student success rates in the course (as measured by the DFW rates), and first-year retention in a STEM discipline (as measured by the number of students who, after taking the first semester calculus course in the fall, were retained in a STEM discipline a year later).
Freshman-level courses, such as calculus, often act like a filter instead of a pump for science, technology, engineering, and mathematics (STEM) majors (Steen, 1988).In the United States, more than 40% of students fail their first-year mathematics course (Wieschenberg, 1994).
Students participating in exit interviews after dropping out of engineering at Purdue University cite difficulty with their first mathematics course as a primary reason for leaving (Budny, LeBold, & Bjedov, 1998).At Indiana University Purdue University Indianapolis (IUPUI), this effect of calculus being a filter is apparent.In recent years, IUPUI has launched an effort called Gateway to Graduation to improve student learning and retention in courses with high first-year student enrollment.With campus recognition of retention issues in Gateway courses, the investigators of this study obtained NSF funding (#0969500), to adapt and adopt best practices, in order to increase student retention in, and persistence toward graduation for, STEM majors.
IUPUI is an urban research and life sciences university, formed in 1969, that serves a large (31,000 students) and diverse student population.Approximately 62% of the entering students each year are first generation college students, 16% of the student body belongs to a minority group, and well over 90% are commuters.Moreover, 32% of students who begin their collegiate experience at IUPUI graduate from the institution within six years, and slightly over 70% of the freshman class are retained to the sophomore year.Over the last few years, the institution has focused on improving graduation and retention rates, giving primary attention to freshman retention and success initiatives.These initiatives include increased residential oncampus housing, a flag system where faculty can alert advisors during the semester of individual students facing difficulties, and discipline-specific peer mentoring centers for Gateway courses (freshman courses with high enrollments and high DFW rates).
To meet national goals of STEM student retention and four-year graduation, it is desirable to graduate approximately 25% of all STEM majors each year, controlling for enrollment growth and other factors.In 2008, 13% (457) of current IUPUI STEM majors received a bachelor degree.This is low when compared to the IUPUI campus average of 16% and IU Bloomington's campus average of 20%.This low percent of the IUPUI STEM student body that graduates is a result of a retention and persistence problem that begins with the firstyear retention rate of 63% for STEM majors, compared to 70% for the campus; where calculus is a required course in the first-year STEM curriculum.
As IUPUI has become a more established institution, student enrollment has increased faster than the rate of university funding and departmental growth.These issues have led to limited resources such as the number of faculty and classrooms available for any one course.This has also resulted in increased section size in entry-level calculus courses over time.Specifically, the number of students enrolling in first semester calculus in the fall semester steadily increased from 271 to 362 students during this six-year study, while the number of faculty and the number of rooms available to teach mathematics on campus has remained fixed at 6 each semester.This problem increases when considering the total enrollment in all mathematics courses, which has grown from 5,928 in 2005, to 7,643 in 2014 (a 29% increase in 10 years)making the classroom and faculty shortage growing problem.Therefore, the investigators of this study, with support from the NSF's STEM Talent Expansion Program (STEP), explored pedagogical methods to improve student success in large-enrollment sections of calculus to, thereby, increase first-year retention and, in turn, raise graduation rates in STEM related departments.Specifically, the primary goal of the STEP project is to increase the number of undergraduate STEM degrees awarded at IUPUI by 10% per year for five years (an increase of an additional 782 STEM graduates by 2015).Therefore, to meet this goal, the investigators of this study looked for ways to increase student success in calculus by examining how students learn mathematical concepts.Watt, J.X., Feldhaus, C.R., Sorge, B.H., Fore, G.A., Gavrin, A.D., & Marrs, K.A.

Basis in Learning Strategies
Calculus concepts are often taught primarily through algebraic representations.The typical student learns how to manipulate functions algebraically to solve the vast majority of calculus problems that would appear on a test.If a student becomes proficient at algebraic manipulation, often referred to as "plug-and-chug," the student will pass the course and be promoted to the next course in the sequence.However, this traditional way of understanding calculus via algebraic representation is limiting, often missing the big ideas of calculus and, thereby, reducing its problem solving nature to plug-and-chug assignments.
Likewise, a purely algebraic approach may fail to account for the complexity of the cognitive processes by which students learn calculus.A student's way of thinking is influenced by how the student develops an understanding of the content.In particular, effective ways of thinking develop through a learning approach undergirded by the principle that students need intellectual stimulus, or what Harel (1997) has termed, the Necessity Principle.Once an intellectual need relative to a mathematical concept is established, an understanding of that concept begins.Purely algebraic conceptual representations may fail to adequately establish this "need" in all learners.Moreover, students' ways of thinking influence how they understand concepts.A way of understanding a concept in a mathematics course is part of a student's "concept image."Concepts imagesthe intellectual representations of concepts whether correct or incorrectare developed through multiple perspectives, and have profound effects on comprehension (Tall & Vinner, 1981;Vinner, 1992;Harel, 1997).To ensure that valid calculus concept images are constructed, multiple representational forms should be utilized in ways that both correct faulty mental pictures of concepts and construct sound mathematical frameworks.
Therefore, the challenge is to teach calculus concepts via multiple representations, so that students learn multiple ways of describing a concept (Douglas, 1986;Gehrke & Pengelley, 1996;Goerdt, 2007;Pilgrim, 2010;Ross, 1996;Smith, 1994Smith, , 1996;;Stewart, 2012;Tucker, 1996).Through multiple representations the student develops and inculcates a conceptual understanding, which will have a profound effect on comprehension and problem-solving ability (Harel, 2004).Not surprisingly, current and preceding Calculus education reforms have identified the importance of addressing issues in the learning of Calculus from multiple representations.For this reason, the VGNA Concept activities were implemented into the large lecture section's recitation periods during Phase 3. Since the 1989 Tulane Conference, Calculus reform has been concerned with, amongst other things, "numerical, graphical, and modeling problems through the use of computers, open-ended projects, writing, applications and cooperative learning" (Ganter & Jiroutek, 2000).The VGNA Concept model seeks to address these concerns of the reform movement.
The individual representations of the VGNA Concept model have been met with some success.For example, the discussion (verbal) that occurs in recitation groups regarding calculus problems has, in some studies, been shown to improve student achievement (Norwood, 1995;Bonsague, 1994;Treisman, 1985).Additionally, the use of writing (placing your verbal understandings in writing) in assignments throughout the curriculum has been growing based on research-based recommendations and assumptionsfirst published in a large-scale study of British schools by Britton et al. (1975) that writing is important not only for communication Watt, J.X., Feldhaus, C.R., Sorge, B.H., Fore, G.A., Gavrin, A.D., & Marrs, K.A. Journal of the Scholarship of Teaching and Learning, Vol.14, No. 4, October 2014.josotl.indiana.edu4 but also for the discovery of ideas and a holistic understanding of a subject.Writing in calculus courses has also been shown to promote the construction of conceptual understandings and new knowledge (Beidleman, Jones, & Wells, 1995;Cooley, 2002).Student learning has also been demonstrated through the implementation of graphical or geometric representations and the discussion of those forms (Weller et al., 2003;Monk & Nemirovsky, 1994).The use of graphical representation is a key organization and learning strategy of many students; however, in calculus except to draw a graph of a function or construct rectangles under a curve to approximate area, the geometric arguments taught to students are minimal.Additionally, numeric data or representation of problems is almost non-existent in the teaching of calculus; yet most real world modeling problems involve analyzing data or measurements in a table.Therefore, the teaching of calculus is primarily done with one perspectivealgebraic understanding, which the authors of this study believe misses the big conceptual understandings of Calculus.
It was observed during the first two years of implementing required recitations as part of the large lecture section of the course (Phase 2) that the recitation instructors, all graduate students in mathematics, focused almost exclusively on algebraic representations of solving problems with little conceptual discussion.Furthermore, many of the students in these recitations were observed in a passive learning mode.The inclusion of collaborative activities in recitation grows from educational theory that group work necessarily involves the articulation of goals as well as ideas, and that these meta-cognitive activities improve learning and retention (Hillocks, 1986;Bruffee, 1984).Academic environments that are more competitive than collaborative, and those that rely on student learning occurring "spontaneously" during lectures, have been associated with "decreased interest or disaffection resulting in little or even regressive change" (Bonsangue, 1994, 121).Significant studies have demonstrated that mathematics students placed into recitation-type collaborative groups experience greater academic success (Bonsangue, 1994;Springer et al., 1999;Herzig & Kung, 2003).Similar successes have also been documented in other STEM fields.Collaborative work on physics "tutorials" has been well documented by the University of Washington group (McDermott, 1994).In chemistry, Peer-led Team Learning is a well-established methodology based on small-group activities performed in recitation sections facilitated by perr mentors (Gosser, 1998).In an attempt to redesign the recitations into an active learning modality focused on forming conceptual understandings via multiple representations, a set of weekly VGNA Concept activities were developed for and implemented in Phase 3.
The VGNA concept activities were designed to 1) place the student in an active learning mode; 2) be done in groups of three or four students; 3) focus on conceptual understandings via multiple representations; and 4) take about 40 minutes to complete (half of the allotted time for each recitation session).The concept for each week's VGNA Concept activity was selected from the lectures corresponding to the same week.The activities were adopted and adapted from the course textbook's resource manual (Shaw, 2012).An example of one such activity follows.

Example: The Chain Rule (Chapter 2.5)
The VGNA Concept activity on the Chain Rule was designed to provide a justification of the Chain Ruleby interpreting derivatives as rates of changeand opportunities to use it in computing derivatives.During lecture, the students were presented two forms of the Chain Rule: x) and (dy/dx) = (dy/du)(du/dx).The activity begins with the students in each group discussing whether these two equations say the same thing.The students are directed to describe the equations in their own words and create a verbal representation (explanation) for their comparison.Each groups' verbal representations will eventually be shared with the entire class.Next the students are given a graph of two functions, f and g, and asked to graph the composite function, h(x) = g(f(x)), and find its derivative at several points based on the graphical estimation of the slope of the tangent line (the algebraic formulas are not given).The students are then asked to justify the Chain Rule with a geometric argument.The students are then given a table of numeric values (see Table 1) to determine the derivative of various composite functions using f and g.

Table 1
Numeric Values of Functions used in VGNA Concept Activity on the Chain Rule The activity concludes with the students finding derivatives of functions in algebraic representations.On all VGNA Concept activities, the algebraic representation is always done last.The activities are collected and graded, one score per group.
A quiz at the end of the recitation session provides formative evaluation to the recitation instructor about the level to which students have inculcated the activity's conceptual understandings.Two questions on the quiz that have proven insightful in assessing conceptual and notational understanding is: (1) Compute: (d/dx) sin x 2 and (d/dx) sin 2 x.
(2) Where do you stop when using the chain rule?Explain why this is false: This VGNA Concept activity on the Chain Rule is one of the more powerful activities in the course.The activity causes a lot of discussion and arguments during recitation because it challenges the understandings of almost every student in the class, from their use of basic notation (i.e., squaring the sine function or its argument), understanding of composite functions, to why the chain rule works.Once students have actively studied the verbal, geometric, and analytic conceptual understandings, we find they are much less likely to make the common algebraic errors on tests (Watt, 2013), leading to lower DFW rates, and higher one-year retention rates, as evidenced by the results of this study.

Methodology
The study involved 1,956 calculus students who, over a six-year period (three two-year phases), enrolled into either a small section of calculus or the large lecture section (see Table 2).The small sections of the calculus course had enrollments below 50 students per section (average was 46) with a traditional style of classroom presentation and discussion format.The large The recitation format evolved over time, but can be divided into three distinct phases lasting two years each.During Phase I (2007 and 2008), the students in the large lecture section (like all students in the course) had optional mentoring sessions at the Math Assistance Center conducted by undergraduate students (peer mentors).During Phase II (2009 and 2010), the students had required mentoring sessions (recitations) conducted by graduate students.A quiz was administered during recitation, and the score became part of the course grade.During Phase III (2011 and 2012), newly created recitation activities focused on developing mathematical concepts via an integrated Verbal, Geometric, Numeric and Algebraic understandings approach (VGNA Concept activities).These VGNA Concept activities were collected, graded, and became part of the course grade.

Participants
The students who enroll in the calculus course are predominately majoring in engineering, physical sciences, or mathematical sciences.They either place directly into the course from the Compass Math Placement Test (developed by ACT) or completed the precalculus course with a grade of C or better.In the fall semester, the majority of these students are freshman, and they are not likely to be familiar with the instructors or their methods of teaching when they register for the course.Therefore, the study only examined the fall semester over six years in an attempt to control for student selection patterns when registering and to more easily determine first-year retention rates.In addition, the student demographics were similar in all sections of the course (see Table 3).

Recitation Designs in the Large-Enrollment Section
As discussed earlier, IUPUI has experienced an increase in the number of students taking first semester calculus during the fall semesterfrom 220 in 2002, to 362 in 2012 (a 60% increase).However, the number of sections of the course remained constant at six because of faculty and classroom limitations.The department was able to secure one time slot in the lecture hall for one section of the course, and the increased enrollment each year was absorbed by this lecture hall section.The same tenured faculty member taught the large-enrollment section every fall semester during this study.In an attempt to allow students in the large-enrollment section an opportunity to ask questions and receive individual attention, various interventions were implemented in three phases (each phase was used for a two-year period of time).The recitations were led by graduate students in mathematics (16 total), all of whom took the department's TA Workshop required of all graduate students teaching in the departmentno graduate student was used for a second year in this study (the author was interested in how the VGNA Concept activities would be adoptable by new graduate students every year).

Control Group
The control group was the small-enrollment sections of the course in aggregate, comprised of five such sections every semester, each with fewer than 50 students.The student demographics of the control group are comparable to that of the large-enrollment section, except for 17 registered graduate students, of which none were enrolled in the large-enrollment section of the course.There were eight different instructors of the small-enrollment sections during the study (four were associate professors who taught one section every semester of the study)all but two were tenured faculty (each teaching once during the study).Therefore, the authors assumed the majority of the teaching in the control group was performed by experienced professors who have taught the course multiple times.

Assessment Instruments
Three measures were used to assess student success in the calculus course: the department final examination scores, the DFW rate, and the one-year retention rate after taking the calculus course.The DFW rate for the course, or a section of the course, is determined by dividing the number of students receiving a course grade of D, F, or W (withdrew from the course) by the number of students enrolled in the course at census.A freshman-level or general education course with a DFW rate above 30% is considered an at-risk course by University College (the freshman advising unit) for students at IUPUI.The majority of freshman-level mathematics courses (13 out of 22 in 2010) were considered at risk for freshman at IUPUI.The departmental final examination is written each year by the coordinator of the course (who did not teach the course during the period of this study).The exam is a paper-and-pencil open-response instrument, with the same number of items testing the same learning objectives each year.The instructors of the course do not see the final examination until the day of the exam.Students from all sections of the course take the departmental final examination at the same time and place during final exam week.The exams are then commonly graded (each instructor of the course grades one page of the exam, for all students in the course).

Results
For this analysis, an ANOVA test was run, using SPSS (Version 21.0), linear and logistic regressions were constructed to compare the control and treatment groups during the three phases of the research, no other explanatory variables were considered.The regression models looked at each phase of the research to determine differences in final exam scores, DFW rates, Calculus II pass rates, and student retention in STEM majors based upon the Calculus I enrollment section type.Figure 1 shows the mean final exam score by phase and section type.During Phase I, there was no statistical difference between the two sections types (p = .632).In Phase II (p<.001, t=3.546) and Phase III (p<.001, t=8.042), the treatment group performed significantly better than the control group on the course final exam.Additionally, the change in final exam scores from Phase I to Phase II (p < .01,F=8.503), and Phase II to Phase III (p = .018,F=5.620), demonstrated significant growth within the large enrollment section.When examining the DFW rates, the treatment group has always had a smaller DFW rate than that of the control group (see Figure 2).However, this difference was not statistically significant (p<.05) until Phase III [p<.01,Exp(B)=1.668].This finding is also consistent with the STEM majors in the course.As with the entire course population, the DFW rates for STEM majors in the treatment group were higher throughout each phase (see Figure 3), yet the difference was only statistically significant in Phase III [p=.01,Exp(B)=1.641].Student success in the follow-up course, Calculus II, was analyzed to see if there was a difference between the control and treatment groups during the three phases of implementation.Figure 4 shows the Calculus II passing rates of students, from each section type, who passed Calculus I. Students from Phase I, who were in the control group had a higher passing rate in Calculus II than the treatment group, but the difference was not statistically significant (p=.584).For Phase II, students involved in the treatment group had a higher Calculus II pass rate than the control group, but as in Phase I the difference was not significant (p=.227).However, in Phase  Figure 5 shows the retention rates for direct admit STEM majors from first to second year by Phase and section type.While the treatment group maintained a higher retention rate throughout all three Phases, the difference was not statistically significant throughout Phases I (p=.471) and II (p=.089).However, with the implementation of the VGNA model in Phase III, there was a statistically significant difference between the enrollment sections [p<.001,Exp(B)=3.87].

Discussion
The purpose of this study was to evaluate the effects of implementing recitations in a large-enrollment section of calculus on departmental final exam scores, DFW rates, and one-year retention rates of students in STEM majors.Over a six-year period, three phases of different recitation formats were studied and the results were compared to the small-enrollment sections of the course.

Department Final Examination Scores
The results suggest that students performed better on the final examination when recitations were required.This is attributed to the fact that student-learning outcomes increase with more active learning, which is more likely to occur in recitations than small or large enrollment classes.Hence, students required to attend recitations outperformed the control groups in Phases II and III and the students in the large enrollment class during Phase I, where recitations were optional.
There was a significant increase in student performance on the departmental final examination when VGNA Concept activities were added to the required recitations.Pedagogically, this was not surprising.When students are placed in an active role in developing their conceptual understandings via multiple perspectives (verbal, geometric, numeric, as well as algebraic), they are more likely to inculcate the concept at higher levels and, therefore, more likely to solve routine problems with higher proficiency and to apply the concept to new situations (increased problem solving).
In order to determine if the increase in final exam scores during Phase III might be a result of the final exam questions evolving to favor the treatment group over time (assessing more conceptual understanding of the content), a review of the final exam questions was performed to determine if the number, type, or difficulty level of the questions had changed over the three phases of this study.A faculty member who teaches calculus at another institution performed this review.It was determined that the number of questions did vary each year, but the concepts being tested and the method of assessing these concepts was identical every year; in addition, the difficulty level of those questions was consistent.

DFW Rates
When using DFW rates to measure student success, it is important to note that this rate is partly instructor specific; however, the DFW rate of any one instructor of the course did not vary more than 6 percentage points.An instructor's rate tended to be higher when given more than 40 students in a section, but the correlation of DFW rate to class size was not statistically significant.Between the two factors, the DFW rate is more instructor specific and less dependent on class size.This probably accounts for why department chairs often assign large-enrollment sections to instructors with overall low DFW rates.
Student success increased as students were placed into more structured and active learning modes.Using DFW rates as a measure of student success, the small-enrollment sections of the course fluctuated between 35% to 44% over the six-years, averaging 39%.In the largeenrollment sections, the DFW rates were the highest (35%) in Phase I when students were passive learners in watching the lecture and not required to attend recitations.In Phase II, the Watt, J.X., Feldhaus, C.R., Sorge, B.H., Fore, G.A., Gavrin, A.D., & Marrs, K.A. Journal of the Scholarship of Teaching and Learning, Vol.14, No. 4, October 2014.josotl.indiana.edu12 DFW rate dropped to 29%, when students were required to attend a recitation focused on placing them in a more active learning mode.In Phase III, the DFW rate dropped to 26%, when the VGNA Concept activities were implemented.
An important observation of the study is the comparison of the DFW rates between STEM majors and the entire population with respect to the recitation type.For the control group (small enrollment sections), the difference in the DFW rate between the STEM majors and entire population was never greater than 4 percentage points for all three phases of the study (not statistically significant).However, for the large enrollment sections during Phase III (both years), the DFW rates were 7 percentage points lower for the STEM majors when compared to the entire population in the same section.This finding that STEM majors (more so than other types of majors) are more likely to pass the course when the VGNA Concept activities were implemented has serious teaching and learning implications for retaining STEM majors.

Pass Rates in the Next Course
The effect of the VGNA Concept activities on student success goes beyond the first course in the Calculus sequence.All of the students in this study were tracked into the second course, where the grades from the two courses were compared for each student.The second semester calculus course does have small and large enrollment sections, and the large enrollment sections of the second course do have required recitations conducted by graduate students; however, the VGNA Concept activities were not integrated into the second course during the period of this study.Like most institutions, students are able to register for any section of the courseabout 62% of the students in the large lecture section in the fall selected the large enrollment section for the second course.
In Phase I and II, there was no significant difference in the pass rate (grades of A, B, or C) in the second course of those students who passed the first course, between control and treatment groups.However, in Phase III, the difference became significant, with those students in the treatment group (large enrollment sections using the VGNA Concept activities) passing at a rate 15-percentage points higher than the control group.In addition, the control group experienced a 6-percentage point drop in their success rate in the second course from Phase II to Phase III (there was no change from Phase I to Phase II).There may be two related dynamics that account for these observations in success rates in the second course.First, the Phase III students that participated in the VGNA Concept activities developed strong conceptual understandings, which have prepared them for future success in the second course.Secondly, the students using the VGNA Concept activities inculcated more than a strong conceptual understanding of the topics being taught; they also learned new ways of knowing or learning knowledge and then used these new ways of learning in the second course.Both of these dynamics may have placed the control students at a disadvantage in the second course, where the treatment students performed at higher cognitive levels, and the control students found it difficult to keep pace with the other students in the class.

One-Year STEM Major Retention
One of the important indicators of the number of future STEM graduates is the first-year retention rate, which sets the trajectory to graduation.IUPUI, like many institutions nationally, has more than half of its first-year STEM majors drop the program by the end of their freshman year.Often Calculus is blamed for this problem by being a filter rather than a pump.Therefore, retaining students as STEM majors one-year after the Calculus course was of interest in this study.All of the students identified as STEM majors during the first course in Calculus were tracked one-year after the course to determine whether they remained a STEM major.During Phase I and Phase II, the difference in one-year retention rates was not significant.However, during Phase III, the treatment group was 20% more likely (88% versus 68%) to remain a STEM major.This is a significant and meaningful result.The Calculus course should adopt more active learning strategies with multiple concept representations, not only to lower DFW rates and increase future success in the Calculus sequence, but to also retain more STEM majors in the first year.

Additional Notes
It should be noted that the 2011 control cohort had one section taught by a graduate student.This cohort was considered an outlier, affecting two of the control group's mean scores for that year.The mean scores affected were: (1) a lowering of the pass rate in the second calculus course, and (2) a lowering of the one-year retention rate.The authors concluded this was the result of an inexperienced teacher (graduate student), who either did not focus enough attention on the key concepts, did not set expectations high enough, or did not pace the course correctly (all common issues for inexperienced teachers).This resulted in too many students not being prepared for the next course.This is noted for the possible implications of assigning inexperienced teachers to the first course in Calculus.When the data from the 2011 control cohort taught by a graduate student was excluded from the analysis, the statistics did not significantly change the results or conclusions from Phase III, which combined all the control sections over a two year period).

Implications for Future Study
From a broader perspective, the primary goal of the STEP project is to increase the number of STEM graduates.However, the participants in the first phase of the study are just now reaching their six-year graduation date, so it will be several more years before this goal can be measured on all students participating in the three phases.In the meantime, students are being tracked into second and third year courses where there is an attempt to measure the effectiveness of the VGNA Concept activities on student success and retention.Specifically, it must be determined if students are transferring these conceptual learning strategies into other math and science courses, and if so, whether these strategies are influencing achievement levels.In addition, this study did not investigate the possible effects of VGNA Concept activities on student success and retention of students in small-enrollment calculus sections.Nor did the study take into account the effect of VGNA Concept activities on demographic characteristics of the students that participated.Further study on the impact of VGNA Concept activities on gender, students of color, stratified age groups, and STEM major would make for interesting research.

Conclusions
Over a period of six years, three different types of recitation sessions were implemented into the large enrollment section of the calculus course.Although each type of recitation session had different pedagogical models, the time on task (classroom seat time) was the same for all phases of this study.During the fall semesters, the results on the departmental final examination, the DFW rates, and the one-year retention rates of STEM students were examined by the type of recitation session used with the large enrollment section.The three types of recitation sessions studied were: (1) optional mentoring at the Math Assistance Center conducted by undergraduate peer mentors, (2) required mentoring conducted by graduate students, and (3) required VGNA (Verbal, Graphical or Geometric, Numeric, and Algebraic) Concept activities, which were also coupled with mentoring conducted by graduate students.The success of the students in the large enrollment section of the course, which included one of the three different types of recitation sessions, was compared to the success of students in the small enrollment sections of the course.The effects of using each type of recitation session on raising departmental final examination scores, lowering DFW rates, and raising one-year retention rates were examined.This study found that the most significant increases in student learning outcomes and one-year retention rates clearly occurred in the third type of recitation, in which students were taken out of their passive learning environments and integrated into environments of active learning (e.g. group work and collaborative learning) where, through the use of the VGNA Concept activities, knowledge construction occurred.
The increase in retention, persistence, and student graduation in STEM-related majors is critical to the nation's economic well-being.It is evident to educators, employers, and politicians that more students need to become successful in flowing through the STEM pipeline.As part of this effort, the mathematics education community has been working on making calculus a pump and not a filter in the STEM pipeline for more than 25 years; however, too many students today are still not being retained in STEM disciplines because of their initial calculus course.This study has demonstrated that, even in large-enrollment sections of calculus, the implementation of highly structured recitation activities that focus on placing the student in an active role of developing their conceptual understandings of mathematics via verbal, geometric, numeric and algebraic representations can increase the student success rate in calculus and increase the firstyear retention rate for STEM students.In time, it is expected that these increases will have an effect on the number of students completing a STEM degreeflowing from the pipeline and into the economy.

Figure 1 .
Figure 1.Results from departmental final examination, mean (stdev) by type of section.

Figure 2 .
Figure 2. Results from DFW rates by type of section for all majors.

Figure 3 .
Figure 3. Results from DFW rates by type of section for STEM majors only.

Figure 4 .
Figure 4. Pass rates (C or better) in Calculus II based upon Calculus I enrollment size.

Figure 5 .
Figure5.STEM one-year retention rates of direct admit STEM majors.

Table 4
Type of Faculty Teaching Sections of the Course