Self-Regulation and Feedback in an Educational Statistics Course

: The purpose of the study was to evaluate the effects of feedback timing and quality on student self-regulatory behavior. Using a non-equivalent control-group design, students were assigned to either an immediate-detailed or delayed-grade-only feedback condition within the online statistics homework program, Aplia. The groups were then compared on levels of self-regulation and ratings of perceived homework effectiveness. Results indicate no significant differences between groups. Implications and the need for updated measurements are discussed.


Introduction
It is understood that a major goal of higher education is to assist students in becoming selfregulated learners.A self-regulated learner is one that possesses the ability to generate internal feedback from external feedback, and then use that information to realign the resulting product with the intended goal (Nicol & MacFarlane-Dick, 2006;Nicol & Milligan, 2006;Pintrich, 2004).Higher education instructors should use formative feedback practices that encourage the development of self-regulation skills, thereby empowering students (Nicol & MacFarlane-Dick, 2006) and preparing them for independent learning beyond graduation and throughout their lives (Boud, 2000;Dunn, Lo, Mulvenon, & Sutcliffe, 2012).To this end, Nicol & MacFarlane-Dick (2006) developed seven principles of feedback practice to which they suggest academic assessment should adhere.These include helping students understand expectations of good performance; helping students develop self-assessment and self-refection skills; providing students valuable information about their learning; encouraging teacher-student and student-peer dialogue about learning; inspiring both positive self-esteem and motivational beliefs; providing students multiple opportunities for repeated practice; and providing useable information to teachers to help mold their teaching.
Homework and examinations have been the traditional means of assessing student progress throughout their coursework and, along with other forms of assessment (e.g.projects, portfolios, presentations), have also been utilized to assess student learning (Nicol & Milligan, 2006).More recently, assessment content has been delivered via computer programs and through websites (Aleven, McLaren, Roll, & Koedinger, 2010;Azevedo, 2009;Biswas, Leelawong, Schwartz, Vye, & Teachable Agents Group, 2005;Greene, Moos, Azevedo, & Winters, 2008;Perry & Winne, 2006;Winne & Nesbit, 2009).Nicol and Milligan (2006) contend that although computer-based assessment differs from traditional paper-pencil classwork and homework, technology-supported assessment can also fit the criteria of good feedback practice, and in turn help students develop necessary self-regulation skills.
One popular computer-based assessment is the online homework program Aplia, which

Literature Review
Self-Regulation and Feedback Self-regulation.Self-regulated learning (SRL) has been identified as a process, rather than an ability, that is used by learners to effectively utilize their intellect to reach academic success (Zimmerman, 2002).It is a combination of the learner's emotions, thoughts, and behaviors that, when used effectively, lead to goal attainment (Zimmerman, 2000).While engaged in the academic task, learners proceed through three established phases of self-regulated learning: 1) forethought, wherein learners' goal-setting and planning interact with expectations, goal-related values, and self-efficacy; 2) performance, wherein learners use self-control measures such as sustaining focused attention and monitoring progress; and 3) self-reflection, wherein learners evaluate learning and, subsequent to performance, make attributions and adjust strategies as a means of furthering progress (Balapumi & Aitken, 2012;Zimmerman, 2002;Zimmerman & Campillo, 2003, Zimmerman & Schunk, 2011).
As Pintrich (2004) further explains, SRL involves four commonly accepted assumptions.The first, which comes from the cognitive perspective, is that students actively construct knowledge.This means that students utilize information presented in the external environment in conjunction with their own thoughts, beliefs, and ideas (the internal environment) to produce their own meanings, learning goals, and learning strategies.The second assumption is that students have the potential for control.This refers to their potential ability to monitor and regulate aspects of their internal environment (e.g.their own thoughts, motivation, and behaviors) as well as control some aspects of their external environment.Not all students will exercise such controls, but the potential is assumed to exist.The third is the assumption that within every learning situation exists a goal, standard, or criterion against which students will compare their progress.Comparisons are made to help students evaluate whether learning is actually occurring, or if necessary changes are needed by administering the use of selected SRL strategies.For example, a student who is studying for a mathematical theories exam might use flashcards to assess her knowledge of the material to determine if adequate preparations were made for an upcoming exam.If the student recognizes gaps in her knowledge, she may choose to modify her study methods and use strategies that will result in successful achievement outcomes.The fourth assumption is that actions involved in selfregulation function as mediators between both internal and external characteristics, and achievement.In other words, what students choose to do to attain self-prescribed goals is based on As reflected in these assumptions, many factors work together to inform and support students' self-regulation, an interaction that is well explained by Bandura (1991).One such factor influencing self-regulation is feedback (Bandura, 1991;Hattie & Timperley, 2007).Feedback originates both internally, and externally.Internal feedback is generated by learners' self-judgment of personal performance and how said performance compares to self-established goals, while external feedback may be relayed in the form of an instructor's assessment of the students' academic product (Bandura, 1991, Hattie & Timperley, 2007).Feedback provided by instructors should be timely, targeted, and goal-directed to enhance, rather than hinder, students' ability to self-regulate (Ambrose, Bridges, Di Pietro, Lovett, Norman, 2010).As Bandura (1991) notes, learners cannot effectively judge their own progress if they have no clear idea of how well they are performing.Indeed, as students engage in the learning process, they are creating their own feedback-loop; information received will later be used to modify learning strategies and improve their effectiveness.The more skilled at self-regulation a student is, the more complex is this internal feedback-modification loop.Students who are less skilled at monitoring their own progress are more dependent on external sources of feedback (Hattie & Timperley, 2007).Therefore, quality of instructor-feedback is paramount.

Feedback
There are varying levels of feedback: information relayed about the task, information given regarding processing the task, information transmitted about students' self-regulation, and information communicated with respect to students themselves.Task level feedback describes how well the specific learning task is being completed and is the foundation upon which self-regulation is established.Task level feedback is conveyed when discussion about the correctness of answers, how to ascertain more information, and how to expand surface knowledge occurs (Airasian, 1997;Hattie & Timperley, 2007;Van der Kleij, Feskens, & Eggen, 2015;Winnie & Butler, 1994).Therefore, task level feedback serves as a corrective function but may only be effective for lower order learning (Van der Kleij, Feskens, & Eggen, 2015).
Task processing feedback deals with mechanisms that undergird, communicate, and extend a task -processes the learners complete in attempts to perform the task.This level of feedback is reliant on learners' perceptivity as well as the utilization of personal techniques to detect errors.Therefore, task processing level of feedback is more effective at fostering deeper learning practices.This is because it scrutinizes strategies students use to detect errors, a mechanism that supplies useful information leading to the modification of faulty techniques.Additionally, it is useful in illuminating the need to seek help (Balzer, Doherty, & O'Connor, 1989;Purdie, Hattie, & Douglas, 1996;Carver & Scheier, 1990;Van der Kleij, et al., 2015).
Self-regulation feedback refers to how learners govern actions that support personally adapted goals.This form of feedback is couched within a learner's sense of personal agency and serves to improve learners' abilities to pursue, receive, and accommodate rather than exhibit defensive attitudes against feedback.A learner's ability to self-assess improves attentiveness to tasks and aids useful interpretation of feedback information (Cleary & Zimmerman, 2001;Zimmerman, 2000;Zimmerman & Bandura, 1994).
Lastly, feedback about self-as-a-person (such as praise) is possibly the most commonly used form of feedback, yet it is the least effective and provides the least information about Williams improvement in task comprehension, task engagement, and self-efficacy (Hattie & Timperley, 2007).Feedback about self merely relay data about the characteristic of the learner, not the task, or the effective processing of the task.However, feedback about self-as-a-person may positively impact learners' motivation and task persistence (Bond, Smith, Baker, & Hattie, 2000;Hattie & Timperley, 2007;Van der Kleij, et al., 2015), so it is not without value.
Accordingly, Nicol and Milligan (2006) set forth seven principles of good feedback in the context of technology-supported assessments.The aim is to provide useful principles to help students in web-based environments develop into self-regulated learners.As previously stated, these seven principles of good feedback include: helping students understand expectations of good performance; helping students develop self-assessment and self-refection skills; providing students valuable information about their learning; encouraging teacher-student and student-peer dialogue about learning; inspiring both positive self-esteem and motivational beliefs; providing students multiple opportunities for repeated practice; and providing useable information to teachers to help mold their teaching.Three of these principles (helping students develop selfassessment and self-refection skills, providing students valuable information about their learning, and providing students multiple opportunities for repeated practice) appear to be well represented in Aplia, thereby warranting further discussion.

Aplia and Principles of Good Feedback
The first principle supported by Aplia, helping students develop self-assessment and self-refection skills, comes through repeated practice.Students are already involved in self-monitoring and making corrections as they compare their academic product against the standard, making judgments about the quality of their work.However, instructors may build on this practice by creating more opportunities for practice, further developing students' capacity to self-regulate (Boud, 2000;Pintrich, 1995).Specifically, instructors may create and administer low stakes quizzes, granting students repeated opportunities to gauge their understanding of the topic under study.Simulation assessments that provide immediate feedback may communicate to the learner information about performance in real-time.This valuable information provides greater positive impact on self-regulation as students are granted opportunities to immediately self-reflect and assess their actions to remediate errors (Thomas and Milligan, 2004;Wiggins, 2001).Aplia conforms to these ideas by providing students with ample practice exercises that are repeatable (by clicking the "try again" button) throughout the course.Additionally, students may click on a practice question and be taken to explanations as well as be further directed to specific sections of the textbook for reading.
Providing students valuable information about their progress is a process that uses external feedback to help learners gain an awareness of their strengths and weaknesses.This awareness provides the impetus to acknowledge and address gaps in learning, another instance in which feedback supports self-regulation (Pintrich, 1995).External feedback from teachers offers a standard against which students can compare their internal development of goals.Teachers providing feedback about specific aspects of the academic task is associated with higher order learning for it encourages students to recognize and correct misconceptions (Nicol & Macfarlane-Dick, 2006).Further, good quality external feedback fosters self-assessment and self-correction of academic performances, shrinking the chasm between students' intentions and effects resulting from the feedback (Freeman & Lewis, 1998;Nicol & Macfarlane-Dick, 2006).Accommodating this principle, Aplia appropriately provides students with a drop-down box under each problem Williams that gives detailed information about how to approach the solving of that problem.Presumably, this guidance will help students modify any erroneous ideas, allowing them greater success on subsequent attempts and on similar problems.
Lastly, the principle of providing students multiple opportunities for repeated practice leads to changes in learning behaviors and improves learning outcomes.Using external feedback to improve on the academic product, such as granting students repeated opportunities to re-submit corrected assignments, positively influences learning behaviors and closes the feedback loop and gaps in performance (Boud, 2000;Sadler, 1989;York, 2003).To help students apply external feedback and reduce the performance gap, feedback should be communicated while the work is in progress and assignments should be allowed to be submitted at different stages of completion so that feedback may be used to improve upon subsequent submissions (Gibbs, 2004;Hounsell, 2004).Aplia addresses this principle nicely in that students are given three opportunities to attempt each problem offered.After choosing their answers for each problem, students can click the "grade it now" button and see their level of success for that problem.By then clicking on the "explanation" drop box, students receive guidance about the correct solving of the problem, and then are given the opportunity to select "try another version".It is reasonable, then, to conclude that students' performance will improve with each attempt, thereby closing the gap between students' understanding and expected performance.
In addition to these principles, the quality and value of external feedback may also be judged based on its timeliness.Aplia offers two modes of feedback utility: immediate feedback, and delayed feedback.Immediate feedback allows students to assess their own performance via the "grade it now" button, wherein students receive their grade immediately upon submission of a graded assignment.The delayed feedback function comes into play when students' assignments are instead graded automatically at the time and date due (similar to traditional hand-grading by a teacher or assistant).Under this function, students are not provided with correct answers or explanations about how to arrive at them.Both types of feedback timing clearly have implications for students' self-regulation.

Immediate Feedback and Delayed Feedback
The definition of timing of feedback (immediate versus delayed) is varied, and may explain why studies report generally inconsistent findings on the impact of feedback's timing ( Van der Kleij, et al., 2015).Notwithstanding, in formative assessments, Shute (2008) defines immediate feedback as information delivered during the course of task completion, or information delivered instantaneously after the learner responds (completes a task or responds to an item), whereas delayed feedback is delivered subsequent to the learner's response, after the passage of time.This span of time varies in length from a few hours to several weeks.However, in computer based environments, immediate feedback is information relayed automatically after students respond, as in Aplia's immediate feedback mode, while delayed feedback is information that is not immediately relayed (Van der Kleij, Feskens, & Eggen, 2015), such as in the case of Aplia's delayed feedback mode.
While learning a task, instantaneous feedback about the task (such as correction of task errors) may result in enhanced learning rates, while immediate feedback about the processing of the task (correction of errors while learning to the point of automaticity) may distract learners (Bangert-Drowns, Kulik, Kulik, & Morgan, 1991;Hattie & Timperley, 2007).For example, Lemley, Sudweeks, Howell, Laws, and Sawyer (2007) explored feedback type and timing in a Williams sample of 352 distance learning high school students, with one group using a paper-pencil mailin-based mode (delayed feedback) and the other using a computer-based mode (immediate feedback).The groups were compared on their final exam scores in four courses (English, History, Health, Exploring Values), as well as their time to complete the courses.The authors found that distance learners receiving immediate feedback fared significantly better on final exams in both the English course (t(56)=3.13,p=.003) and the Exploring Values course (t(178)=4.08,p=.0001) than those receiving delayed feedback, while no differences were found for History and Health courses.However, students receiving delayed feedback completed English (t(56)=3.11,p=.003), History (t(54)=3.73,p=.001), and Health (t(56)=2.43,p=.018) courses within a significantly shorter span of time.
In contrast, Van der Kleij, Eggen, Timmers, and Veldkamp (2012) place 152 undergraduates into three conditions: immediate knowledge of correct response with elaborate feedback, delayed knowledge of correct response with elaborate feedback, and knowledge of results with no knowledge of correct response and no feedback.Even though they found no significant differences in outcomes between conditions, they did find that students in the first group (immediate knowledge of correct response with elaborate feedback) invested significantly more time in reading feedback than those who received delayed feedback, F(2,147) = 24.40,p<.001, ƞ 2 =.25, with only the group receiving immediate feedback showing significance.Inconsistencies such as these may be due to the differing outcomes being measured.As Shute (2008) points out, immediate feedback appears to foster lower order learning outcomes while delayed feedback best supports higher order learning outcomes.Even so, feedback timing appears to have at least some impact on student learning.
In light of previous research concerning the potential influence of feedback, the current study seeks to investigate whether students' self-regulatory behaviors are affected by the immediacy and quality of feedback provided by the online homework program Aplia.As a means of validation, students' subjective opinions of the effectiveness of the program, in terms of feedback and appropriateness, are also considered.Results of this study will assist in judging the beneficial effect of feedback type and timing on students' self-regulation for learning as experienced in statistics classes.
Since students play a major role in guiding in their own learning processes, and since SRL involves the combination of learners' emotions, thoughts, and behaviors, the reception of feedback varies from one learner to the next.Therefore, feedback must be purposefully and thoughtfully incorporated into instruction to have any appreciable impact on learning outcomes (Bangert-Downs et al., 1991;Hattie & Gan, 2011;Nicol & Macfarlane-Dick, 2006;Stobart, 2008;Timmers & Veldkamp, 2011;Zimmerman, 2000).The principles of good feedback, when effectively utilized, leads to the type of online instruction that helps to facilitate the development of selfregulated online learners.Varying levels of feedback along with the timing of feedback may be structured into the online learning module, as in the case of Aplia, to strengthen impact as well as increase students' receptivity to feedback to improve learning and achievement outcomes.
Therefore, to assess the effects of feedback immediacy and quality on both self-regulation and perceived effectiveness, the following null hypotheses will be addressed: 1) There will be no difference between feedback groups in reported self-regulatory behaviors in terms of planning, monitoring, and resource management as measured by the General Strategies for Learning (GSL) scale.
2) There will be no difference between feedback groups in reported self-regulatory behaviors in terms of monitoring their own progress, identifying misunderstandings in their Williams learning, and determining strategies to clarify such misunderstandings, as measured by the Clarification Strategies for Learning (CSL) scale.
3) There will be no difference between feedback groups in students' reports of overall effectiveness of the Aplia online homework program, as measured by the Assessment Experience Questionnaire (AEQ).
4) There will be no difference between feedback groups in students' reports of overall effectiveness of the Aplia online homework program, as measured by the Effectiveness of Computer Based Assessment (ECBA) questionnaire.

Participants
During each semester of the 2013-2014 academic year, four sections of a graduate-level introductory educational statistics course in a large Southwestern university were asked to participate in the current study.The same instructor taught all sections, and no students declined to participate.Of the 72 participants, 49 (68.1%) were female and 23 (31.9%) were male, with a mean age of 30.68 (̅ = 29.90,SD = 9.75) years.Forty-one (56.9%) were white, 10 (13.9%) were Hispanic, 6 (8.3%) were Asian, and 12 (16.7%)reported their ethnicity as "other".Thirty-six (50%) were master's students and 36 (50%) were doctoral students.Though students were not directly asked, enrollment records indicate a variety of majors were represented, including educational psychology, counselor education, sports psychology, higher education, family/consumer sciences, forensic sciences, and hospitality administration.
Before the study began, the four sections were randomly assigned to one of two online homework conditions: immediate feedback or delayed feedback.Two of the sections (fall 2013, n = 25, and summer 2014, n = 13) experienced the delayed feedback condition, while the other two sections (both in spring 2014, n = 34) experienced the immediate feedback condition.All students received weekly homework assignments that reflected each week's lesson, and the assignments were due at 11:00pm the night before their scheduled class day the following week.All other aspects of the course were the same for all students (syllabus, topics, and exams).
Students in the immediate feedback condition (n = 34) received Aplia-generated problems based on the textbook used for the course.Once they worked a problem, the students would receive instant feedback on the correctness of the solution, along with an explanation of how to work the problem correctly if their first attempt was incorrect.They were then given an opportunity to work the problem again, with a similar problem being presented.After three attempts, students were required to move on to the next problem in the homework problem set where they were again offered instant feedback, explanations, and three opportunities to work alternate problems of a similar nature.Alternate problems were similar in that the problem type was the same, but the situation given and data involved would change.
Students in the delayed feedback condition (n = 38) received the same number and type of textbook problems, but did not receive feedback on correctness of their solutions until the immediate passage of the date and time the assignment was due.For example, if the assignment was due at 11:00pm on a Sunday, Aplia automatically graded the assignments at that time.This meant that regardless of what day during the week individual students completed their assignments, they were required to wait until the due date/time arrived to receive feedback.Students in this condition received information about the correctness of answers after grading (i.e.Williams which ones were correct and which one were not correct), but no explanation of how to work the problems and no opportunity to work alternate problems was given.

Instruments and Procedure
Near the last day of classes, students were given a questionnaire designed to collect demographic information, to assess their levels of self-regulation, and to gather their opinions concerning the overall effectiveness of the homework assignments.Self-regulation was measured using the General Strategies for Learning scale (GSL) and the Clarification Strategies for Learning scale (CSL) (Dunn, Lo, Mulvenon, & Sutcliffe, 2012).The GSL consists of five questions designed to measure students' aptitude for self-regulation in the areas of planning, monitoring, and resource management in a way that requires intrinsic motivation.The CSL consists of three questions designed to measure students' aptitude for monitoring their own progress, identifying misunderstandings in their learning, and determining strategies to clarify such misunderstandings.Questions for both the GSL and CSL are answered on a 5-point-Likert scale with choices ranging from "strongly disagree" to "strongly agree", and the scales are scored using the mean across items.Reliability coefficients for these subscales in the Dunn et al. (2012) study were .74(GSL) and .61(CSL), while in the current study the reliability coefficient for GSL was lower at .60 and higher for CSL at .71.
Student opinions of the overall effectiveness of the Aplia homework assignments were gathered using five subscales of the Assessment Experience Questionnaire (AEQ, V3.3) (Gibbs & Dunbar-Goddet, 2007).The AEQ consists of nine subscales addressing various aspects of the assessment environment.Four of these (coverage of the syllabus, deep approach, surface approach, and learning from the examination) were not included in the current study as they were judged to be general assessment questions rather than questions about specific assessment techniques.The subscales used (quantity of effort, quantity and quality of feedback, use of feedback, appropriate assessment, and clear goals and standards) contained a total of 14 questions and offered answer choices on a 5-point Likert scale ranging from "strongly disagree" to "strongly agree".The AEQ was scored by taking the mean across all items.Factor analysis was provided as validity evidence for responses, with all items loading at 0.5 or above on the appropriate subscales.Reliability coefficients for the subscales used were reported by Gibbs et al. (2007) as .69(quantity of effort), .61(quantityand quality of feedback), and .70(use of feedback) and by Ramsden (1991) as .71(appropriate assessment) and .80(clear goals).Cronbach alpha values for the AEQ subscales in the current study were generally higher at .80 (quantity of effort), .75(quantity and quality of feedback), .75(use of feedback), .75(appropriate assessment), and .70(clear goals).
Students' opinions of the effectiveness of the Aplia homework was also collected using the Effectiveness of Computer Based Assessment Questionnaire (ECBA) developed by Workman (2004).The questionnaire assesses students' perceptions of the effectiveness of computerbased/aided education (CBE/CBA), and consists of 15 questions answered on a Likert scale ranging from "strongly disagree" to "strongly agree".Scores were obtained by calculating the mean across items.Evidence of a modest criterion-related validity for responses was established by Workman (2004) through correlation between perceived effectiveness and performance with test scores from both computer-based education (r = .56,p < .01)and computer-aided education (r = .26,p < .05).Though Workman provides no information regarding reliability of this instrument, Cronbach's alpha for the ECBA in the current study was high at .95.

Williams
At the beginning of each semester, students were assigned 15 homework assignments using the Aplia program.At the close of each semester, students were awarded 1 participation point for each completed homework assignment provided their score was passing with at least a grade of 60%.This practice was used to discourage students from attempting only the first question and abandoning the assignment to gain the participation point for that assignment with minimal effort.In order to ensure students would be more likely to benefit from the feedback provided, homework questions were selected that reflected lower or mid-level difficulty (Clariana, et al., 2000).Additionally, care was taken to make sure students in both the immediate and delayed feedback conditions received homework questions that were as similar as possible in type, quantity, and difficulty.Periodically throughout the semester, students were given 3 exams over content covered in class and on homework assignments.Exams were given in paper-and-pencil format regardless of feedback condition.

Results
The two sections in spring 2013 experienced the immediate feedback condition and represent the experimental group.Since both sections met weekly for 14 weeks, it was assumed that students in these sections would not differ due to class meeting patterns.The fall 2012 and summer 2013 sections experienced the delayed feedback condition, and served as the control group.The fall semester entailed 14 weekly class meetings, while the summer semester students attended class daily for 4 weeks.It was recognized that this difference in attendance patterns could potentially influence students' self-regulation and their perceptions of assignment effectiveness.Therefore, one-way multivariate analysis of variance (MANOVA) was performed between the two sections to test whether the dependent variables of self-regulation (as measured by GSL and CSL) and perceived effectiveness (as measured by AEQ and ECBA) differed.Results of this analysis indicate no significant differences between the two sections on the set of dependent variables (λ= 0.652, F (8, 29) = 1.939, p = 0.09).Therefore, the fall and summer sections were assumed to be indistinguishable in terms of the dependent variables and their data was therefore combined to create one control group.
Before addressing hypotheses, descriptive data concerning the research variables were calculated.Table 1 displays the group means and standard deviations for both the immediate and delayed feedback groups on the GSL, CSL, AEQ, and ECBA.Data indicate students receiving immediate feedback reported a lower mean on propensity for self-regulation, and put forth slightly less effort on their assignments than those receiving delayed feedback.Yet the same students report higher means for use of feedback, appropriateness of the assessments (assignments), and clear goals and standards.Significance of these observations was then probed through further analyses meant to address the hypotheses.
The first goal of the study was to determine whether there would be significant group differences in students' self-regulatory behavior in two areas: 1) general strategies for learning (GSL), and 2) clarification strategies for learning (CSL), and is reflected in the first two hypotheses.To make this determination, and considering the relatedness of the variables, MANOVA was initially chosen to address these hypotheses jointly, utilizing student feedback group as the independent variable, and self-regulation scores from the GSL and CSL as the dependent variables.As a precursor, Pearson's r correlation was conducted to ensure the dependent variables were appropriately related, thus confirming that an acceptable level of variance would be shared (Tabachnick & Fidell, 2007, pg. 244).The results indicate that considerable variance is shared between the two dependent variables (r = .47,p = .000).Though the level is within the acceptable range of r = .20-.60 proposed by Meyers, Gampst, & Guarino (2013, pg. 228), Tabachnick & Fidell (2007, pg. 244) suggest caution when variables are moderately correlated as the power of MANOVA is likely to be jeopardized.In light of this caution, separate independentsamples t-tests were conducted to address the first and second hypotheses independently.The first t-test involved feedback group as the independent variable, and GSL scores as the dependent variable.Results indicate no significant difference between feedback groups on GSL scores (t =1.43, df = 70, p = 0.156).The second t-test also utilized feedback group as the independent variable, and CSL scores were used as the dependent variable.Results of this t-test also revealed no significant differences between feedback groups on CSL scores (t = 0.86, df = 70, p = 0.395).Therefore, there was insufficient evidence for rejection of hypotheses one and two, suggesting that regardless of feedback type students' propensity for self-regulation was not affected.
The second goal of the study was to determine whether there would be significant group differences in students' perceptions of effectiveness of the Aplia online assignments, and is reflected by the third and fourth hypotheses.To this end, the third hypothesis was evaluated with an independent t-test with feedback group as the independent variable and ECBA scores as the dependent variable.Results confirm no significant difference between the groups on ECBA scores (t = 0.015, df = 70, p = 0.988).Therefore, no evidence was found for the rejection of the third hypothesis, implying no group differences in student's perceptions of effectiveness of the Aplia computer-based assignments.
The fourth hypothesis also concerned group differences in students' perceptions of adequacy of the Aplia assignments across five measures of effectiveness.A one-way MANOVA was selected using student feedback group as the independent variable, and perceived effectiveness scores from the five subscales of the AEQ as the dependent variables.Initially, Pearson's r Williams correlations were performed among the five dependent variables to confirm acceptable levels of correlation between the variables.Table 2 presents these correlations.With the exceptions of the relationships between quantity of effort (AEQ-QE) and quantity and quality of feedback (AEQ-QQF), and quantity of effort (AEQ-QE) and clear goals and standards (AEQ-CGS), all correlations were significant and in the range of acceptability (r = .20-.60) recommended by Meyers, et. al. (2013, pg. 228).Again, Tabachnick & Fidell's (2007) admonishment of power issues was considered, and given the strength of the relationship between the five dependent variables, these power issues were expected.However, with the infeasibility of conducting multiple t-tests as a method for assessing the fourth hypothesis, it was decided to continue with the chosen MANOVA to address the fourth hypothesis and accept the potential threat to power.Box's M was then conducted with results suggesting the covariance matrices between the two groups could be assumed to be equal (Box's M = 21.90, p = 0.164), and satisfying the assumption of homogeneity.Results of the MANOVA indicate that there was no significant difference between the two feedback groups in the set of AEQ scores, λ = 0.92, F(5, 66) = 1.09, p = 0.38, partial η 2 = 0.076.Therefore, there was inadequate evidence for rejection of the fourth null hypothesis, and it was determined that feedback type did not influence students' perceptions of assignment effectiveness.

Discussion and Conclusions
The purpose of the current study was to assess whether students' self-regulatory behaviors were affected by the timing and quality of feedback offered within the online homework program Aplia, referred to throughout the study as feedback type.Students' subjective opinions of the effectiveness of the program, in terms of feedback and appropriateness, were also evaluated.Results suggest that whether students experienced immediate and detailed feedback, or delayed and grade-only feedback, their reports of self-regulatory behaviors were not significantly different between groups.Also, students reported no significant group differences in effectiveness or appropriateness of the Aplia assignments.
The lack of group differences was initially unexpected when considering Aplia's adherence to at least three of Nicol's & MacFarlane-Dick's (2006) principles of good feedback.The principle Williams Second, reliability for GSL and CSL were low, indicating the potential influence of using the MSLQ and its derivatives outside of the intended classroom context.Researchers of future studies are advised to be mindful of this hindrance when considering these instruments for the study of self-regulation in students using online homework programs.
A third and related potential limitation of the current study may be inherent in the online environment itself, and the less-than-appropriate usage of instruments meant for face-to-face classroom assessment.In the current study, self-regulation was measured using the modified scales of the GSL and CSL (Dunn, Lo, Mulvenon, & Sutcliffe, 2012) that were based on two of the Learning Strategies subscales of the MSLQ (Pintrich, et al., 1993), which has historically been used to measure SRL.However, the MSLQ was originally developed to assess motivational orientation among college students in traditional classroom settings, as was the modified version of the two subscales used in the current study.Although the MSLQ was developed within a specific learning context, the instrument is commonly used to measure the self-regulatory learning processes of students engaged in online learning (Artino, 2008;Dunn, Rakes, & Rakes, 2014;Greene & Azevedo, 2010;Kirmizi, 2013).Additionally, to make the MSLQ applicable within the online environment, some researchers have made adjustments to the instrument.These modifications, as well as use of the original instrument outside its intended context, have resulted in findings that are inconsistent with real world expectations (Artino and Stephens, 2009;Matuga, 2009).Confounding results from these and other studies seem to indicate that perhaps the use of the MSLQ, and instruments derived from it such as the GSL and CSL, may not be an appropriate assessment of self-regulation in online learning environments.With the understanding that SRL is context bound, it seems justifiable that any assessment tool used to measure online SRL should also be representative of that particular learning environment.Therefore, this potential limitation of the current study can be said to provide evidence for the need of an updated instrument for measuring SRL in the online learning environment.
Fourth, the analyses used to evaluate the fourth hypothesis was, as expected, markedly deficient in power.With the MANOVA used to evaluate this hypothesis at a mere 0.363 observed power value, any significant differences between groups that did exist were likely not detectable.For example, when the differing group means on AEQ-Quantity of Effort are considered (4.00 vs. 3.75), diminished power is a possible explanation for the lack of significance.One possible reason for this lack of power is the moderate correlations between the dependent variables (Tabachnick & Fidell, 2007, pg. 244).Results of the current study should therefore be received with these potential limitations in mind, and may be considered as guideposts for the improvement of future studies.