An Investigation of the Relationship between In-Class and Out-of-Class Efforts on Student Learning : Empirical Evidence and Strategy Suggestion

In this paper we explore and discuss an important research question in higher education – is there a trade-off relationship between in-class and out-ofclass efforts for students? We used an empirical model to test the trade-off hypothesis between these two efforts. We identified a trade-off between in-class and out-of-class efforts, especially for those students who do not perform well on examinations. We clarified possible reasons for this relationship in a lowerperforming student group and noted potentially harmful implications for higher education. We recommended that instructors work individually with students in setting appropriate goals for each exam and frequently offering feedback. Doing so can strengthen rapport between students and faculty, thereby enhancing students’ motivation to learn and confidence in utilizing faculty as a learning resource. We also recommended a classroom-based gameplay strategy to promote students’ motivation to learn and encourage their participation.


Introduction
In addition to an instructor's teaching skills and a student's quality, both in-class effort and outof-class effort are two primary factors in determining the extent of a student's learning outcome.Since these two major factors influence a student's acquisition of knowledge, we wondered what the relationship would be between them.We normally would expect students to jointly employ these two factors to pursue their knowledge--students not only frequently attend classes but also study regularly outside the classroom.The reason is that students receive information via the instructor's classroom lecture but need time to absorb and process it outside the classroom so that it can become their own knowledge.Nevertheless, a question raised here is: for some students, would it be possible for these two factors to trade off?Some students frequently skip classes but may study hard outside the classroom.For that reason, is it possible that these two factors may substitute for each other for these students?
Acknowledging that students are a diverse group, it is possible for in-class effort and outof-class effort to be complementary for some students while substituting for each other among other students.Thus, the overall effect will depend on which effect is dominant.The substitution of in-class effort and out-of-class effort during the development of knowledge may mean one of two things: (1) the instructor may lecture very well--clear, organized, and understandable such that relative to in-class effort, students do not need to devote much out-of-class effort to studying for the course; or (2) students may not understand the instructor's lecture mainly due to his/her poor instructional skills or inability to match students' needs for learning-hence, students may prefer to study by themselves outside the classroom rather than devoting more effort inside the classroom.
In addition, can gender be a factor in determining the relationship?According to findings from a previous study by Woodfield, Jessop, and McMillan (2006), male students missed classes more frequently than did female students.This evidence inspires us to reasonably suspect that inclass effort and out-of-class effort for male students may possibly substitute for each other.Our logic could be wrong, but the idea may be described as follows: when students who frequently skip a class still successfully pass the course, there is a possibility that they may work harder outside the classroom.In other words, these students' in-class and out-of-class efforts may substitute for each other.Moreover, according to data used for this study (see Table 5), lowerperforming students are more likely to be absent from classes than higher-performing students.While there are many reasons for being absent, lower-performing students may become frustrated, lose their trust and confidence in using faculty as a learning resource in the classroom, and elect to study by themselves outside the classroom rather than sitting inside the classroom.Therefore, we also reasonably suspect that in-class effort and out-of-class effort may likely substitute for each other among lower-performing students.To determine whether or not our thoughts may be correct, we further tested these two hypotheses.
Three research questions were developed for this study: (1) Are in-class effort and out-ofclass effort related?If yes, is the relationship based on complements or substitutes?(2) If the relationship is based on substitutes, who would most likely exhibit such behavior-higherperforming or lower-performing students?(3) Can gender be a factor in determining whether inclass and out-of-class efforts may substitute for each other?In addition to these three research questions we discuss three implied questions: (1) why would these two efforts trade off each other?(2) What would be the impact on higher education if these two efforts trade off each other?(3) What methods could be suggested to educators to assist students (especially lower-performing students) in achieving success?
Although this topic may be related to several researchers' work on the relationship between students' exam performance and lecture attendance (e.g., Anikeeff, 1954;Jones, 1984;Brocato, 1989;Park and Kerr, 1990;Van Blerkom, 1992;Gunn, 1993;Romer, 1993;Day, 1994;Durden and Ellis, 1995;Rodgers, 2001;Chung, 2004;Stanca, 2006;and Dolnicar, Kaiser, Matus, and Vialle, 2009), we focused on a different perspective--the relationship between in-class effort and out-of-class effort.We used the complementarity test to investigate the trade-off hypothesis between these two efforts.Our study findings will provide educators with a closer look at the relationship between students' in-class effort and out-of-class effort, offering new ways to understand student behavior and to assist them in achieving success.
In this paper we not only provide a detailed discussion of the results from our investigation of the relationship between in-class effort and out-of-class effort, but also offer a thoughtful discussion of why a relationship between these two efforts would arise and how instructors may assist students in attaining success.In addition, we divide data according to performance (i.e., higher-performing and lower-performing) and gender (i.e., male and female) to ascertain whether these groups behave differently-whether or not the relationship between in-class effort and outof-class effort serves as a substitute, which will be an important result as we seek to understand why some students skip classes more frequently, and what we can do about it, while others do not.More importantly, this paper extends the literature on the issue of student effort-investment and academic achievement in higher education.This paper makes a number of contributions to the higher education literature, because this study offers the first descriptive look at the use of the complementarity test to investigate this issue, linking the relationship between in-class and out-of-class efforts on student learning.In other words, this paper is beneficial to both faculty teaching and student learning in higher education.

A Brief Literature Review
While the relationship between in-class effort and out-of-class effort has not yet been investigated and discussed, it is related to the topics of motivation, effort, and academic performance.Motivation is like an engine that moves learning goals forward (Rothestein, 1990;Woolfolk, 1990).Without motivation, no matter how smart the student is, he or she will not learn anything.Students with differing motivations for enrolling in the same class will experience different levels of academic achievement (Nicholls, 1984;Dweck, 1986).That is, a student's motivation determines his/her effort-investment and hence academic performance.More clearly, study strategies (Elliott, McGregor, and Gable, 1999) and quality of effort (Plant, Ericsson, and Asberg, 2005) are important to a student's learning when time boosts learning.These studies indicate that quality of effort is determined by motivation.Below, we briefly review the literature in these three topics.
The topic of motivation has been broadly studied and defined by psychologists such as Maslow (1954), Aderman (1999), Murray, Poole, andJones (2006), andVansteenkiste et al. (2009).Motivation is an important concept in psychology that offers insight into why we may behave the way we do.It is an internal process that reflects our desire to achieve certain goals, such as academic achievement.Goal achievement requires motivation and more-a mediator is also needed, such as effort (classroom attendance, and the amount of time devoted to studying and completion of assignments) to achieve goals.Therefore, numerous researchers (e.g., Goodman et al., 2011;Kusurkar et al., 2013) investigated the relationship between students' motivation and academic performance as mediated by effort.
Motivation can be divided into two types: intrinsic motivation and extrinsic motivation.Findings from Goodman et al. (2011) indicated that both intrinsic and extrinsic motivations are positively and significantly associated with students' academic performance as measured by GPA (Grade Point Average).Both types of motivation could positively affect the amount of effort invested in academic performance.Thus, effort is an important mediator in linking the relationship between intrinsic/extrinsic motivation and academic performance, and intrinsic motivation is an even stronger predictor of academic achievement than extrinsic motivation.
While motivation includes intrinsic and extrinsic motivations, the quality values of motivation are quite different.Kusurkar et al. (2013) used self-determination theory to differentiate them into two categories: autonomous and controlled.Between these two motivations is a relative autonomous motivation, which is a measure of the balance between autonomous motivation and controlled motivation.Findings from Kusurkar et al. (2013) demonstrated that relative autonomous motivation positively and significantly affected students' study strategies and efforts and in turn influenced their academic performance.
As discussed earlier, efforts play a very important role as a mediator between motivation and academic performance.Efforts include in-class effort (i.e., lecture attendance) and out-of-class effort (i.e., the amount of time devoted to studying and completion of assignments outside the classroom).Several researchers (e.g.,Jones, 1984;Rodgers, 2001;Cohn and Johnson, 2006;Kember et al., 1995;Zulauf and Gortner, 1999) have mainly focused on the effects of efforts (either in-class or out-of-class effort) on academic performance.
The relationship between attendance (in-class effort) and grades can be indirect or direct (Jones, 1984).Since students have a motivation to earn a college degree, they will attend class, implying that attendance and grades may be indirectly related.However, attendance and grades can also be directly related; that is, more frequent attendance may lead to higher grades.Lower grades will discourage students from attending class; thus, attendance and grades will be directly related.In addition, heterogeneous variables, such as intelligence and motivation, among students might influence students' attending behavior and hence affect their performance (Rodgers, 2001).Moreover, the effect of all levels of attendance on student performance is not identical-low performance among non-attendees is particularly significant among those missing a large fraction of classes (Cohn and Johnson, 2006) In addition to in-class effort, researchers such as Kember et al. (1995) and Zulauf and Gortner (1999) focused on the impact of out-of-class effort on academic performance.Kember et al. (1995) found that whether students would spend more or less time on studying outside the classroom depended on their learning approach.Students using a surface approach had a higher lecture attendance rate and spent more time on studying and hence received higher grades.A deep approach showed that students did not receive higher grades unless they offered sufficient work.Findings from Zulauf and Gortner (1999) demonstrated that the amount of time spent studying outside the classroom exerted a positive and significant effect on students' academic performance as measured by GPA (grade point average).They also found that the time management score had a positive impact on GPA, implying that better time management skills could significantly improve students' academic achievement.
Above all, we learned that previous researchers had either studied the relationship between students' motivation and academic performance as mediated by effort or the relationship between effort (in-class or out-of-class effort) and academic performance.In this study, we investigated the relationship between in-class effort and out-of-class effort.In the following sections, we show our detailed investigation and discuss our findings.

Data
The data used in this study are the same as the data used in Lin (2014).Please refer to Lin (2014) for a detailed description of the data and experimental design.However, we still briefly illustrate the data and variables used in this study below.
Total participants in this study were 389 business students in Introduction to Microeconomics classes in spring 2007, 2009, and 2011, and fall 2012.These students were taught by the same instructor.The data sources include instructor-reported data, university-supplied data, and student-self-reported data (i.e., survey response).The instructor-reported data consist of students' attendance record (i.e., in-class effort) and three exam scores 2 (i.e., learning outcomes); while the university-supplied data contain students' GPA (grade point average) and SAT (Scholastic Aptitude Test) scores.In addition, the student-self-reported data included students' work hours for pay per week and out-of-class effort devoted to studying for the class during each exam.In a questionnaire, students were asked: (1) how many hours for pay do you work a week?(This question was only asked once in the first exam.)(2) Overall, approximately how long did you study for the class during this exam period?(This question was asked in each exam.)There were five choices for the question: (i) I study 1-5 hours before the test; (ii) I study 6-10 hours before the test; (iii) I study 11-15 hours before the test; (iv) I study 16-20 hours before the test; and (v) I study more than 20 hours before the test.
It should be noted that the variables of student-quality index (i.e., a combination of GPA and SAT scores) and work hours for pay per week were not used in the empirical model, but they were used for the discussion.Moreover, we used both GPA and SAT scores to create a variable of student-quality index.The GPA, regardless of a student's major, is a measure of a student's motivation and academic ability3 ; while SAT scores are a measure of a student's innate ability4 .To create student quality index, we first combined these two elements to measure a student's quality value.A simple way to combine these two is to use a multiplicative function of GPA and SAT score (i.e., a student quality value (QUA) = GPA SAT  ) 5 .We then converted QUA values into an index that may be described as follows: , where the maximum QUA value among all students. 6oreover, descriptive statistics (means and standard deviations) for variables used in this paper are presented in Table 1.In addition, Cronbach's alpha was 0.84, indicating strong internal consistency among these exams.

Econometric Model
The complementarity test has been broadly used in the economics literature to examine the relationship between labor and capital.Thus, we applied the complementarity formation used by economics researchers (e.g., Lin, 2003) to investigate the trade-off hypothesis between in-class and out-of-class efforts.The empirical model (i.e., the complementarity formation) is shown as follows: where t = the current exam period; t-1 = the last exam period; G = exam scores (i.e., learning outcomes), A = attendance (i.e., in-class effort), L = frequency of studying for an exam (i.e., outof-class effort), t u = stochastic disturbance terms assuming a mean 0 and a variance 2  .If in- class effort and out-of-class effort are substitutes, 0 and the effect should be significant.

Results
In this section we present our empirical evidence from Equation (1) for three hypotheses based upon our three research questions.Our presentation focuses on the coefficient ( AL a ), which is the most important result needed here.
Hypothesis 1: In-class effort and out-of-class effort are related, while the relationship can be complements or substitutes.For Hypothesis 1, the data included all 389 students.The results for Equation (1) for all students are reported in Table 2.In the earlier periods (Periods 1-2), as Column (1) of Table 2 shows, the coefficient ( AL a ) was not statistically and significantly different from zero at any level, implying that these two efforts are not related.However, in the later periods (Periods 2-3), as shown in Column (2) of Table 2, the coefficient ( AL a ) was negative and was statistically and significantly different from zero at the 10% level, implying that these two efforts are related and may be trading off each other.We now combine all three periods.As Column (3) of Table 2 shows, the coefficient ( AL a ) was also negative and was statistically and significantly different from zero at the 1% level.This was because the effect of the later periods may dominate the effect of the earlier periods.The overall result implies that in-class effort and out-of-class effort are significantly related and the relationship may be substitutes, implying that in-class and out-of-class efforts may trade off for some (or many) students.Consequently, overall, we had sufficient evidence to support Hypothesis 1-in-class and out-of-class efforts are significantly related for many students, and the relationship may be substitutes.The results for Hypothesis 1 leave two questions: (1) who would most likely exhibit such behavior-higher-performing or lower-performing students?(2) Why would the trade-off relationship only occur in the later periods (i.e., Periods 2-3) rather than in the earlier periods (i.e., Periods 1-2) or both?The first question is answered in the results for Hypothesis 2; while the second question is examined in the discussion section.
Hypothesis 2: The trade-off relationship between in-class effort and out-of-class effort would most likely occur in the lower-performing student group.In hypothesis 2, we split data according to higher-performing and lower-performing students based on their final course grades.We defined higher-performing students as students who received A or B grades for the course, while lower-performing students received C or lower grades (i.e., D and F grades) for the course.The results for Equation (1) for higher-performing and lower-performing students are reported in Table 3.In the earlier periods (Periods 1-2), as shown in Columns (1) and (2) of Table 3, the coefficient ( AL a ) is not statistically and significantly Lin different from zero at any level for both the higher-performing student group and the lowerperforming student group.In the later periods (Periods 2-3), while the coefficient ( AL a ) is still not statistically and significantly different from zero at any levels for the higher-performing student group, the coefficient ( AL a ) is negative and statistically and significantly different from zero at the 10% level for the lower-performing student group.We then combined all three periods.As Columns ( 5) and ( 6) of Table 3 show, the coefficient ( AL a ) for the higher-performing student group is still not statistically and significantly different from zero at any level, but the coefficient ( AL a ) for the lower-performing student group is negative and statistically and significantly different from zero at the 1% level.This is also because the effect of the later periods may dominate the effect of the earlier periods.The results imply that no remarkable relationship exists between in-class effort and out-of-class effort in the higher-performing student group; however, overall, the relationship between in-class effort and out-of-class effort may substitute for one another in the lowerperforming student group.In short, overall, we have sufficient evidence to support Hypothesis 2. The evidence shows that the trade-off relationship between in-class effort and out-of-class effort would most likely occur in the lower-performing student group.The results of Hypothesis 2 also raise the same question as for Hypothesis 1: why would the trade-off relationship occur for lower-performing students in the later periods (i.e., Periods 2-3) rather than in the earlier periods (i.e., Periods 1-2) or both?We look at this question in the discussion section.

Hypothesis 3: Gender is a factor in determining whether in-class and out-of-class efforts may
substitute for each other.For hypothesis 3, we split data according to gender.The results for Equation (1) for male and female students are presented in Table 4.In the both earlier periods (Periods 1-2) and later periods (Periods 2-3), as shown in Columns (1)-( 4) of Table 4, the coefficient ( AL a ) is not statistically and significantly different from zero at any level for both the male student group and the female student group.However, when we combine all three periods, according to the results shown in Columns ( 5) and ( 6) of Table 4, the coefficient ( AL a ) is negative and is statistically and significantly different from zero at the 1% level for the male student group and at the 5% level for the female student group.As a result, overall, we do not have sufficient evidence to support Hypothesis 3 because the evidence shows that in-class and out-of-class efforts may substitute for each other for both male and female students, implying that gender cannot be a significant factor in this issue.Therefore, Hypothesis 3 is rejected.

Discussion
The results offer sufficient empirical evidence to support the belief that in-class and out-of-class efforts may be trading off each other for some students.The evidence shows that lower-performing students are more likely to exhibit such behavior.Therefore, the question here is: why would these two efforts trade off each other for lower-performing students in the later periods rather than in the earlier periods?Before answering this question, let's take a quick look at three indicators-quality index, attendance rate, and work hours for pay per week for both higher-performing and lowerperforming students-which could help us to understand the empirical results.As Table 5 shows, based upon the one-tailed test (the upper tail test) 7 , both the quality index and attendance rate for higher-performing students are statistically and significantly higher than for lower-performing students, while lower-performing students' work hours for pay per week are statistically and significantly longer than higher-performing students'.The information on work hours for pay per week may offer an answer.A number of researchers have investigated the impact of student employment on school performance.The impact can be positive, negative, or unrelated.Several researchers, such as Singh (1998), Eckstein andWolpin (1999), andOettinger (1999), have shown negative effects of part-time employment on school performance; while others have found that academic achievement may improve with low work hours but drop with long work hours (e.g., Schill et al., 1985; Lillydahl, 1990; Quirk et   7 We used the one-tailed test (the upper tail test) by formulating null ( 0 H ) and alternative ( a H ) hypotheses: , and  , 2001).However, researchers, such as Schoenhals et al. (1998), Warren et al. (2001), andDustmann et al. (2007), did not find a relationship between student employment and school performance.
A related study by DeSimone in 2008 showed that students who worked 30 hours or more each week had significantly lower grades.As shown in Table 5, the average number of work hours for pay per week for lower-performing students was over 30 hours.No surprisingly, they performed poorly because their work hours significantly crowded out their time for academics. 8or example, they have less time to study for the class and their employment schedules may sometimes conflict with class schedules so that they have to skip classes.Moreover, their quality index was lower than that for higher-performing students.Therefore, the only way to improve their grades was to reduce work hours and significantly devote their efforts to activities both inside and outside the classroom.
However, things usually do not work as expected.First, it is impossible for students to reduce their work hours or quit their jobs because without work, they may not be able to afford college.A survey done by King and Bannon in 2002 indicated that 63% of full-time working college students admitted that they would discontinue their academic careers if they stopped working.Second, lower-performing students may be less motivated, especially when they are not interested in the class.More importantly, they probably cannot grasp the lecture immediately.Thus, after the midterm, they may become less willing to learn more reluctant to attend the class.If they do not get assistance right away to solve their questions, they probably will not grasp the next lecture and will have even more questions.This is because concepts and models taught in earlier lectures may be used in later lectures.Not understanding an earlier lecture makes it difficult to move on to the next lecture.As questions accumulate, students eventually lose interest and choose not to frequently attend future classes.As Table 6 shows, based upon the one-tailed test (the lower tail test)9 , the attendance rate for lower-performing students significantly dropped from 88.46% in the earlier first exam period to 80.04% in the later third exam period.Since they may believe that they cannot catch up or grasp the lecture, it is possible that they may elect to study outside the classroom by themselves.Then, when they receive lower grades such as D or F on the midterm, they become frustrated and fail to utilize faculty as a learning resource.This further decreases their motivation and willingness to learn.Consequently, these students reduce their inclass effort while increasing out-of-class effort, resulting in a trade-off relationship between inclass and out-of-class efforts after the midterm.In addition, the evidence implies that many lower-performing students may believe that they will perform better if they study outside the classroom and so do not attend the instructors' lectures, since they can find the information in textbooks and/or online.The evidence shows that they are not exactly correct since their grades still do not improve after following this plan.Much of the information from instructors' classroom lectures may not be found in textbooks and online.Therefore, even though these students may try to study hard outside the classroom, they still may not do well on exams.Thus, we may inquire: what would be the impact on higher education if these two efforts trade off each other?Based upon the empirical evidence found in this study, the trade-off relationship does not seem to have a positive impact on higher education because it may result in more students (especially lower-performing students) attempting to skip classes more frequently, thereby increasing the gap in performance-inequality values between higherperforming and lower-performing students.That is, higher-performing students always attend classes and receive important information and thus perform well, which encourages them to make progress; while lower-performing students frequently skip classes and miss important information and hence perform poorly, which discourages their progress.As mentioned earlier, a lot of information from classroom lectures may not be found in textbooks and online, which means that even if students who skipped a number of classes try to study hard outside the classroom, they may not necessarily have access to missed information.Although we have seen some higherperforming students who frequently skip classes still perform well in that class, we do not encourage this behavior.The empirical evidence studied by Lin (2014) demonstrated that even higher-performing students' exam performance progress may be significantly decelerated by skipped classes.

Strategy Suggestion
Since the trade-off relationship does not seem to have a positive impact on higher education, do educators have access to methods that may be used to assist students (especially lower-performing students) in achieving success?Students must have many reasons for missing classes, but one possible reason for some students' lower performance is a loss of trust and confidence in instructors when they do not understand their lectures.Eventually, these students may choose to skip future classes more frequently.In addition, lower-performing students' reluctance to ask questions in class is a common problem.To assist lower-performing students, instructors must be very patient and may take the initiative to contact and meet with them.Lower-performing students may feel more comfortable asking questions in professors' offices.That is, educators need to build a strong rapport with students in order to improve their learning motivation and ameliorate their trust and confidence in using faculty as a learning resource (Granitz, Koerning, and Harich, 2009;Young, 2005;Huff, Cooper, and Jones, 2002).Young (2005) provided a detailed discussion about enhancing a student's motivations to learn.As pointed out by Young (2005), an enthusiastic teacher should offer students elevated personal interaction with supportive feedback, clear goals, and expectations focusing on learning over grades; doing so will enhance students' motivation to learn.
Based upon the idea described above, instructors should work individually with students who perform poorly on the first exam in setting appropriate goals for the next exam (or, if possible, work individually with each student in setting appropriate goals at the beginning of the semester).Elliot, McGregor, and Gable (1991) demonstrated that achievement goals are important predictors of motivational study strategies and exam performance.Without clear achievement goals, students lack clear motivations and study strategies and perform poorly.
Therefore, instructors may ask students questions such as: "What is your goal for the next exam?""Why did you set that goal?" "How are you going to reach your goal?" "Why do you think that you can reach that goal?"After the goals are set, instructors need to monitor students and provide them with feedback (such as office hour conversations) from time to time and talk with them individually to ensure that they are still committed to their goals and being conscientious about their work, and answer their questions.As Trautwein, Ludtke, Roberts, Schnyder, and Niggli (2009) noted, conscientiousness and competence beliefs are significant indicators of academic effort and achievement.For those students who abandon their goals, instructors need to determine the reasons for decreased commitment and aid them in modifying or resetting goals to make them more appropriate to their level of effort/ability.Note that feedback is necessary in assessing whether students' goals are still effective and ensuring their continued commitment to them.Lacking feedback, students may be unaware of their progression or regression, making it more difficult for them to determine the level of effort required to effectively achieve their goals.After each exam, instructors still need to talk to those students individually.Students should be congratulated on reaching their goals; instructors should then work with them individually in setting new goals for the next exam.Students who did not attain their goals should receive assistance in identifying reasons and developing possible solutions.More importantly, instructors need to encourage and work further with them in setting more appropriate new goals for the next exam.
In following this strategy, students (especially lower-performing students) will believe that professors care about and are concerned for them and want them to learn.In addition, students will comfortably ask questions of, and seek guidance from, their professors.As a result, students may develop personal connections with their professors and hence strengthen rapport between students and faculty.Engaging in this course of action will enhance students' motivation to learn and confidence in utilizing faculty as a learning resource.More importantly, a strong rapport between students and faculty will also simultaneously increase the benefits of studying for a class.
In addition to the goal-setting strategy, professors may also provide active learning exercises, such as the use of game-play in the classroom, to promote students' motivation to learn and encourage students' participation (e.g., Mahmoud and Tanni, 2014;Lin and Dunphy, 2013).As noted by Lin and Dunphy (2013), game-play learning may serve as an important pedagogical method.The intent of game-play is to promote students' motivation to learn/interest in learning models and help them easily learn abstract and complicated concepts.
In short, while it may not be possible for students to reduce their work hours, students' motivation to learn may improve and hence their in-class effort may, too.They may both frequently attend classes and study regularly outside the classroom.As a result, students will eventually achieve their goals and receive better grades (say, A or B grades), indicating that they are making progress in gaining knowledge, and that the relationship between in-class and out-ofclass efforts is now complementary.

Limitations
This research has two limitations-(1) quality of effort; and (2) time spent for studying-that could possibly result in some potential errors.We illustrate these two limitations below.

Quality of Effort
The effort used in this study is "quantity of effort" rather than "quality of effort".While quantity of effort can be simply measured by the amount of time, quality of effort cannot be.This is because "amount of time" is not a strong predictor of learning outcomes due to the way time is being used, whether sitting in the classroom or in a library studying.Plant, Ericsson, Hill, and Asberg (2005) demonstrated that the amount of time invested by college students in a class cannot be a good predictor of academic performance.In a classroom, some students fall asleep, spend the hour texting, play on their computers, or otherwise completely check out, etc.
We understand the disadvantage of using the amount of time to measure effort, but it is a challenge to find an accurate proxy to measure "quality of effort" because "quality of time" cannot be simply observed and measured.It is not an easy task for the instructor simultaneously to observe and record who is concentrating and who is not, while focusing on teaching.In addition, it is even more difficult for an instructor to measure a student's quality of effort outside the classroom.
Although "quality of effort" is needed for the study, it cannot be easily observed and measured, creating a significant challenge in data collection.To collect such data, we will have to develop a strategy that maintains the confidentiality and anonymity of student data and also avoids any possibility of negativity.These will be left to future research on this issue.

Time Spent for Studying
In addition to the quality of effort limitation, time spent studying is another limitation in this study.Time spent studying outside the classroom cannot be observed by the instructor-thus, students' self-reports have to be adopted.For that reason, it is possible that some students may deliberately upward-bias self-reporting in this context, which would lead to an error.Fortunately, the regression model used in this paper requires differencing data that may either eliminate the error or minimize its significance.

Conclusions
In this study we explored and discussed in depth an important research question in higher education (Do in-class and out-of-class efforts trade off?).We used an empirical model (the complementarity formation) to test the trade-off hypothesis between these two efforts.In light of the empirical results, three major findings are offered: 1.For many students, in-class and out-of-class efforts are significantly related and may substitute for each other.2. For lower-performing students, in-class effort and out-of-class effort are more likely to substitute for each other.3. Gender is not a factor when ascertaining whether in-class and out-of-class efforts may substitute for each other.Moreover, we clarified the possible reasons for the occurrence of the trade-off relationship in the lower-performing student group and looked at why this relationship could harm higher education.We suggested that instructors work individually with students in setting appropriate goals for each exam and frequently offer students feedback.Our suggestion is based on goalsetting theory10 which has been widely and successfully used in the workplace as a means of improving and sustaining work performance (e.g., Milliman, Zawacki, Chulz, Wiggins, and Norman, 1995) and also has been applied in environments in which students are pursuing knowledge, such as with English-as-a-Foreign-Language (EFL) and English-as-a-Second-Language (ESL) programs (e.g., Jackson, 1999, Cooper, Horn, andStrahan, 2005;Kitsantas, 2004).Following this strategy may strengthen rapport between students and faculty, thereby enhancing students' motivation to learn and confidence in utilizing faculty as a learning resource.In addition to the strategy of goal-setting, we also suggested the strategy of game-play in the classroom to promote students' motivation to learn and encourage students' participation.
In summary, the main contribution of this study to higher education is our verification of an important fact: a trade-off relationship may exist between in-class and out-of-class efforts for many lower-performing students.Verifying this fact offers educators new ways to understand student behavior and use that understanding in assisting them to achieve success.In addition, this study may also be used to explain why online classes become so popular to many students, especially those who are employed, since their employment schedules may sometimes conflict with class schedules or they have less time for travel to campus and for rest.For that reason, they would prefer to study by themselves outside the classroom rather than sitting inside the classroom to learn lectures from instructors.Therefore, ways in which to apply this research to the topic of online classes may be an interesting topic for a future study.
mean for attendance rate in the higher-performing student group; mean for quality index in the higher-performing student group; L QI   = the population mean for the quality index in the lowerperforming student group; = the population mean for working hours per week in the higher-performing student group; and L WH   = the population mean for working hours per week in the lower-performing student group.
population mean for attendance rate in the exam period 3 in the lower-performing student group; 1 L AT   = the population mean for attendance rate in the first exam period in the lower-performing student group.13.20% 80.04% 20.47% 0.000**** Note: M = mean; SD = standard deviation; and n = total observations.****Denotes statistical significance of the p-value at the .001level.

Table 1 :
Means and Standard Deviations of Variables

Table 2 :
Results for Hypothesis 1 -All Students Note: Number in parentheses is t-value.All periods are the combination of Periods 1-2 and Periods 2-3, so the observations are double of the initial observations.***p< .01;**p< .05;*p< .10

Table 3 :
Results for Hypothesis 2 -Higher-and Lower-Performing Students

Table 4 :
Results for Hypothesis 3 -Male and Female Students

Table 5 :
Results of One-Tailed Test (Upper Tail Test) Note: M = mean; SD = standard deviation; and n = total observations.****Denotes statistical significance of the p-value at the .001level.