Blended Instruction in Symbolic Logic

This study examines through action research whether blended instruction in an upper-level philosophy course in introductory symbolic logic can help undergraduate philosophy students to achieve better learning outcomes than undergraduate philosophy students in a traditional, face-to-face version of the same course. The authors conclude that the change from traditional instruction to blended instruction did have a positive and significant effect on student learning as measured in course grades and student assessment scores for one course learning objective, as well as a positive but non-significant effect on student assessment scores for two additional course learning objectives.

Charles Graham (2006) also helpfully categorizes blended instruction according to the goal of the blending: enabling blended instruction incorporates online instruction primarily to make instruction available to those who could not otherwise obtain it or to allow students to obtain instruction more conveniently; enhancing blended instruction uses online instruction primarily to allow students to access additional learning activities or materials online that are not accessible through face-to-face instruction; and transforming blended instruction uses online instruction to make possible intellectual activity that would not have been possible through face-to-face instruction alone. The present study examines whether transforming blended instruction in an upper-level philosophy course in introductory symbolic logic can help undergraduate philosophy students to achieve better learning outcomes than undergraduate philosophy students in a traditional, face-to-face version of the same course.
The research question arose in the context of an introductory symbolic logic course that the primary investigator teaches roughly annually to an enrollment of 10 to 25 undergraduate students, most of whom are majoring in philosophy. Students in traditional, face-to-face sections of the course frequently raised the concern that they needed to spend more time in class on face-to-face problemsolving activities. In response to this concern, the instructor changed the mode of course delivery from a traditional face-to-face course to a blended course. In the traditional sections, face-to-face class meetings were divided between face-to-face lectures about the course material and face-to-face problem-solving activities focused on exercises selected by the instructor. In the blended sections, students viewed online lectures and participated in an online discussion forum in preparation for faceto-face class meetings. In the online discussion forum, students indicated which of the assigned exercises gave them trouble and why they found those exercises difficult to complete. The instructor then reviewed the online discussion prior to each face-to-face class meeting, responded to individual discussion posts that raised issues that could be addressed with a brief comment, and selected exercises identified as challenging by several students to address in problem-solving activities in the face-to-face class meetings. Face-to-face class meetings were then devoted exclusively to problem-solving activities focused on types of exercises that multiple students identified as challenging. The instructor's motivation for moving from traditional delivery to blended delivery of the course was to see whether course-level, transformational blended instruction would promote student learning as reflected in student grades and student performance on the 4 course learning objectives.
These studies provide a helpful context for evaluating blended instruction in higher education, since students are likely to learn more effectively and instructors are likely to teach more effectively through a mode of instruction with which they are more satisfied. The perceptions of students and instructors about the effectiveness of blended instruction cannot provide the whole story about student learning, however, since these subjective perceptions do not provide objective evidence about whether students are learning what their instructors hope they will learn through blended instruction.
Various studies take different approaches to obtaining objective evidence about the effectiveness of blended instruction in promoting student learning. Some, such as Clark et al. (2016), observe the behavior of students in the classroom in order to determine whether students exhibit desired learning behaviors more frequently in the classroom of a blended-instruction course than in that of a traditional-instruction course. Such studies also provide helpful evidence that students in blended-instruction courses are doing regularly the kinds of activities that tend to lead to positive learning outcomes. These studies do not necessarily establish, however, whether learning actually results from these kinds of activities in the blended-instruction courses in question because they provide no direct evidence about this question.
Other studies examine student course grades in blended-instruction courses in order to determine whether blended instruction promotes student learning in these courses (Alducin-Ochoa & Vásquez-Martínez, 2016;Larson & Sung, 2009;López-Pérez, Pérez-López, & Rodríguez-Ariza, 2011;Napier, Dekhane, & Smith, 2011). This is a more helpful method of inquiry, since if students have learned more through blended instruction than they would have learned through traditional face-toface instruction alone, then their greater learning should be reflected in improved course grades. Consequently, the first goal of the present study is to determine whether the change from traditional delivery to blended delivery of an upper-level philosophy course in introductory symbolic logic for undergraduate philosophy majors can improve student grades in the course. In regard to this goal, the authors propose the following hypothesis: H1. Blended delivery of an introductory course in symbolic logic for undergraduate philosophy majors has a significant and positive effect on students' course grades.
It is important to keep in mind, however, that according to the definition of "blended instruction" provided by McGee and Reis, quoted above, the goal of blended instruction is not only to help students learn more in general, but also to help students accomplish specific learning outcomes (McGee & Reis, 2012). Course grades are a crude measure of the degree to which students have fulfilled specific learning outcomes. For one thing, two different students could earn the same course grade while fulfilling various course learning outcomes to different degrees; the students' course grades by themselves would provide us with no evidence about whether students had fulfilled some course learning outcomes more fully than others. For another thing, course grades are often affected by factors such as student participation in class discussions or students' on-time submission of course assignments that may not be directly relevant to whether students have fulfilled any specific course learning outcome. Consequently, course grades are not the best source of evidence about the degree to which students have fulfilled specific course learning outcomes.
Other studies seek to assess whether blended instruction improves student learning with respect to specific course outcomes. Some studies seek indirect evidence of student learning through students' self-reports about how much they learned from blended instruction either exclusively (Rix, 2011) or in combination with direct evidence about student learning (Cabi, 2018;Jeong, Cañada- Cañada, & González-Gómez, 2018;Utts, Sommer, Acredolo, Maher, & Matthews, 2003). Student opinions about what helps them learn are a valuable source of evidence about student learning, since it is the students themselves whom blended instruction seeks to benefit. Students' self-reported perceptions of their learning are not, however, the best source of evidence of what they have learned, since presumably the students' instructors are in a better position than the students to determine whether the students have fulfilled specific course learning outcomes.
Other studies have examined direct evidence of student learning with respect to specific course learning outcomes. In some cases, such studies have found that blended instruction helps students to fulfill course learning outcomes to a greater degree than traditional instruction (Alsancak Sirakaya & Ozdemir, 2018;Kozikoglu, 2019;Sun & Wu, 2016;Vernadakis, Antoniou, Giannousi, Zetou, & Kioumourtzoglou, 2011), but other studies found no significant difference between student learning in blended-instruction courses and traditional-instruction courses (Cabi, 2018;Roscoe, 2012;Utts, Sommer, Acredolo, Maher, & Matthews, 2003). This mixture of positive and neutral results about the effects of blended instruction on student learning with respect to specific course learning outcomes suggests that more research about the question is warranted.
Some studies focus more narrowly on one or two specific learning outcomes, rather than more broadly on all of the learning outcomes for a course. Examples of specific learning outcomes that have been studied include critical thinking and clinical reasoning (Snodgrass, 2011), information literacy (Anderson & May, 2010), metacognition and collaborative learning (Van Vliet, Winnips, & Brouwer, 2015), and self-directed and self-regulated learning strategies (Uz & Uzun, 2018). Generally, such studies suggest that blended instruction is better able than traditional instruction to promote the learning outcomes in question.
In sum, the existing literature suggests that blended instruction can be equally effective or more effective than traditional face-to-face instruction in promoting student learning with respect to a variety of learning outcomes and across a variety of academic disciplines (Lamport & Hill, 2012). More research is required to determine whether blended instruction is merely equally effective or actually more effective than traditional face-to-face instruction at promoting student learning. Since far more of the existing research has been done in STEM fields than in the social sciences or the humanities, more research also needs to be done to determine whether blended instruction is effective at promoting student learning in the humanities. Therefore, the second goal of the present study is to determine whether the change from traditional delivery to blended delivery of an upper-level philosophy course in introductory symbolic logic for undergraduate philosophy majors can improve student learning with respect to each of the course learning objectives. With respect to this goal, the authors propose the following hypotheses: H2. The blended delivery of an upper-level philosophy course in introductory symbolic logic for undergraduate philosophy majors has a significant and positive effect on student learning with respect to translating arguments from English into the languages of symbolic logic.
Journal of the Scholarship of Teaching and Learning, Vol. 21, No. 2, June 2021. josotl.indiana.edu H3. The blended delivery of an upper-level philosophy course in introductory symbolic logic for undergraduate philosophy majors has a significant and positive effect on student learning with respect to testing the validity of arguments expressed in the languages of symbolic logic.
H4. The blended delivery of an introductory course in symbolic logic for philosophy majors has a significant and positive effect on student learning with respect to proving the validity of arguments expressed in the languages of symbolic logic.
H5. The blended delivery of an introductory course in symbolic logic for philosophy majors has a significant and positive effect on student learning with respect to explaining key properties of symbolic logic.
Since the testing of H2, H3, H4, and H5 would require the exploration of partial correlations between the delivery mode of a course, on the one hand, and student learning with respect to 4 specific learning objectives of the course (i.e., translation, validity-testing, proof construction, and explaining key properties of symbolic logic), on the other hand, the authors decided to analyze the partial correlation data to determine whether student achievement on any of the 4 course learning objectives is partially correlated to student achievement on any of the other learning objectives.
The authors saw no reason to expect student achievement on any one course learning objective to affect student achievement with respect to any other objective, since the skills related to each learning objective are quite disparate. Course learning objective 1 relates to students' ability to create translations of English-language arguments into the languages of symbolic logic, whereas none of the other 3 learning objectives presuppose this ability. Both course learning objective 2, concerning the creation of validity tests of arguments expressed in the languages of symbolic logic, and course learning objective 3, concerning the creation of proofs of the validity of arguments expressed in the languages of symbolic logic, relate to determining whether arguments expressed in the languages of symbolic logic are valid, but the methods used to make the determination are quite different and are applied in different situations. Course learning objective 4, concerning the explanation of properties of symbolic logic, is different from all of these insofar as it relates to the understanding of concepts as opposed to the creation of a translation, a validity test, or a proof of an argument. Therefore, with respect to the question whether student achievement on any one course learning objective affects student achievement on any of the other objectives, the authors propose the following hypothesis: H6. Student achievement with respect to each of the 4 course learning objectives (i.e., translation, validity-testing, proof construction, and explaining key properties of symbolic logic) has no significant effect on student achievement with respect to any of the other course learning objectives.

Methods
The primary investigator of the present study taught five sections of an introductory symbolic logic course in philosophy in five different semesters. The enrollment of the sections ranged from 14 to 25. In most cases, more students enrolled in the course than completed through to the final examination. The first three sections of the course were offered through traditional face-to-face instruction, with both lectures about the course material (about 60 minutes per week) and problem-solving activities applying the course material to exercises selected by the instructor (about 90 minutes per week) taking place in a classroom face-to-face. The next two sections of the course were offered through blended instruction, with lectures recorded by the instructor about the course material being offered online (about 30 minutes per week) and problem-solving activities taking place in a classroom face-to-face (about 150 minutes per week) (see Table 1). In preparation for face-to-face class meetings, students were assigned to complete a reading from the course text, view an online lecture about the material covered in the assigned reading, attempt exercises from the course text related to the assigned reading, and contribute to an online discussion by identifying particular exercises that they found challenging and explaining what they found difficult about those exercises. Each online lecture was a rehearsed, polished, and slightly compressed version of a lecture that the instructor would have delivered about the same material in a face-to-face class meeting. Students were required to make six contributions to the online discussion for each of four units, for a total of twenty-four contributions over the course of a fourteen-week semester.
Assigning students enrolled in the blended-instruction sections to view the course lectures online outside of regularly scheduled class meetings permitted the instructor to devote significantly more time in traditional face-to-face class meetings to problem-solving activities. Assigning students to discuss exercises that they found difficult in online discussions required students to articulate the difficulties they encountered in the exercises. This had the metacognitive benefit of helping students to achieve an understanding of the challenges that the exercises posed for them. Viewing student posts to the online discussion prior to face-to-face class meetings permitted the instructor to focus problemsolving activities in face-to-face class meetings on exercises that students found challenging. In these ways, the blended instructional methods applied in the course under study encouraged students to engage in metacognition about the struggles they encountered in their learning activities and permitted the instructor to address gaps in student understanding in a more focused way than traditional instructional methods made possible. Thus, this was an example of course-level, transforming blended instruction (Graham, 2006). The first three traditional face-to-face offerings of the course and the next two blended offerings of the course were otherwise similar. The sections were offered with the same course description, the same four course learning objectives, and the same means of summative evaluation of student performance through four examinations (worth 75% of the course grade). At least 10 quizzes (worth 15% of the course grade) were administered during face-to-face class meetings. Participation in class discussions (worth 10% of the course grade) was also required, both in the form of participation in face-to-face learning activities and in the form of participation in an online discussion forum about the homework exercises.
The research question was explored through action research (Hartog, 2018). The primary investigator of the present study was dissatisfied with student performance in the course in question, as reflected in course grades and student performance on the course learning objectives. In order to try to improve student performance in the course, the primary investigator changed the course delivery Journal of the Scholarship of Teaching and Learning, Vol. 21, No. 2, June 2021. josotl.indiana.edu method from traditional instruction to blended instruction. The present study is an attempt to determine through action research whether this intervention in the course did, in fact, improve student learning in the course.
Student course grades and exam grades were collected in the traditional-instruction sections and the blended-instruction sections of the course as overall measures of student learning. Students' prior cumulative grade point averages were collected to determine whether improvements in student performance were due to students' stronger academic backgrounds rather than to the change in instructional method. Direct evidence about student learning with respect to specific course learning objectives was collected from student scores on specific final exam questions that were designed to measure student fulfillment of those learning objectives. The final exam was cumulative, incorporating questions covering material from each of the four units of the course.

Results
After the primary investigator collected data about student course grades, student final exam grades, and student performance on the four course learning objectives, the authors compared the data obtained about each of these indicators of student learning in the traditional-instruction sections (n = 41) and in the blended-instruction sections (n = 42). After checking for normal distributions, the authors performed t-tests to determine whether differences in the mean scores for each of these indicators in the traditional-instruction sections and the blended-instruction sections were significant. The results of this analysis are displayed in Table 2.
In these results, negative values of t simply indicate that the mean value for the blendedinstruction sections was lower than the mean value for the traditional-instruction sections, while positive values indicate that the mean value for the blended-instruction sections was higher. The degrees of freedom (df) values are non-integers, which are obtained for a given test using a more complex formula provided by the R Statistical Computing application. As indicated in Table 2, mean student grades for both the course and the comprehensive final exam improved from the traditionalinstruction sections to the blended-instruction sections, although only the improvement in student grades for the course was significant to the level p < .05. Likewise, student performance on learning objective 1 (translation), learning objective 2 (validity-testing), and learning objective 3 (proof construction) improved from the traditional-instruction sections to the blended-instruction sections, although only the improvement on learning objective 1 (translation) was significant to the level p < .05. Student performance on learning objective 4 (explaining properties of symbolic logic) worsened slightly from the traditional-instruction sections to the blended instruction sections, although the change was not significant to the level p < .05.
Since the delivery method variable is binary (blended versus traditional delivery), a pointbiserial correlation test was performed on the data, as a first correlation test. The results of this test provided identical results in terms of significant relationships to the more complex partial correlations test, but without the additional insights that the partial correlations test provided. Therefore, the results of this point-biserial correlation test are not included here to avoid reporting redundancies.
In order to examine more closely the relationship between course delivery method and student performance on specific course learning objectives, the authors analyzed partial correlations between course-delivery method and student performance on a specific course learning objective while controlling for the influence of each of the other learning objectives. In order to minimize the influence of each student's prior cumulative GPA, the Spearman test (based on the ranked values for each variable) was chosen over the Pearson test. The results of this analysis are expressed in Table 3.
In these results, negative values of r indicate a negative correlation (a positive change in course delivery method from traditional instruction to blended instruction indicates a negative change in Journal of the Scholarship of Teaching and Learning, Vol. 21, No. 2, June 2021. josotl.indiana.edu learning objective performance), while positive values of r indicate a positive correlation (a positive change in course delivery method indicates a positive change in learning objective performance). As Table 3 indicates, when the authors control the influence of student performance on course learning objectives 2 (validity-testing), 3 (proof construction), and 4 (explaining properties of symbolic logic) on the relationship between course delivery method (traditional-instruction or blended-instruction) and student performance on course learning objective 1 (translation), the significant partial correlation r = 0.296 (Spearman method) is obtained, with p < .01. The results show no additional significant partial correlation between course delivery method and student performance on any of the other three course learning objectives.
Finally, in order to examine more closely the relationship between student performance on one course learning objective and another, the authors analyzed the partial correlations between student performance on each of the possible pairs of different learning objectives while controlling for the influence of course delivery method and the influence of student performance on other learning objectives. The partial correlations obtained from this analysis are expressed in Table 4, while the p-values for these partial correlations are expressed in Table 5.
When the authors examined the partial correlations between student performance on pairs of course learning objectives while controlling for the influence of course delivery method and the influence of student performance on other learning objectives, the following significant partial correlations were found: r = 0.513 (Spearman method) for the partial correlation between learning objectives 1 and 2; r = 0.232 for the partial correlation between learning objectives 2 and 3; and r = 0.225 for the partial correlation between learning objectives 1 and 3, where p < .05 in all cases. By contrast, none of the partial correlations between learning objective 4 (explanation of properties of symbolic logic) and the other course learning objectives were significant, with p > .05 in each case.  .118 --* p < .05, ** p < .01, *** p < .001

Discussion
The results of the present study show a positive and significant effect of the change in course delivery method on student course grades, with course grades improving from 3.21 to 3.65 on a scale from 0.00 to 4.00, where p < .01. Consequently, the results of the present study confirm the authors' hypothesis H1. In this respect, the present study confirms the conclusions of Alducin-Ochoa and Vásquez-Martínez (2016) and López-Pérez et al. (2011) that appropriately designed course-level, transformational blended instruction can improve student performance in a course as measured in course grades.
The results of the present study show a significant and positive effect of blended instruction on student performance on course learning objective 1, which concerns the translation of arguments from English into the languages of symbolic logic. Student performance improved from 32.0 to 35.0 on a scale from 0.0 to 40.0, where p < .01. Consequently, the results of the present study confirm the authors' hypothesis H2. In this respect, the results of the present study are consonant with the results of other recent studies that suggest that appropriately designed course-level, transformational blended instruction can improve student learning outcomes (Alsancak Sirakaya & Ozdemir, 2018;Kozikoglu, 2019;Sun & Wu, 2016;Vernadakis, Antoniou, Giannousi, Zetou, & Kioumourtzoglou, 2011).
The results of the present study also, however, disconfirm the authors' hypotheses H3 and H4. With respect to hypothesis H3, the results show a positive but non-significant effect of blended instruction on student performance with respect to course learning objective 2, which concerns testing the validity of arguments. Assessment results for course learning objective 2 increased from 34.6 to 36.3 on a scale from 0.0 to 40.0, but p > .05. Similarly, with respect to hypothesis H4, the results show a positive but non-significant effect of blended instruction on student performance with respect to course learning objective 3, which concerns constructing proofs of the validity of arguments. Assessment results for course learning objective 3 improved from 65.4 to 68.8 on a scale from 0.0 to 80.0, but p > .05. As an instructor, the primary investigator welcomes any improvement in student performance with respect to course learning objectives, and the instructor plans to continue to deliver the course through blended instruction because of the positive effects of blended instruction on student course grades and student performance on course learning objectives 1, 2, and 3. As scholars of teaching and learning, however, the authors conclude that only hypotheses H1 and H2 are confirmed and generalizable to other relevantly similar courses, whereas H3 and H4 are not. In this respect, the results of the present study are consonant with other recent studies that suggest that course-level, transformational blended instruction has no significant effect on student learning outcomes (Cabi, 2018;Roscoe, 2012;Utts, Sommer, Acredolo, Maher, & Matthews, 2003).
Likewise, the results of the present study disconfirm the authors' hypothesis H5. The results indicate a small decline in student performance with respect to course learning objective 4, concerning the explanation of properties of symbolic logic, with assessment scores declining from 16.9 to 16.6 on a scale from 0.0 to 20.0, with p > .05. Unlike student performance on course learning objectives 1, 2, and 3, student performance on learning objective 4 is not affected positively by course-level, transformative blended instruction. It is worth noting that course learning objective 4 concerns the cognitive process of understanding and the conceptual level of knowledge about symbolic logic, according to the revised version of Bloom's taxonomy of educational objectives developed by Anderson and Krathwohl (2001). By contrast, all of course learning objectives 1, 2, and 3 concern the cognitive process of creation about the procedural level of knowledge about symbolic logic in the revised version of Bloom's taxonomy (Anderson & Krathwohl, 2001). Perhaps blended instruction is more effective at promoting higher-order learning objectives, such as those related to the cognitive process of creation and the procedural level of knowledge, than it is at promoting lower-order learning objectives, such as those related to the cognitive process of understanding and the conceptual level of knowledge. We will discuss this issue further in the conclusion.
The results of the present study also disconfirm the authors' hypothesis H6. The authors saw no reason to expect that student performance on any one course learning objective would be tied to student performance on any other course learning objective. Nonetheless, an analysis of the partial correlations between student performance on each of the learning objectives and student performance on each of the other learning objectives, controlling for the influence of course delivery method and performance on other learning objectives, indicates strong and significant partial correlations between student performance on course learning objectives 1 and 2 (r = 0.513), student performance on course learning objectives 1 and 3 (r = 0.225), and student performance on course learning objectives 2 and 3 (r = 0.232), where p < .05 in each case. Thus, the present study suggests that student performance on each of three higher-order course learning objectives is tightly tied to student performance on each of two higher-order learning objectives (Anderson & Krathwohl, 2001). The results of the present study also indicate that student performance on course learning objective 4 has only weak, nonsignificant partial correlations with student performance on each of course learning objectives 1, 2, and 3. This suggests that student performance on the lower-level course learning objective is not tightly tied to student performance on any of the other three higher-order course learning objectives (Anderson & Krathwohl, 2001). Perhaps student performance on each of course learning objectives 1, 2, and 3 is tightly tied to student performance on each of the other two learning objectives because each of these three learning objectives represents higher-order thinking in the revised version of Bloom's taxonomy. Similarly, it may be that student performance on course learning objective 4 is not tightly tied to student performance on any of course learning objectives 1, 2, and 3 because course learning objective 4 represents lower-order thinking in the revised version of Bloom's taxonomy. We will discuss this issue further in the conclusion.

Conclusion
The point of action research in the context of higher education is for instructors of courses to change the way they teach the courses and to collect and analyze evidence to determine whether the changes promoted student learning in the courses. The authors conclude that the change from traditional instruction to blended instruction of an introductory symbolic logic course for philosophy majors did have a positive and significant effect on student learning as measured in course grades and student assessment scores for one learning objective, as well as a positive but non-significant effect on student assessment scores for two additional learning objectives. The present study thus confirms the value of blended instruction in this course.
Since the traditional-instruction sections and the blended-instruction sections of the course were not taught simultaneously to audiences of students assigned at random to one section of the course, however, the results of the study cannot be generalized broadly. As a result, the authors caution against assuming that instructors of courses that differ in significant, relevant ways from an upperlevel introductory course in symbolic logic aimed primarily at philosophy majors and typically enrolling 10 to 25 undergraduate students will obtain similar results from changing from traditional instruction to blended instruction in their courses.
The present study also presents some evidence to suggest that course-level, transformative blended instruction promotes student learning with respect to some kinds of learning objectives more effectively than others. The authors note that the present study indicated that blended instruction had positive effects on learning objectives related higher-order thinking in the revised version of Bloom's taxonomy (i.e., course learning objectives 1, 2, and 3) and significant and positive effects on one of these learning objectives (i.e., course learning objective 1). By contrast, the present study indicated that blended instruction had a slightly negative, non-significant effect on student learning with respect to a learning objective related to lower-order thinking in the revised version of Bloom's taxonomy (i.e., course learning objective 4). Student performance on the learning objectives that are related to higherorder thinking in the revised version of Bloom's taxonomy (i.e., course learning objectives 1, 2, and 3) were also tightly tied to one another, but student performance on the learning objective that is related to lower-order thinking in the revised version of Bloom's taxonomy (i.e., course learning objective 4) is not tightly tied to student performance on any of the other learning objectives (i.e., course learning objectives 1, 2, or 3). (Anderson & Krathwohl, 2001) This possible relationship between blended instruction and learning objectives focused on higher-order and lower-order thinking merits further study.
Finally, it is worth noting that the present study is one of only a few that explore the potential to use blended instruction to promote student learning in the humanities and social sciences as opposed to STEM fields (Murphree, 2015;Roscoe, 2012). Existing studies have barely begun to explore the potential of blended instruction in humanities and social science courses. To return to the definition of "blended learning" offered by McGee and Reis at the beginning of this article, scholars of teaching and learning need to explore the untapped potential of blended instruction to offer appropriate means of promoting student learning outcomes in the humanities and social sciences.