Learning to see the infinite : Measuring visual literacy skills in a 1 st-year seminar course

Visual literacy was a stated learning objective for the fall 2009 iteration of a first-year seminar course. To help students develop visual literacy skills, they received formal instruction throughout the semester and completed a series of carefully designed learning activities. The effects of these interventions were measured using a one-group pretest-posttest study design where students were asked to look at two different—but stylistically similar—paintings and write a response to the following two questions: what do you see and what do you think it means? Students’ responses were analyzed based on the visual evidence they recorded and the strength of their arguments using Toulmin’s argument model. After instructional interventions, paired t-tests indicate that students made significantly more basic (t(15) = 4.291, p < .001) and advanced observations (t(15) = 2.440, p = .014), offered more supporting visual evidence for their bestsupported claims, and made stronger connections between their claims and the visual evidence.


Background
Visual literacy was first coined in 1969 by John Debes (Debes, 1969).An exact definition remains elusive because of the complexity and multidimensional nature of the skills involved, but at its core visual literacy refers to the ability to interpret, negotiate, and make meaning from information presented in an image (Avgerinou & Ericson, 1997).Drawing on the science of learning, James E. Zull argues in the The Art of Changing the Brain (2002, p. 146) that faculty should make "extensive use of images to help [students] learn," both by teaching with visuals and by requiring them to use visual forms to represent what they know.Little, Felten, and Barry (2010, p. 46) have argued that "visual literacy…is a critical skill for twenty-first-century students and ought to be a central component of liberal education."They go on to suggest that instructors teaching courses in academic disciplines outside those commonly affiliated with visual literacy (for example, art history and media studies) and those teaching first-year general education through capstone courses should explicitly help students develop visual literacy skills.This is not only because images saturate our daily lives but also because many fields, including those in the arts, sciences, social sciences, and health sciences, rely heavily on images as data sources and require a heightened level of visual literacy to understand and interpret the world (Brumberger, 2011).
Though an ever-expanding literature exists on teaching visual literacy skills in higher education (Felten, 2008), only a handful of studies measuring students' general visual literacy skills have been published (see, for example, Arslan & Nalinci, 2014;Brumberger, 2011).Even fewer studies exploring the impact formal instruction has on students' visual literacy skills have been reported (Hollman, 2014;Linenberger, & Holme, 2014;Yeh, 2010).The current study, which was embedded in my highly interdisciplinary, discussion-based, first-year seminar course known as Falling from Infinity,3 helps to fill this void.Specifically, we examined whether after a series of instructional interventions focused on a subset of visual literacy skills my students were better able to 1) make more basic and advanced observations in an image, 2) offer more supporting visual evidence for a claim they made about that image, 3) and make stronger connections between their claim and the visual evidence.
Falling from Infinity drew on a diverse set of perspectives-literary, poetic, artistic, mathematical, scientific, religious, philosophical-and invited my students to spend a semester grappling with uncountable numbers, immeasurable spaces, and unending times.The diverse nature of the topic lends itself well to the use of visuals as an important and essential vehicle for exploring course content.Because researchers (Wineburg, 1999) have found that students often need significant structured guidance to form meaningful metacognitive routines when exploring and making meaning from images (for example, asking themselves "what else do I need to do to better understand the image"), visual literacy was as one of the explicit learning objectives of the course.I implemented a series of learning activities designed to help students practice foundational visual literacy skills (e.g.making observation) early in the semester to support more complex ones later on (e.g.critically analyzing images).Many of the visual literacy learning activities were developed in class through small group, collaborative exercises or large group discussions.Others were assigned as part of reflective journal assignments.For the latter, students completed a short task related to course content outside of class and then wrote a 1-2 page reflection on the experience.For example, students were asked to look at images outside of class and briefly reflect on how this new information complicated their understanding of the infinite.Other activities supported more complex assignments, such as creating a photo essay and writing a 3-4 page reflection evaluating the ways the photo was effective in representing the infinite and in what ways it fell short.
A thorough discussion of all the specific teaching interventions used throughout the course is beyond the scope of this work, but we highlight here two of the more substantial activities.Curious readers can explore the complete week-by-week timeline of the course elsewhere (Palmer, in-press).The first illustrative example occurred near the mid-point of the semester, after much of the visual literacy foundation had been laid.I began class by formally defining certain terms, like observation (a neutral, non-judgmental and verifiable statement) and inference (a meaning, an interpretation, or an assumption based on observation) and then systematically engaging my students in critical analysis of the images we were encountering.To start our in-class discussion, I displayed only a portion of Joel Sternfeld's photograph Warren Avenue at 23rd Street, Detroit, Michigan.I choose this particular photo, one that has nothing to do with infinity, to avoid having students overlay their prejudices and biases about infinity.They were first prompted to make as many observations about the visible part of the photograph as possible, preceding each observation with the statement "I see…" Next, I revealed another portion of the photograph and asked them to add to their list of observations.I continued to reveal the photograph slowly, piece-by-piece until the image was complete.Once the students were satisfied that they made all relevant observations, I prompted them to make inferences, beginning each with, "I infer…because…" In follow-up questions, I prompted them to consider the strength of their inferences, asking "How well supported is your inference based on the observations we just made?"After this exercise, students examined van Gogh's Starry Night, and again individually wrote down all their observations, compared notes with their neighbor, created a combined class list, and then made the best supported inferences possible.
In another example, students looked at Caspar David Friedrich's painting Monk by the Sea before class.In class, I asked them to respond to the question, "What do you see?"I then asked them to individually write down 3-5 adjectives that described their emotional response to the painting.Afterward, they shared these in the large group as I wrote and categorized their adjectives on the board.This particular painting is unique in that it typically evokes polar opposite responses: some viewers feel it evokes "awe" and a feeling of "tranquility" while other feel it evokes "dread" and "hopelessness."During the conversation, I asked the students, "What in the painting leads you to feel…?"After all responses were collected and reported out on the board, we discussed the question, "Which set of responses is correct?"What my students discovered through this activity is that some claims, when based solely on the observational data alone, are more easily supported than others.

Methodology
To help students develop visual literacy skills, they received formal instruction throughout the semester and completed a series of carefully designed learning activities like those described above.The effects of these interventions were measured using an IRB-approved, non-graded pre-/post-semester methodology where students (n=16) were asked to look at two different-but stylistically similar-paintings and write a response to the following two questions: what do you see and what do you think it means?The paintings were Salvador Dali's The Persistence of Memory and The Disintegration of the Persistence of Memory.Our hypothesis was that following the interventions students would: • make more basic and advanced observations in an image; • offer more supporting visual evidence for a claim they made about that image; • make stronger connections between their claims and the visual evidence.Toulmin's argument model (Toulmin, 1969), with particular focus on claim, supporting evidence, and warrant (the inferences or assumptions taken for granted by the writer that connect the claim and the supporting evidence), was used to analyze students' responses.This model was chosen based on its relative simplicity and because it mirrors the pedagogical strategy I used during classroom activities; that is, I routinely asked students to make observations and then support their claims based on those observations.Students were not introduced to or given formal instruction in Toulmin's argument model.
When analyzing students' responses, we defined supporting evidence as the observations of visual information they included and coded these observations as either basic or advanced.A basic observation described an object or feature of the painting without significant qualifiers, e.g."I see a clock."An advanced observation described an object or feature of the painting beyond merely identifying it, such as the position of the object/feature relative to others or its location relative to the painting, e.g.foreground/background; the texture of objects or the texture of the painting itself; the contrast or juxtaposition of objects or features; the source and/or direction of light; a minute, easily overlooked detail of the piece; or, an observation the viewer made about his or her own experience viewing the painting, e.g."I looked at the painting in a counterclockwise manner." Students' responses to the pre-and post-assessments were initially analyzed independently by two raters.Observations were coded as either basic or advanced.The raters' results were then compared to make certain every observation was identified and coded consistently.Observations were counted regardless of whether they were found in the response to the question "what do you see" or "what do you think it means" since students often included additional observations when supporting their claim.
A similar methodology was used to analyze the claim, supporting evidence, and strength of the warrant.After identifying the main claim, raters coded the supporting visual evidence, rated the warrant as strong, moderate, weak, or none, and then again compared results.When differences arose, raters discussed the student's response and came to consensus on the appropriate coding.A strong warrant indicated that links connecting the claim and the evidence were clear, all inferences were appropriate, and the assumptions reasonable.A moderate warrant indicated that most of the links connecting the claim and the evidence were clear, most inferences were appropriate, and most assumptions reasonable.A weak warrant indicated that there were no clear links connecting the claim and the evidence, inappropriate inferences, or faulty assumptions.If no claim was present, the warrant was coded as "none."If a student included more than one claim, only the best-supported claim was analyzed.

Findings
Student's responses to the pre-and post-semester assessments were coded and analyzed for the number of basic and advanced observations they made, the number of pieces of visual evidence supporting their main claim, and the strength of their warrant.

Basic Observations
The number of basic observations made per student in the pre-and post-semester assessments is shown in figure 1.At the start of the semester, before the instructional interventions, the median value for the number of basic observations made was 13 (range = 0-33), with three students making less than 10 basic observations.Near the end of the semester, following the instructional interventions, the median value for the number of basic observations made was 26 (range = 13-68).Two students made approximately the same number of basic observations in the pre-and post-semester assessments, but all other students made more in the post.A paired t-test found strong evidence that the interventions significantly improved the post basic observation score: t(15) = 4.291, p < .001.

Advanced Observations
The number of advanced observations made per student in the pre-and post-semester assessments is shown in figure 2. At the start of the semester, the median value for the number of advanced observations made was 2 (range = 0-7), with 11 of the 16 students making 2 or fewer advanced observations.Near the end of the semester, the median value was 3 (range = 2-14), and nine of the 16 students made more than 3 advanced observations.Two students made approximately the same number of advanced observations in the pre-and post-assessments, but

Evidence Supporting Main Claim
The number of pieces of evidence that students provided for their best-supported claim is shown in figure 3.For the pre-assessment, students provided on median 2.5 pieces of evidence (range = 0-6).Three students had no visual evidence to support their claim; one student made no substantive claim.For the post-assessment, students provided on median 8.0 pieces of visual evidence for their best-supported claim (range = 3-40) and every student provided at least three pieces of supporting evidence.

Strength of Warrant
The strength of the students' warrant in the pre-and post-assessments is shown in figure 4. At the start of the semester, three students provided no warrant, six had a weak one, five a moderate one, and only two had a strong warrant.Near the end of the semester, all students provided a warrant: one weak, nine moderate, and six strong.

Conclusion
The results of this study suggest that the classroom interventions I incorporated into the first-year seminar course, Falling from Infinity, significantly improved students' ability to make necessary and appropriate observations in images and to develop stronger claims supported by their observations.Because I embedded numerous learning activities focused on improving visual literacy skills throughout the course, it is difficult to pinpoint which activities had the greatest impact on my students' learning.A portion of every class period, whether explicitly or implicitly, was devoted to some aspect of visual literacy.Though we believe the two activities described in the background section of this paper were key interventions, most likely, it was the sum of activities that led to the positive gains rather than any one particular intervention.The fact that advanced observations weren't defined or categorized until late in the semester and that students exhibited lower gains in this area supports the idea that the interventions had a cumulative effect.Future studies could probes this hypothesis and also help identify the specific activities that have the greatest impact on improving the visual literacy skills explored herein.Such studies could also shed light on whether formal instruction in Toulmin's argument model might in and of itself lead to similar or possibly improved outcomes.
Palmer, M.S. & Matthews, T. Journal of the Scholarship of Teaching and Learning, Vol. 15, No. 1, February 2015.josotl.indiana.edu3 Palmer, M.S. & Matthews, T. Journal of the Scholarship of Teaching and Learning, Vol. 15, No. 1, February 2015.josotl.indiana.edu5 all other students made more in the post.A paired t-test found strong evidence that the interventions significantly improved the post advanced observation score: t(15) = 2.440, p = .014).

Figure 3 .
Figure 3. Number of total observations per student (n=16) supporting the primary claim of the pre-(light teal) and post-semester (dark teal) assessments.

Figure 4 .
Figure 4. Strength of the warrant for students' primary claim for the pre-(light gold) and post-(dark gold) semester assessments.None indicates that no claim was present.