SoTL Evidence on Promotion and Tenure Vitas at a Research University

The development and adoption of promotion and tenure (P&T) policies supporting scholarship of teaching and learning (SoTL) activities began in earnest at our large, Midwestern, land-grant university, over fifteen years ago. The purpose of this research was to present the results of a five year project collecting and analyzing the evidence of SoTL on P&T vitas. Every occurrence of SoTL in peerreviewed publications, presentations, and funded external and internal grants was counted. Of the 343 successful cases, there were 201 to associate professor and 142 to professor. Of these cases, 48% (n=163) had at least one SoTL artifact included on their vitas. These results suggest SoTL is a meaningful form of scholarship performed by faculty, however, there is additional cultural change necessary for greater integration of SoTL in faculty work.


Introduction
The Scholarship of Teaching and Learning (SoTL) lies at the intersection of teaching and research in which faculty "frame and systematically investigate questions related to student learning…not only to improving their own classroom but to advancing practice beyond it" (Hutchings, & Shulman, 1999, p. 12). Shulman (2000) distinguished between scholarly teaching which is "is well grounded in the sources and resources appropriate to the field" whereas SoTL involves making "our work as teachers…public, peer-reviewed and critiqued, and exchanged with other members of our professional communities so they, in turn, can build on our work" (p. 50). Researchers have proposed that SoTL develops individual faculty members' teaching and research programs, as well as informing and improving the teaching and research functions within universities (Berstein, 2013;Fanghanel, 2013;Martensson, Roxå, and Olsson, 2011). Research also suggests, however, at many institutions, faculty engaged with SoTL are undervalued as measured by prestige, pay, and promotion and tenure decisions (Forrest, 2013;Kern, Mettetal, Dixson, & Morgan, 2015). Additionally, particularly at research extensive institutions, there may exist a lack of appreciation and thus engagement for faculty to engage in SoTL (Brown, Ralston, Baumgartner, & Schreck, 2015).
As stated by Kern et al., (2015) academic institutions have struggled in determining how to embrace professional development in teaching and SoTL into the institution's reward structure. While researchers have made gains in determining how faculty move from scholarly teaching to SoTL, the challenges of SoTL engagement on each campus "are substantial" (Gayle, Randall, Langley, Preiss, R., 2013, p. 91). McKinney (2004) echoes this observation stating "…SoTL work receives little support, reward, or recognition" (p. 7). At the researchers' university, policies supporting and encouraging SoTL have been integrated into the institution. The purpose of this research, therefore, was to examine the degree of participation in SoTL publications, presentations, and grants as listed on curriculum vitas presented for promotion and/or tenure decisions, to better understand the degree of university wide SoTL involvement.

Brief Review of Literature
While it has been suggested that institutional mandates of faculty engagement with SoTL do not work (Ciccone, Huber, Hutchings, & Cambridge, 2009;Hutchings, Borin, Keesing-Styles, Martin, Michael, Scharff, Simkins, & Ismail, 2013), support from upper administration does provide opportunities for SoTL to thrive on university campuses (Kreber, 2002). The outcomes and impact of SoTL have become institutionalized on campuses through advancement of teaching practices, curriculum, faculty development, and assessment initiatives (Hutchings, Huber, & Ciccone, 2011). For SoTL institutionalization to occur, a culture shift needs to happen, "during which time actions by campus leaders, change agents and facilitators lay the groundwork for, and effect, institutional change" (Ginsberg & Bernstein, 2011, p. 1). However, the existing evidence regarding the institutionalization of SoTL on campuses are anecdotal. For instance, Ginsberg & Bernstein (2011), note in chronological fashion, the ways in which SoTL was implemented at Eastern Michigan University through Kezar's work on the Principles of Change (2001). They state that while progress is visible, "At the time of this writing, the results remain to be seen, however, we are optimistic that the increased functions of external constituents will serve to support the acceptance of SOTL on our campus and on theirs" (2013, p. 11).
While SoTL has become increasingly more noticeable with journals, conferences, and books on the subject (Witman & Richlin, 2007), Hutchings, et al., notes that "the number of participants in most settings is still small…" (2013, p. 43). Past researchers have suggested that while faculty members generally care about their teaching and their students, few take the time to systematically investigate, reflect upon, and publish their findings (Calder, Cutler, & Kelly, 2002). And while some institutions and faculty members have embraced SoTL, not all consistently embrace the broader vision of scholarship (Cruz, 2014;Shapiro, 2006). Even if accepted, engagement with SoTL can be limited by the perceived challenges of navigating human subjects' research protections as implemented by institutional review boards (Linder, Elek, & Calderon, 2014). Debates regarding the most effective methods to conduct SoTL can also hinder faculty engagement in this type of work (Gurung, 2014). Furthermore, while some institutions evaluate faculty members holistically, others require faculty to categorize their efforts squarely into teaching or research categories, relegating SoTL to teaching, rather than research efforts (Kern, et al., 2015).
Concerns regarding what constitutes SoTL, particularly within the evaluation process for full-time faculty is often nebulous, if not contested (Secret, Leisey, Lanning, Polich, & Schaub, 2011). In their research of members of psychology departments' perceptions of SoTL, there was confusion regarding what products constituted SoTL (Gurung, Ansburg, Alexander, Lawrence, & Johnson, 2008). Further research investigated faculty perceptions of the status of SoTL at one university (Secret, Leisey, Lanning, Polich, & Schaub, 2011). Their work suggested that the majority of faculty agreed that scholarly peer-reviewed publications constituted SoTL, however, there was a disconnect between tenured and tenure-track faculty, where the non-tenured faculty were more likely to judge teaching activities that were less likely to undergo the peer-review process as SoTL. The authors state revising evaluation guidelines and making distinctions between research and teaching could give faculty clearer direction, "serve as an impetus for revising promotion and tenure policies and procedures," such that all "faculty have consistent and equal opportunity for advancement" (Secret, et. al, 2011, p. 15).
In their work analyzing the frequency, types, supports, and barriers for the application of SoTL results, McKinney and Jarvis (2009), reported on the level and forms of concrete applications of SoTL to improve teaching and learning. Their results implied that resources, including funding, collaborative opportunities, recognition within the institutional reward structure, and SoTL circles or communities were necessary for the application of this form of work. Other researchers argue that for sustained engagement, SoTL must become an integral aspect of the institution, rather than reliance on individuals operating alone (Williams, Verwood, Beery, Dalton, McKinnon, Strickland, Pace, & Poole, 2013).
To create an institutional culture favorable to SoTL, researchers have suggested providing tangible incentives and support for SoTL (Haigh, Gossman, and Jiao, 2011). Finding common intersections between research-based frameworks about student learning, such as the National Survey of Student Engagement, has also been advised as a valuable connection between SoTL and institutional initiatives (Schroeder, 2007). Schroeder argues that "we need a better strategy than that of falling back on wishful thinking that more and more scholars doing SoTL, or more and more publications and dissemination will somehow add up to a tipping point of institutional impact or reform" (p. 3). Networks of SoTL champions who create communities within and outside of the university setting may also help to provide support for SoTL, however, there remains the need for institutional acknowledgement and reward structures for widespread SoTL adoption (Marcketti, VanDerZanDen, & Leptien, 2015). Within their article focused on SoTL and accountability, Hutchinds, et al., quote a director for educational research as stating, "There is no support for SoTL work unless it is clearly linked to accountability efforts for the institution" (2013, p. 41).
The places in which we work influence and mold our perspectives. In his work on contextualizing SoTL, Blair quotes John Dewey's Experience and Education, who stated there is a "general principle of the shaping of actual experience by environing principles" (2013, p. 128). Blair comments that "understanding one's own perspective and paradigm through understanding one's own situation could help secure the development of a practice that is situationally valid and more suited to supporting those who learn within such a place. And sharing such understanding through published work could help secure the development of such practice within and across the academy" (2013, p. 129).
Organizational change is a process which Curry (1992) described as involving three stages: (1) mobilization, whereby the system is prepared for change; (2) implementation, whereby change is introduced into the system; and (3) institutionalization, whereby the system is stabilized in its changed state. Building on this work, Braxton et al. (2002) provided eleven recommendations for institutional policy and practice designed to institutionalize the scholarship of teaching and learning (among Boyer's other domains of scholarship). These recommendations included: assessments of faculty scholarship for tenure and promotion decisions that used Schulman and Hutching's (1998) criteria of scholarship. The following research examined the degree of SoTL engagement faculty at a research extensive university had as listed on their curriculum vitas for promotion and/or tenure. Marcketti  Similar to other universities, promotion and tenure (P&T) policies drive discussions of faculty time, productivity, and effort. The development and adoption of P&T policies supporting SoTL activities began in earnest over fifteen years ago as a partnership between the faculty senate, teaching center, and the office of the provost. The current statement embracing SoTL in the ISU Faculty Handbook encourages transparency, flexibility, and acceptance regarding diverse forms of faculty scholarship, encompassing Boyer's scholarship of discovery, application, and teaching (1990). The handbook states recognition of the contributions SoTL makes to the discovery of knowledge about teaching and learning in higher education. All college, departmental, and program governance documents must comply with this university document. The ISU Faculty Handbook compares and contrasts scholarly teaching with SoTL and states, in part (2016, p. 62): At Iowa State University, SoTL contributes to the discovery of knowledge about teaching and learning in higher education and must be held to the same standards of rigor, relevance, peer review, and dissemination as other forms of disciplinary research and creative activity. While SoTL may be an important part of the promotion and tenure process, it should not displace high quality scholarly teaching in annual performance reviews and in promotion and tenure decisions. Although all faculty should engage in scholarly teaching, not all faculty need to engage in SoTL. Scholarly teaching is part of a faculty member's teaching responsibilities; if a faculty member chooses to pursue SoTL, this work is part of their scholarship/creative activity/research responsibilities.
Past research exploring individual experiences within the framework of Iowa State University found institutionalizing SoTL through definition and valuing in the faculty handbook helped enable and encouraged faculty members' SoTL work (Marcketti, VanDerZanden, & Leptien, 2015). Additional institutional means of supporting SoTL include selected departments inclusion of SoTL as part of position descriptions, as well as a component of the faculty members' Position Responsibility Statement (PRS). Because the PRS is the basis for all annual, probationary three-year, promotion and tenure reviews, and post-tenure reviews, the documentation and importance of SoTL is further solidified. An additional means of easing faculty members' participation with SoTL was the streamlining of the institutional review board (IRB) process in which most forms of educational related research are considered exempt. The form for approval of exempt research is a short document that recommends, but does not require informed consent documentation or detailed responses as compared to the long form necessary for review board approval. [Note: if it is determined that the proposed research is not exempt, the faculty would then have to go back and complete a full IRB request.] In recognition of teaching and SoTL excellence, in 2013, the university established the Morrill Professor title which recognizes faculty members whose professional work has demonstrated outstanding success in teaching and learning in undergraduate, graduate and/or Extension/outreach programs. The Morrill Professorship is similar in its nomination process and prestige to the University and Distinguished Professor titles and awards. Even with the language of inclusion in the Faculty Handbook, position descriptions, and personal responsibility statements, the elimination of barriers through IRB exemption procedures, and recognition of teaching and SoTL through establishment of the Morrill Professorship, faculty members may choose not to explore SoTL if they do not feel their immediate supervisors and department colleagues understand or value this line of inquiry (Twait, 2014).

Methods
Working with the university's provost's office, the authors examined each curriculum vita of successful cases to associate professor or professor during the years 2010 to 2015. Using an excel document that listed the faculty members' name and department, the authors entered information regarding the number of total peer-reviewed publications, academic presentations, successful external and internal grantsmanship, and all other publications, which included extension pamphlets, technical documents, and such. Every occurrence of SoTL within these same categories was ascertained and documented. The authors compared numerous data points across the years to ensure they were counting and considering similar examples of SoTL. Upon completion of the counting, the authors calculated the percentage of total SoTL contributions according to faculty rank and college.

Results
Over the five year period, there were 343 successful cases, 201 to associate professor and 142 to professor. Of these 343 cases, 48% (n=163) had at least one SoTL artifact listed on their vitas. A total of 44% (n=89 out of 201) of associate professors engaged in SoTL, with professors engaging at a higher rate of 51% (n=72 out of 142). Associate professors produced a total of 2,422 SoTL outputs, accounting for 6% of their combined 39,137 scholarly artifacts. Professors produced a total of 965 SoTL outputs, accounting for 4% of the combined 23,598 scholarly artifacts. Table 1 includes detailed information regarding each of these scholarship outcomes. When associate professors and professor data was combined, there was a total of 3,417 SoTL artifacts accounting for 5.4% of the total number of scholarly outputs. When examined within each category, SoTL contributions were most numerous in the grants category (Figure 1).

Figure 1. SoTL Percentages within Curriculum Vitae Categories
Among the associate professors, engagement with SoTL came from each of the colleges of the university: Library 100% (n=1), Human Sciences 90% (n=18), Design 54% (n=5), Engineering 44% (n=13), Veterinary Medicine 43% (n=9), Agriculture and Life Sciences 40% (n=14), Business 38% (n=5), and Liberal Arts and Sciences 32% (n=22). Table 2 includes the range of SoTL artifacts within each category for those associate professors that included at least one example of SoTL on their vitas. Of the associate professors with evidence of SoTL on their vitas, two identified SoTL as their primary emphasis, with six publishing SoTL at least half as much as their disciplinary focus. The majority of the publications and presentations were in disciplinaryspecific journals. In every college, except human sciences and veterinary medicine, faculty at the rank of professors contributed to the SoTL in greater percentages than the associate professors. Every college had some evidence of SoTL participation including the Library 100% (n=1), Human Sciences 84% (n=10), Design 60% (n=6), Engineering 54% (n=15), Veterinary Medicine 20% (n=2), Agriculture and Life Sciences 48% (n=11), Business 50% (n=3), and Liberal Arts and Sciences 50% (n=26). Table 3 includes the range of SoTL artifacts within each category for those professors that included at least one example of SoTL on their vitas. While a greater number of faculty contributed SoTL at the professor level, there were fewer faculty with a majority of their publications in SoTL (n=2).  When examined annually, the percentage of SoTL participation generally increased among associate professors (Table 4). From 2010-11 to 2014-15, there was a 25% increase in SoTL engagement among associate professors. For four out of the five years, professor engagement with SoTL was at the 50% mark or higher, with a noticeable decrease in 2014-15. Additional data will need to be collected to decipher if the noticeable decrease in professor engagement with SoTL is an anomaly or a hint at a larger trend.

Discussion and Conclusion
The promotion and tenure data provide evidence of SoTL engagement among faculty at Iowa State University. When viewed holistically, slightly greater percentage of senior faculty engaged with SoTL, which may not be surprising, since criteria for tenure requires faculty members to establish a national reputation in their disciplinary field. As faculty members earn tenure, they may feel greater freedom to engage in SoTL. While it was heartening that faculty conducting SoTL represented every college of the university, some colleges had greater participation than others. This may suggest that some departments and colleges value and are more supportive of SoTL than others. One area of potential surprise was the greater number of SoTL grants received as compared to presentations and publications. This suggests faculty may not have outlets within their disciplinary conferences and journals for SoTL opportunities, that they may not be aware of SoTL publication and presentation outlets, or may be dismayed by the sheer number of SoTL publications that fall outside of their disciplines. Perhaps, also, faculty may be interested in teaching related grants for curriculum improvement, but do not consider the additional effort required to produce SoTL presentations and publications worth their time. This research provides evidence of SoTL as a documented form of scholarship in each college of Iowa State University. Institutional policies and procedures that recognize and value SoTL include inclusion in the Faculty Handbook, position descriptions, and professional responsibility statements, elimination of barriers through IRB exemption procedures, and establishment of the Morrill Professorship recognizing teaching excellence. While previous scholarship suggests that SoTL "champions" believed this institutionalization supported their SoTL work (Marcketti, VanDerZanden, & Leptien, 2015), this research noted participation, rather than perception of SoTL by the faculty members, departmental promotion and tenure committees, department chairs, and external reviewers. Thus, further research could investigate through interviews or a survey method the perceptions of faculty members in departments in which SoTL has thrived and those in which SoTL is not evidenced on vitas. In this way, the authors may gain additional knowledge of the practices that enable or hinder engagement with SoTL at a research extensive university. Finally, the results, particularly those noting increased participation with SoTL grants than publications and presentations suggest that the university's teaching center staff need to focus programming efforts to help faculty turn the results of grants into peer-reviewed outputs such that their work becomes scholarship that is shared as part of the body of knowledge of higher education. The results of the research have been shared with faculty within Iowa State University at Provost-led discussions of promotion and tenure, at the Center for Excellence in Learning and Teaching annual teaching symposium for new faculty and graduate students, and within SoTL programming. Extending on the statements of Blair (2013), the sharing of this SoTL work within the university setting advertises that this work is valued, thus institutionalizing this work even more. It is imperative that this counting work continues, particularly demonstrating evidence stated in the Faculty Handbook that "if a faculty member chooses to pursue SoTL, this work is part of their scholarship/creative activity/research responsibilities."