Transferring skills from classroom to professional writing : Student-faculty peer review as an extension of cognitive apprenticeship

In discipline-specific writing courses, students develop professional skills in reading, writing, and peer review. However, students have limited opportunities to peer review professional writing outside a writing classroom or with faculty, especially if they do not perform research. Therefore, it is unclear how students apply classroom-acquired peer review skills to a professional setting. This study examined the transfer of peer review skills learned in a science writing course to an authentic setting in which undergraduate students peer reviewed for the instructor after completing the course. In this case study, eight students volunteered to give feedback to the instructor on a draft of a literature review intended for journal submission. Student feedback was qualitatively evaluated for types and themes. Additionally, students provided their perspectives on this process through preand post-questionnaires, where they indicated a struggle with confidence and content while reading and reviewing. This study supports student-faculty peer review as an authentic tool for situated learning. The benefits to students include increased confidence in reading, writing, and peer reviewing literature, an opportunity to practice classroom skills, and a chance to collaborate with professionals during the writing process. I conclude with additional suggestions to increase student-faculty collaboration and cognitive apprenticeship through peer review as a tool in any discipline.


Peer review within the classroom
Peer review has long been used in writing courses as a learning tool that benefits both the reviewer and the reviewee.The reciprocal process of giving and receiving feedback results in improved student writing in several disciplines (Y.H. Cho and Cho, 2011;Lundstrom and Baker, 2009;Rouhi and Azizian, 2013), indicating it is a valuable tool in teaching writing.For example, during peer review in the second-language classroom, students improve their own writing skills through reviewing their peers' writing (Lundstrom and Baker, 2009).
Several studies have investigated if students are able to provide quality feedback and if this feedback is comparable to feedback from instructors.Liang and Tsai (2010) found that student peer review can be as critical and valid as that of an expert or instructor, especially if students perform multiple review activities.Not surprisingly, multiple rounds of peer review enhance not only the ability of students to review but also the ability to write (Liang and Tsai, 2010), supporting the need for frequent peer review within the classroom.Patchan et al. (2009;2011) also found that students' feedback in science is comparable to teaching assistants' and instructors' feedback, though the comments often tend to be longer, focus more on writing style, or lack directive explanation.Students often are less able to question the technical content of an article in science, which is a skill that may need additional coaching or practice (Patchan et al., 2011).This may also reflect the writing ability of the reviewer, as this can influence the quality of feedback.Highability writers give better feedback than low-ability writers in some conditions (Patchan, Hawk, Stevens, and Schunn, 2012), suggesting that a solution to improving feedback may simply be additional practice in writing and peer review.
Because there are multiple types of peer review, there are different ways to integrate this skill into a writing project or assignment.Peer review can be informal or graded, online or in person, identified or anonymous.While any of these methods can be successful, students sometimes seem unaware that peer review is not only a learning tool, but also a professional skill in academia.In the classroom, the process of peer review seeks to improve a paper.In professional peer review, the process is required to ensure articles meet journal or academic standards.In both of these processes, there are common difficulties in producing consistent, critical and thoughtful reviews (Trautmann, 2009), something practice and reflection may improve.It isn't always clear what students learn about the professional side of peer review by performing peer review for writing assignments in the classroom (Trautmann, 2009).However, by implicitly asking students to participate in a classroom peer review process that mirrors the professional peer review process of academic journals, students can improve their final written product, understand publication in their disciplines, and importantly, learn a greater responsibility to provide quality feedback (Guilford, 2001).This supports the need for students to understand peer review as a process and professional skill, not only a technique used within the writing classroom to achieve a better grade.

The need for peer review outside the writing classroom
Peer review outside the classroom space can be an opportunity for students to practice this skill, as well as improve their own writing.For example, an online English-learning community provides an opportunity for students to collaboratively revise writing in a non-classroom context (Razak andSaeed, 2014, 2015).Here, the majority of peer review comments reflect language changes while content comments are less common (Razak and Saeed, 2015).This is not surprising, as content comments are difficult to make in the sciences as well (Patchan et al., 2011).However, this deepens the need for more frequent opportunities to practice peer review for content.In primary research articles and grant proposals, professional peer review is a staple of data communication in the sciences.Peer review requires individuals to read deeply into the literature, to critique technical content, such as research design, and to analyze how well the author communicates the resulting data and conclusions.These activities can be particularly difficult for undergraduate students who do not yet have this experience in peer review and primary literature content.
Students report increased confidence in reading and critiquing primary literature the more they are exposed to it (Kozeracki, Carey, Colicelli, and Levis-Fitzgerald, 2006), highlighting the need for this activity early and often.Science faculty recognize the need for students to learn communication skills, but they do not believe they have the time or ability to incorporate these skills into their classrooms (Coil, Wenderoth, Cunningham, and Dirks, 2010).Yet, there are large gains to be made from including these skills beyond writing courses, as reading and writing opportunities also enhance students' understandings and critical readings of scientific primary literature (Coil et al., 2010).One solution may be to integrate these skills into a variation of contexts within a student's education.Huiling (2008) described a grant writing activity with graduate students, where they learned about the grant writing process, wrote their own grant proposals, and then served as a mockcommittee to review grant proposals.The students followed the rules of the grant application as reviewers, practicing authentic peer review within an assignment.The students learned about the writing process as part of the bigger culture of grant writing through integrated interactions with faculty.This kind of apprenticeship experience could be applied beyond a classroom into a laboratory environment, a broader program goal, or a department research experience (Huiling, 2008).In this way, writing and peer review skills could be practiced and transferred in several different environments with faculty to create a more collaborative learning experience for students that does not necessarily need to be situated within a writing classroom.

Peer review as a mode of cognitive apprenticeship and situated learning
Cognitive apprenticeship is a method of teaching that takes students through a series of steps meant to immerse them in a genuine process (Collins, Brown, and Newman, 1989).Through modelling, an instructor demonstrates a new skill important to the profession.The instructor then uses coaching to guide students through this skill in a series of scaffolding activities and feedback.Students must then articulate this skill and reflect on it before exploring a new aspect as a demonstration of their learning.In the classroom, reading and writing as a process can be taught as a means of cognitive apprenticeship.In a typical cognitive apprenticeship designed to increase skills in reading and writing, a student will work with an experienced faculty member who will model and guide the student through the thought process behind these skills.Here, students in any discipline can be immersed in the reading, writing, and thinking that happens when researchers communicate ideas and results (Kolikant, Gatchell, Hirsch, and Linsenmeier, 2006).
Peer review itself can also be used as a means of cognitive apprenticeship, where students coach their peers through peer review and then apply peer review as a scaffolding step (Yang, 2011).Articulation, reflection, and exploration in peer review involve justifying the reasons for changes in subsequent revisions.This process not only improves the writing of the author, but may also encourage the author to participate in peer review during future writing assignments (Yang, 2011).Collaborative tasks, in which different combinations of students and/or instructors work together, can enhance overall cognition.When students and instructors progress through an apprenticeship series of modelling, providing feedback and reflection, students increase cognition, engagement with the material, and motivation (Schraw, Crippen, and Hartley, 2006).
While writing and peer review can be modes for cognitive apprenticeship, the most common form of student cognitive apprenticeship may be through undergraduate or graduate research.In science, as well as other disciplines, this apprenticeship can also involve communicating research and reading about research with mentors (Feldman, Divoll, and Rogan-Klyve, 2013).Faculty can model and coach student mentees through the principles of basic research.Students then apply these skills through further stages of exploration, as they progress towards research independence.In a study that investigated how students become scientists through undergraduate research experiences, this apprenticeship model created greater confidence in students as they were less fearful of making mistakes and more willing to try new experiments (Hunter, Laursen, and Seymour, 2007).
Performing research is an undeniably essential tool in training students within a discipline.However, there are other facets of the academic profession that complement research.For example, the dissemination of research through written and oral communication is an essential extension of this process.Furthermore, professionals regularly engage in informal and formal peer review with each other's research.To further engage students in an authentic experience that encompasses the true meaning of an apprentice in research, it is important to understand how students experience and learn from these tasks, such as presentations, authorship on manuscripts, and peer review.The authenticity of a complete experience may be necessary to fully train and immerse students in all that research entails.
This true immersion is also known as situated learning, where a student acts as a cognitive apprentice to learn about a profession in an authentic context.Brown et al. (1989) stated, "Unfortunately, students are too often asked to use the tools of a discipline without being able to adopt its culture.To learn to use tools as practitioners use them, a student, like an apprentice, must enter the community and its culture" (p.33).It is interesting that while we teach components of research to our students in the classroom and through research experiences, we rarely involve students in our personal research as we continue through the process of investigation and writing.Doing so, however, would be the ultimate example of modeling.When faculty are asked to peer review their students' papers in a non-classroom setting, the students' written work improves and faculty members are more directly engaged with their students (Reynolds and Thompson, 2011).The benefits of the reciprocal situation, where undergraduate students perform peer review with or for faculty, have not yet been examined in detail.

Overview and aims of the current study
While cognitive apprenticeship has been investigated in terms of the canonical research experience between students and faculty, this study specifically investigates the role of peer review as a potential tool for situated learning in research.Students in this case study were previously enrolled in my technical science writing course.In this course, the objectives were to develop students' skills in reading, writing, and peer reviewing scientific literature published in academic journals.After the course had finished, eight students volunteered for an authentic experience in peer reviewing a paper that I had written.This research aimed to provide insight into the challenges students faced while performing peer review in a professional rather than classroom environment.Because research depends on the ability to read and review literature, often in collaboration with other faculty, this case study also aimed to investigate the relationship between students and faculty in terms of peer review.I analyzed the feedback from the student peer review and qualitatively assessed student confidence and challenges using a pre-and post-questionnaire.To the best of my knowledge, this is the first study to examine peer review as a mode of professional development and to investigate the transfer of peer review skills to a non-classroom experience.

Course context and content
Course description.Students who completed a science writing course were eligible to participate in this study.This technical writing course is required for physics, environmental science, biology, chemistry, biochemistry, and forensic science undergraduate majors.Students learn how to find, read, analyze, and paraphrase information from primary research articles and reviews.Students use these skills in a semester-long assignment to write an original review on a research topic of their choice.This review must be geared towards a science audience of their peers.The final submission of the review consists of the following parts: a letter to the editor, abstract, key words, introduction, main text that synthesizes and analyzes current research, conclusion, figure and/or tables, and references in style of the American Chemical Society.Students receive a Guideline to Authors at the beginning of the semester that details the formatting and submission requirements, similar to an academic journal.
Peer review within the course.The writing process included practice in both formal and informal peer review of each other's work at various stages of drafting and revision.Because this class was comprised of several different science majors, students would often peer review an article on an unfamiliar research topic.This peer review was meant to simulate feedback that faculty would obtain from their peers before submitting an article to a journal.The publication process was integrated within the course, as students needed to submit the article to a mock academic journal with a letter to the editor that responded to the revisions requested by their peer reviewers.In the class, students were encouraged to give specific feedback and suggestions for improvement, as well as ask scientific questions.They were asked to comment on both style and content.Some of this peer review was anonymous, mirroring a blind peer review process.

Participants for peer review feedback
In total, sixty-nine students were invited to peer review an article for me, the instructor, after the completion of one semester.Eight students initially volunteered and consented to the study, completing both the pre-questionnaire and peer review steps.Seven of these eight students also completed the post-questionnaire.These students included the following majors: Environmental Science (1), Physics (1), Forensic Science (3), Biochemistry (1), and Biology (2).Three students were entering their senior year while five were entering their junior year.Five had not performed scientific research outside a laboratory class.This study was performed with approval from the University Institutional Review Board (IRB).This IRB approval did not permit the release of individual information on these students or their course performance, as this would not guarantee anonymity given the small number of participants.Therefore, the analysis in this study does not include details on the individual students relative to their feedback.

Experimental design of peer review assessment.
To investigate students' experience and confidence in peer review, consenting students were given a pre-questionnaire, peer editing task, and post-questionnaire.All correspondence was performed via e-mail.This research was performed in conjunction with an authentic writing project.Pre-questionnaire.The pre-questionnaire surveyed students regarding their experience and confidence in peer reviewing.In an open-ended format, students commented on their previous experiences in science writing and editing in settings other than the science writing course (e.g. research experiences and/or other coursework).It asked students to comment on the challenges they perceived with peer editing and reviewing and their personal beliefs as to what made a successful journal article.Lastly, the pre-questionnaire explored students' knowledge of how to peer review.
Peer review task.In the peer review task, students were asked to provide feedback on the first draft of a scientific literature review article written by myself, who was their instructor in the science writing course.This review article was approximately 3,000 words, used 10 references, and included an abstract and several figures.I provided students with the Guidelines to Authors for the intended journal, as well as an example article from the same journal.Students were also reminded that their feedback was valued and would be considered as changes were made to the paper before submission.Unlike peer review assignments during the course, this task was unstructured.Students did not receive any further instruction or a rubric.Instead, they were told to peer review in the manner they felt would best help me to improve the paper before submission.The topic of the review was histone ubiquitylation and transcription regulation.None of the students had prior experience with research in this area, and most were not in a major where they would learn this information at this stage in their education.However, this literature review was written for a graduate level audience rather than a niche-specific audience, putting this content at a slightly higher level than the current knowledge of most of these students.The idea for performing this authentic peer review experience stemmed from the fact that students had successfully peer reviewed literature reviews outside their subject area as part of the course design.For example, a biology major may have peer reviewed a physics or biochemistry paper, or a biology paper on a different research topic.Therefore, the students had practice peer reviewing unfamiliar content.The major difference here was that students did not receive a form to fill out for peer review.This form would have told them what type of content to look for.Instead, they needed to rely on the guidelines to authors and examples I provided, as well as their classroom experiences and current knowledge.
Post-questionnaire.The post-questionnaire asked students to reflect on the peer reviewing task.In this open-ended questionnaire, students were asked what steps they took to edit the article and what challenges they encountered.Students also commented on how their experiences in the science writing course influenced them during the peer review task.Lastly, students reflected on how this non-classroom experience could influence their academic careers.
Evaluation.Due to the small number of participants and nature of the experiments, the data was qualitatively analyzed as a traditional case study (Cresswell, 1998).The pre-questionnaire, peer editing task, and post-questionnaire were evaluated for reoccurring themes such as student perspectives and confidence.

Results and Discussion
Experience and confidence in science writing and peer review Prior Experience.Eight students answered a pre-questionnaire at the beginning of this study.Outside of the science writing course or a science laboratory course, students reported no experience in writing or editing in the sciences unless they had conducted some research.Students who had performed research had experience writing or reviewing abstracts, posters, and/or research articles with their faculty advisors or lab members.However, aside from these research experiences, students did not have an opportunity to practice science writing other than writing lab reports.Similarly, these students had no experience peer reviewing science writing in any science class other than the science writing course.They may have occasionally helped a peer informally for a class or lab, but peer editing or writing had never been a formal part of a non-English course.Therefore, most of the experience these students had in writing and reviewing came from their recent science writing course.Additionally, those students who did perform research and had some experience writing did so under the guidance of a faculty advisor or with help from graduate students.None of the students in this study reported any experience reviewing a piece of writing from a faculty member, professional, or graduate student.This experience was therefore novel to all of them.
Confidence and Expectations.While taking the science writing course, the greatest challenges in peer review that students reported were related to technical content and/or confidence in their ability to give quality feedback.Not surprisingly, they also predicted the same challenges would occur when reviewing a paper written by an instructor or advisor.
There were interesting differences in how these participants expected they would peer review scientific articles, depending on the author and the purpose.If they were reviewing a paper for a classmate, the majority of students would focus on writing style, audience, and accuracy.These mirror previous findings, where science students often focus on writing style more than technical content (Patchan et al., 2009).In contrast, if reviewing something more professionalsuch as a piece of writing by a faculty member meant for a peer reviewed journal -students reported that they would place greater value on the content.They stated that they would want to verify that an article conveyed importance and relevance, that the author analyzed data appropriately, and that the author made insightful and original conclusions.These statements suggest that students hold higher standards for professional writing because they see it as authentic, as opposed to writing for a classroom assignment.One student even recognized that as a future reader of this article, it would be more important to review it for a higher quality.This is especially interesting if one considers that students' reported their greatest challenge in peer review was confidence with content.Across the board, the most common attributes that students valued in science writing, in any context, were style, audience, and topic relevance.
In the classroom, students received guidelines on how to peer review for each informal or graded assignment.However, in this study, students did not receive these instructions.In an openended question, students were asked what tools they thought might be helpful for peer review of an instructor's article.Interestingly, 7 out of the 8 students stated that an example of writing from that journal would be helpful so that they could verify format, audience, and style.This indicated that students recognized that because this publication was authentic, it would need to follow a specific set of guidelines and expectations before acceptance.

Student peer review of an instructor's writing
Types of feedback from the peer review task.When I had finished a draft of a review article, I sent a copy of this draft to participants and asked for their feedback before submission.Because the majority of them recognized the need for specific materials, they also received a copy of the journal's guidelines as well as an example from the journal.It was essential that students knew this was as an opportunity to practice their peer review skills and that their feedback would be valued and implemented.
With the exception of individual D, students provided feedback using marginal comments on the draft.For this analysis, comments were counted as distinct from questions.There were several types of comments that students made and a wide range in the number of comments from each individual.While five of the eight students made six or less comments, three students made over fourteen comments each (table 1).Some comments simply identified problems (nondirective) or praised the author, while other comments identified solutions (directive) (K.Cho, Schunn, and Charney, 2006).An example of a directive comment in this study included "…Mention that [it] is a protein complex.I sort of had to assume this".Nondirective comments in this study were general comments that lacked corrective suggestions or any explanations.Overall, students gave at least as many directive comments as they did nondirective comments, though the individual number and type varied by student (table 1).While there was an overall high amount of praising comments, there was one student in particular who contributed the majority of these comments.Although students made both directive and non-directive comments, Patchen et al (2009) found that students make more praise comments than instructors and make equal comments on both problems and solutions, though they often lack explanations.Cho et al (2006) also found that students use directive and praise comments more frequently than nondirective comments.These results are supported here, as praising comments were common and directive comments were frequent.
Comments that simply re-stated content to check for understanding were rare.In these examples, a student was presumably checking for clarification without asking a question.When students did not understand the content, it was more common for them to ask questions relating to the content or indicate specific concepts they felt were unclear (tables 2, 3), suggesting they were comfortable indicating this difficulty to an instructor.It was less common for students to write nonspecific comments (table 2), such as "this was unclear", "word choice", or "great figure".Nonspecific comments can be moved anywhere in a paper because they do not reflect specific content.Some students did make specific praise comments that connected the comment to the text and provided reasoning, such as "…an appealing introductory sentence -it gives a brief explanation of why DNA is compacted into nucleosomes in the first place".This matched their peer reviewing tasks in class, where I encouraged specific and suggestive feedback.

Table 3. Reasons for comments or questions asked by students as they peer reviewed
Comment and question content from the peer review task.If taken together, almost forty percent of comments and questions related to style.An example of a comment considered purely about style would be "Long sentence that becomes confusing.Try breaking it up".Still, many of the comments and questions related to content.Interestingly, the pre-questionnaire had indicated that students expected to focus more on content for this type of peer review than for a classroom assignment.Therefore, it appears most students attempted to comment on technical content throughout the article.This mirrored their course activities, where they were encouraged to practice questioning unfamiliar content.Not surprisingly, the students who commented the most on content (F and H) were also majors in the same discipline as the article, suggesting that this content was easier for them.This would be most like the process of professional peer review, where peer reviewers are usually experts in the same discipline as the article they are reviewing.
Content comments and questions varied in the level of comments, which could also reflect their knowledge level in the discipline.For example, one student might offer a technical content suggestion, such as "Briefly explain that the code directs the DNA processes by making it more/less accessible".Another student would suggest a way to re-structure a paragraph or sentence so that technical content was easier to understand indicate a transition that helped them understand a connection between two pieces of data.For example, "I didn't get exactly what the effect of the chromatin structure obstacle is.Maybe state as a last sentence the exact significance blatantly".These types of comments were considered content comments and not style comments because the comment focused on structure to enhance content understanding, indicating a confidence in organizing unfamiliar content.questions, such as "Does this mean…" or "Do you mean…".Here, these questions could be a variation of a comment where students are re-stating content for understanding but in the form of a question.However, because asking questions is a developed skill, the choice to phrase these clarifications as a question rather than a statement could indicate students' willingness to participate in a written conversation with the author.Furthermore, students' inexperience with the content was evident as their feedback lacked higher-level questions that built on prior knowledge (table 4).These types of questions would have connected the content of the reviewed article to previously learned scientific knowledge.Not surprisingly, it was more difficult for these peer reviewers to build on prior content knowledge because their backgrounds were not in this field.However, their ability to ask questions and make comments on a paper when they were unfamiliar with the topic supports the notion that they were able to build on prior knowledge of peer review and thereby transfer peer review skills.
Overall results and conclusions of the peer review task.Overall, the majority of students were able to provide thorough and helpful feedback.They demonstrated their ability and willingness to peer review an article for a faculty member.Additionally, the types of comments mirrored the exercises they performed as part of classroom peer review.Suggesting that students valued this as an authentic activity, students commented outside the basic text, checking citations and citation use as well as the figures, without being specifically asked to do so.Experience or lack of experience, did not correlate with an ability to peer review or ask content-related questions (table 4).Two students who provided little feedback on the article noted that difficulties with content prevented them from a lengthier peer review.This is supported by other studies that find content peer review is a difficult task for students (Patchan et al., 2011;Razak and Saeed, 2015), suggesting that students' confidence with content knowledge may remain a challenge to using peer review as a tool for cognitive apprenticeship.

Challenges and perspectives of peer review in a novel situation
Reflection on peer review.After completion of the peer review task, seven of the eight students responded to an open-ended post-questionnaire that asked them about their experience peer reviewing the instructor-written article.Students listed the challenges they faced.Most of the comments related to the technical content of the article.A lack of knowledge on the topic caused low confidence in their ability to review the article and find errors.However, almost all students reported that they used online resources for background reading on the topic before peer reviewing the article.Therefore, the students attempted to learn the material in advance to increase their understanding and ability to comment on content.Interestingly, for most students, a low level of confidence in the subject matter made it easier for them to ask clarifying scientific questions about the content.One student attributed this questioning ability to curiosity while another mentioned, "I thought that if I had questions, there might be someone else out there with the same questions." The participants also confirmed other additional preparation they undertook for this task.Six of these students read the example article and guidelines before providing feedback, and found this material helpful in knowing what to expect and how the article should be formatted.Additionally, it may have been challenging for students to provide peer review to a faculty member if they felt intimidated by the process.Yet, six of the students reported viewing me as a peer who genuinely needed and wanted help on a writing task, rather than a faculty member.Though one student did note some intimidation by this process, the majority mentally prepared for the peer review task by acknowledging a more collegial relationship, rather than an instructor-student or advisor-student relationship.This relationship may have been necessary for them to feel confident enough to comment at this level.This type of relationship mirrors a better cognitive apprenticeship, where the student becomes more independent and encultured into the profession.
Transfer of peer review skills from the classroom to a new task.Students reflected on their classroom experiences compared to this novel experience in several open-ended comments in the post-questionnaire.The following quotes from students answered the prompt: "Did knowledge of the peer review process and professional publishing of journal articles (learned in the course) influence your feedback, either directly or indirectly?If so, please explain."Therefore, even though the content was challenging at times, students knew what to peer review in an article due to their previous experience.They reported following a series of unwritten "steps" to stay focused, based on classroom peer review.
Students also commented on how the class influenced their feedback by answering the question "At any point in this process, did the quality of feedback you received throughout the semester (by peers or me) affect how you gave feedback to me?" Example comments included:

"I found that it is important to give constructive comments as well as comments on what you did well in an article." "Receiving feedback [in class] helped guide what to do."
"I received a lot of good feedback from you and my peers, so that motivated me." "I think [the quality of the feedback I received] affected my feedback because it made me more confident in offering my thoughts." Overall, the quality of the feedback they had received from their peers in class, as well as from their instructor, helped give them the confidence to give similar feedback in this new situation, and focused their attention on specific parts of the article.One student even commented that receiving different kinds of peer review during the class helped construct this feedback, indicating that multiple rounds of peer review were beneficial in learning this skill.
Peer review as a non-classroom experience.Because cognitive apprenticeship aims to train students in a chosen profession, the students were asked what experiences would help them contribute to scientific research and become more confident peer reviewers as they progressed through their careers.In addition to performing research, students noted they needed more opportunities to read, write and peer review literature.One student commented, "Science is perceived as only accessible to the very elite of the field," arguing that students need more exposure to scientific articles and the chance to prove they have valid opinions and can learn from scientific literature.
Peer review is an opportunity to participate in the research process, especially when not all students have the time, opportunity, or desire to perform long-term research projects.Another student mentioned, "I would love the chance to review and edit articles outside the classroom…", as there was never the opportunity to do that outside the discipline-specific writing course.Yet another student commented, "Students just need more opportunities in general to write, read, and edit scientific papers", an opinion that many faculty would echo yet may not feel they have time to implement (Coil et al., 2010).Additional comments supported the need for more student opportunities: "I think it would be interesting to see how our feedback and [peer] reviews differ in a few years.Maturity, experience, schooling, and research may or may not have an impact on how we review articles at the present time.""I think that the best way for me to progress as a science writer is practice and reading.The more papers I read, and the more topics I write about, the better scientist I become.I am not sure of any way that could increase my confidence in my skills other than getting out there and continuing to write, read, and edit science papers.""…learning about the importance of scientific writing and how it actually affects the world/society encourages me to participate in scientific writing" "…the grad student with whom I work submitted a paper for publication… He asked me to read it and offer my comments.I was definitely more confident offering comments because of this project and the [course]." These open-ended responses confirmed that students felt that while a writing classroom experience is helpful in learning reading, writing, and reviewing skills, more situated learning and enculturation into a discipline is necessary for confidence and ability in these skills (Brown et al., 1989).Exploring skill application in new environments allows the transfer and improvement of these skills.Student responses also highlighted that these experiences are better when they are authentic, as experiences that incorporate the genuine attributes of a discipline are more likely to have the deepest cognitive impacts.Furthermore, as the original author of manuscript, I found the student feedback helpful and I implemented many of their suggested changes in a subsequent draft.This experience was therefore valuable to both the students and to me.

Summary
This study investigated the transfer of peer review skills from the classroom to an authentic nonclassroom experience.The following themes arose from the data: 1. Students generally felt that they should focus more on technical content when peer reviewing an article for a faculty member or an academic journal.
2. Students found content and confidence the most challenging aspects of peer review, regardless of the author.3. Students were able to provide directive and technical comments in a new context on unfamiliar content.4. Students used their classroom peer review experience and feedback to model how they provided feedback to the instructor. 5. Students reported a lack of opportunities to peer review outside of a writing classroom.
They felt they would improve confidence and reading ability in discipline-specific literature if given more opportunities to peer review and read academic literature.

Peer review as a professional development tool
The present study found that students were able to transfer peer review skills to a non-classroom setting and peer review using the same methods that they used in classroom peer review.One way to foster the transfer of peer review, or any skill, is to include a reflection component, as this is an essential step in cognitive apprenticeship.During the Science Writing course, students completed multiple reflections on their writing and peer review skills.These activities asked them to respond to peer reviewers' comments and create goals for their own writing, as well as explain what they had learned about the writing process from reviewing other papers.In this study, students also reflected on their peer review experience in the pre-and post-questionnaire.By asking students to perform these reflections, one can uncover student challenges and beliefs in relation to this learning (Allan and Driscoll, 2014).Therefore, pairing peer review with reflection can be a valuable tool that facilitates the growth of both the student and teacher.
In the post-questionnaire used in this study, students reflected on their experience as reviewers.The results indicated that students felt the peer review performed in the classroom prepared them for peer review in a new situation by teaching them basic skills in peer review and content analysis.However, they reported that their greatest challenges are still content and confidence related.Students' doubts and self-confidence in peer reviewing with faculty could affect the type of feedback they provide or their willingness to participate in extracurricular peer review activities (Pearce, Mulder, and Baik, 2009).There is also some concern that students will not put time or effort into peer reviewing without a grade or set of guidelines (Pearce et al., 2009).This concern could be supported by a belief that some students are not equipped or ready to participate in research, and that those students who are involved in research are learners, not coworkers (Gonyo and Cantwell, 2014) capable of peer review.In professional peer review, content knowledge issues are usually avoided by choosing reviewers within the manuscript's discipline.While this is not always possible when students practice peer review, it should not deter faculty from using peer review as a learning tool.This study suggests that students value authentic experiences, regardless of their ability or confidence level.Furthermore, several students practiced commenting on content outside their expertise.
In addition to undergraduates, graduate students and post-doctoral associates may also not have enough opportunity to peer review journal articles with their mentors.Yet, there is evidence that graduate students and faculty evaluate primary research that has been submitted to a journal equally well (Navalta and Lyons, 2010).Peer review as a professional development tool must be supported by providing opportunities to peer review in a professional setting at all levels.This is the only way to fully enculture learners into a discipline, as it welcomes them as complete members of a research community as apprentices, and improves their confidence as readers, writers, and peer reviewers.Brown et al. (1989) described successful cognitive apprenticeship that "…extend[s] situated learning to diverse settings so that students learn to apply their skills in varied contexts" (p.458).To move towards situated learning and authentic apprenticeship, it is essential to model this writing and peer review process with students and then ask them to participate in it.It also gives students more experience reading authentic literature, in contrast to the textbooks that might de-emphasize the research experience in an effort to convey basic knowledge.There are several ways a course or program could use peer review as a mode for cognitive apprenticeship in any discipline.

Suggestions for implementing peer review in cognitive apprenticeship
Within a course.Several studies have described peer review as a mode of cognitive apprenticeship within course experiences, either as part of a regular writing process or grant review process (Huiling, 2008;Kolikant et al., 2006;Yang, 2011).Therefore, classroom experiences have been established as working methods for the modelling, coaching, scaffolding, articulating, reflecting, and exploring steps of cognitive apprenticeship.Further use of cognitive apprenticeship within a course could include the following: 1. Non-writing courses can also use a mock peer review process on several types of communication.To efficiently explore peer review, it should be performed on projects, posters, presentations, and smaller writing assignments.2. Courses can involve non-teaching faculty for a peer review component.For example, students could peer review with their own advisors or research mentors as part of the writing process.3. Instructors could model peer review of a journal article and then have students write and compare peer reviews of the same article.4. Faculty could share their own research process with students, including the writing process.This may include asking students to give feedback to faculty, graduate students, or postdocs on papers before submission.
Outside a course.Peer review within the context of undergraduate or graduate research is an obvious example of cognitive apprenticeship.Often, this does occur through informal peer review of a student's writing for a dissertation, thesis, conference proposal, or poster.However, not all students are able to participate in research.Furthermore, some students may want additional practice through extra credit or a student club.These out-of-classroom experiences could include the following: 1. Faculty can informally ask graduate and undergraduate students to read and review several stages of writing, prior to submission.Students could also read formal, anonymous peer review feedback from these submissions and faculty can model how they respond to reviewers' comments and make changes to their own writing.2. Before research presentations (e.g. a departmental seminar), faculty members could share written, unpublished research.Students and faculty could meet before seminars to discuss the research and compare peer reviews of the writing and the technical content.This would have additional benefits in that students would be better able to understand the research prior to the oral presentation and have time to develop more complex content questions.
3. Departments could provide community opportunities for students to participate in research clubs, where students would meet on a regular basis to discuss current research interests in a department.These discussions would center on a piece of writing, which students would review before the discussion.They could also do this as mockreviewers for a journal.4. Students could run regular peer review sessions for each other to offer feedback in an informal and comfortable environment.This is beneficial to both the reviewer and the writer (Y.H. Cho and Cho, 2011;Lundstrom and Baker, 2009;Rouhi and Azizian, 2013). 5. Students could develop and run a publication or organization, where students write and peer review all submissions.

Limitations and Future Directions
Due to its authentic and situated nature, this study was limited to a small number of participants and could not include a control group.The IRB regulations did not allow a comparison between individual student feedback and performance in class.The IRB also did not allow this study to comment on these students in terms of a representation of the overall course population.Additionally, student self-reported their challenges and perspectives as part of a reflection process in the pre-and post-questionnaire.Further research is needed on the application of peer review as a situated learning tool in a non-classroom environment.For example, it would be interesting to explore how the quality and types of student comments and questions change as they peer review in different settings.Additionally, student and faculty peer review comments could be compared outside a writing classroom.Because students reported that they would focus more on technical content when an article was intended for publication, asking students to peer review literature written for different purposes could reveal how authenticity affects feedback.
Students in this study were able to ask some content-related questions on the article as they peer reviewed.Still, the ability to ask higher-level content questions is associated with deeper learning of a subject.Students who participate in a more active-learning environment can develop complex or hypothesis-like questions, in contrast to students who learn content more traditionally (Marbach-Ad and Sokolove, 2000).Therefore, the active process of peer review within and outside of writing classrooms could improve both technical content knowledge and the ability to comment on it and question it.A future exploration should investigate how these comments change over time, especially if faculty model how to comment on and question technical content and then ask the students to practice these skills in subsequent peer review experiences.This cognitive experience would be improved by asking students to evaluate these edited drafts at different stages of the process.

Klucevsek
Journal of the Scholarship of Teaching and Learning, Vol. 16, No. 6, December 2016.josotl.indiana.edu110 My exposure to all of the articles that we read in class taught me a lot about what to expect and look for while reading this paper.""Without the knowledge from class, I would not have known what sort of things I should have been aware of and looking for.It also helped me to read through the article in a logical fashion.""Having edited before, I had a mental checklist of what to do following the class format."

Table 4 . Type of student questions paired with the presence or absence of prior experience peer reviewing science writing in a research experience
While participants were more likely to comment on the article rather than ask questions, the types of questions that students asked reflected levels of cognition.Basic questions ask to identify something, such as "What is…".Over half of the questions asked were clarification