Reconstructing Response to Student Writing: A National Study from across the Curriculum

Reconstructing Response to Student Writing

Reconstructing Response to Student Writing

A National Study from across the Curriculum

Dan Melzer

UTAH STATE UNIVERSITY PRESS

Logan

© 2023 by University Press of Colorado

Published by Utah State University Press

An imprint of University Press of Colorado

1624 Market Street, Suite 226

PMB 39883

Denver, Colorado 80202–1559

All rights reserved

Printed in the United States of America

The University Press of Colorado is a proud member of the Association of University Presses.

The University Press of Colorado is a cooperative publishing enterprise supported, in part, by Adams State University, Colorado State University, Fort Lewis College, Metropolitan State University of Denver, University of Alaska Fairbanks, University of Colorado, University of Denver, University of Northern Colorado, University of Wyoming, Utah State University, and Western Colorado University.

ISBN: 978-1-64642-448-1 (hardcover)

ISBN: 978-1-64642-367-5 (paperback)

ISBN: 978-1-64642-368-2 (ebook)

https://doi.org/10.7330/9781646423682

Library of Congress Cataloging-in-Publication Data

Names: Melzer, Dan, author.

Title: Reconstructing response to student writing : a national study from across the curriculum / Dan Melzer.

Other titles: Reconstructing response to college writing

Description: Logan : Utah State University Press, [2023] | Includes bibliographical references and index.

Identifiers: LCCN 2023000214 (print) | LCCN 2023000215 (ebook) | ISBN 9781646423675 (paperback) | ISBN 9781646424481 (hardcover) | ISBN 9781646423682 (ebook)

Subjects: LCSH: English language—Rhetoric—Study and teaching (Higher) | Report writing—Evaluation—Study and teaching (Higher) | Peer teaching. | Students—Self-rating of. | Group work in education. | Team learning approach in education.

Classification: LCC PE1404 .M454 2023 (print) | LCC PE1404 (ebook) | DDC 808/.042071—dc23/eng/20230127

LC record available at https://lccn.loc.gov/2023000214

LC ebook record available at https://lccn.loc.gov/2023000215

This publication was supported, in part, by University of California, Davis.

Cover image © johnwoodcock/istock.

To the memory of my friend and mentor, Dr. Richard Straub.

Contents

Acknowledgments

1. A National Study of Response to Student Writing

2. A Constructivist Heuristic for Response

3. Teacher Response to Writing

4. Peer Response to Writing

5. Students’ Self-Assessment of Their Writing

6. Reconstructing Response

Postscript: Reflections on Two Decades of Researching College Writing and Responding

Appendix: Institutions Included in the Research

References

Index

Acknowledgments

I would like to thank two reviewers who had a major impact on the organization and development of this book, Michael Rifenburg and an anonymous reviewer. Their advice regarding my research methods, the scope and organization of each chapter, the presentation of my data and analysis, and attention to diversity and access in response had a transformative effect on the multiple drafts of this book. I would also like to thank Rachael Levay, editor at Utah State University Press, for her smart guidance and advice throughout the process. Finally, I would like to thank my colleagues at the University of California, Davis, Amy Lombardi and DJ Quinn. Their help in analyzing my data led me to new understandings and findings. In addition to these thanks for the helpful feedback I received during the process of drafting and revising the book, I would like to acknowledge the Writing Studies and TESOL scholars whose work on response and self-assessment has influenced my own research and teaching, among them Chris Anson, Kathleen Blake Yancey, David Boud, Lil Brannon, Nancy Falchikov, Dana Ferris, Lynne Goldstein, Brian Huot, Ken and Fiona Hyland, C. H. Knoblauch, and Nancy Sommers.

Reconstructing Response to Student Writing

1

A National Study of Response to Student Writing

When I am writing my papers I usually take into account which target audience will be reading my paper, especially if I am writing for a scholarship or a job. However in a school setting I am not too worried about my audience because the audience is one person, the teacher. Sometimes it’s hard writing papers to a new teacher because in the beginning I do not know my teacher very well and it is hard to decide what kind of style of writing to use. However after the first paper I see the comments and then I can kind of get a feel to whom I am writing for.

Excerpt from a first-year writing student’s midterm reflection

The perspective on instructor response described in the student reflection above is an all too familiar one to teachers. Over the course of my college teaching career, I have certainly been guilty of designing response in the narrow, teacher-focused way this student describes. Too often as a writing teacher, I constructed response in my classes as a means to meet what must have felt to students as idiosyncratic criteria I handed down to them in a rubric that was designed in large part to justify a grade on a final draft. I always looked forward to reading my students’ drafts, but I wondered if all of the response I was giving my students was worth the effort. Whether it was bringing home a pile of stapled student essays to pore over all weekend in my earliest days of teaching, or trying to find a quiet place to scroll through electronic files of student drafts to insert my supposedly helpful comments later in my career, I wondered if students were paying close attention to my feedback and applying it to future drafts, and if students were able to transfer my suggestions to the writing they were doing in their other courses. I promote language diversity in my assignments, but as a white cis male, I worried about the ways that implicit biases impacted my feedback, and I wondered how my response was received by the diverse student populations of the institutions where I taught. Even though I tried to get students involved in giving feedback by asking them to respond to each other’s drafts, it often felt to me that students viewed peer response as a waste of time, since I was the one who would ultimately be giving them a grade. Response also felt isolated and disembodied. Whether it was returning at the end of class a set of marked-up essays early in my career or emailing students an electronic file with comments later in my career, there was little dialogue, and students were mostly passive recipients of my comments.

I began to ask myself some fundamental questions about the way I was constructing response for my students; questions that I explore in the research reported on in this book. I began to question my own dominant role in response, and I wondered what the research had to say about the role of peer response and student self-assessment. I began to rethink what the focus of response should be, and I began to explore what the alternatives were to focusing on sentences and paragraphs of a rough or final draft. I became curious about what kind of response students were getting from teachers in their other classes. What did teachers across disciplines focus on when they responded, and what were students’ perspectives on the feedback they received from their college teachers? When students engaged in peer response or self-assessment, how did their feedback and self-reflections differ from teacher feedback? Were there national trends in the ways teachers across disciplines respond that could help inform my teaching and the advice I gave to teachers in writing across the curriculum (WAC) faculty development workshops?

My interest in exploring response to college writing on a national scale began with a previous large-scale research project—a study of over two thousand college writing assignments from across disciplines that was reported in the book Assignments across the Curriculum (2014). In Assignments across the Curriculum, I analyzed teachers’ evaluation criteria included in their writing assignments and rubrics, but I did not look at student writing and teacher response to student writing. I found that teachers who claimed to value content and critical thinking in their assignment prompts often focused on grammatical and citation style correctness in their assessment rubrics. I found an overall obsession with correctness of form, language, and format in assignment criteria and assessment rubrics. And I found that most teachers played the role of judge, asking for regurgitation of textbook or lecture information in exams. However, I also found that it was common for teachers across disciplines to respond to drafts and to make use of peer response, especially in courses that satisfy a writing-intensive requirement. What I did not have evidence of in the data reported in Assignments across the Curriculum was the extent to which teachers’ responses to student writing matched the examiner role they set for themselves in their assignment prompts, or what role students played in peer response. I also did not have evidence of how students responded to the assignments and the comments they received from teachers in the form of student drafts in progress and final drafts.

Reconstructing Response to Student Writing is part 2 of an ongoing research project that aims to provide a national view of college writing in the United States, offering evidence that Assignments across the Curriculum lacks and providing a bird’s eye view of the other side of the coin of assigning writing—responding to it. In a review of response and assessment research in higher education, Carol Evans (2013) observes that most studies of response are small scale, single subject, opportunistic, and invited (77). Understandably, most studies of college teachers’ response to student writing focus on a single course or a small number of courses, providing depth but not necessarily breadth. Perhaps this is one reason there have been few book-length studies focused on response. Evans calls for more large-scale response and assessment research, and Writing Studies scholars have begun to answer that call. Recent large-scale studies of response involve corpuses of thousands or even tens of thousands of teacher or peer comments (I. Anson and C. Anson 2017; Dixon and Moxley 2013; Lang 2018; Wärnsby et al. 2018). These researchers analyze big data to discover patterns about teacher and peer commenting on writing that provide a valuable complement to smaller scale studies. However, by focusing solely on written comments, these large-scale studies of response understandably lack context. Recently researchers have called for a greater focus on student perspectives in studies of response (Anson 2012; Edgington 2004; Formo and Stallings 2014; Lee 2014; Zigmond 2012), and large-scale studies lack the important context of the students’ perspectives on the feedback they receive from peers and the teacher. Both small- and large-scale studies usually focus on one actor in the response construct (typically the teacher or peers) and one component of response (e.g., comment types, mode of delivery, the impact of feedback on revision). For pragmatic reasons, it is understandable that response researchers would narrow their focus in this way, but this narrowing often results in researchers not being able to capture the complex social contexts of response constructs.

To date no researcher has completed a national study of response to college writing that attempts to include and synthesize the many actors that make up scenes of response, the multiple components of response constructs, and the perspective of the most important actor in response: students. In Reconstructing Response to Student Writing, I present the results of a corpus study that aims to provide a panoramic view of response to college writing in the United States while also providing richer contexts than prior large-scale studies of response and a consideration of the multiple factors and actors that make up response constructs. My corpus includes teacher and peer responses to over one thousand rough and final drafts of student writing as well as student reflection on response and self-assessment of their writing from first-year writing courses and courses across the college curriculum. In addition to reporting on my analysis of tens of thousands of teacher and peer comments, I consider the impact of these comments on students’ drafts. Most importantly, throughout Reconstructing Response to Student Writing I provide students’ perspectives on teacher and peer comments and students’ own self-assessment of their writing. I also introduce a heuristic that takes into account the varied factors that should be considered when researching response and when designing response constructs. The heuristic is aligned with recent response research, which draws primarily on social-epistemic theories of literacy and learning.

Constructivist Response Research

Constructivism emphasizes both the social context of learning and the learner’s central role in the creation of knowledge. A constructivist approach to response takes into account the prominence of social-epistemic theories in recent response research (Anderson 1998; Askew and Lodge 2000; Crook 2022; Evans 2013; K. Hyland and F. Hyland 2019; Molloy and Boud 2014; Price and O’Donovan 2006; Siczek 2020; Villamil and de Guerrero 2020), the growing body of knowledge on student self-reflection and self-assessment (Boud 1995; Falchikov 2005; Yancey 1998b), and recent research on transfer and writing (Anson and Moore 2016; Moore and Bass 2017; Yancey et al. 2014). Constructivist response considers the entire social construct of responding: the student, teacher, class, assignment genre, discipline, and sociocultural and sociopolitical contexts. Constructivist response encourages social interaction and dialogue rather than response as a one-way transmission from teacher to student.

In a constructivist model of response, each factor of the response construct affects the others. For example, the common practice in English as a Second Language (ESL) courses of dynamic written corrective feedback improves students’ ability to correct sentence-level errors, but it may also reinforce students’ perceptions that good writing is merely correct writing. Assignment genre choices will ultimately affect the teacher’s approach in responding, depending, for example, on whether the genre assigned has strict or flexible writing conventions. Changing the mode of response and moving peer response from face-to-face in the classroom to an online forum will affect students’ orientation to peer response, depending on students’ comfort with the technology, their preferences regarding face-to-face versus digital feedback, the type of technology used, and so on.

Constructivist response emphasizes the learner’s central role in constructing response, including student self-assessment and peer response. In this way, the research on self-reflection and transfer is relevant to constructivist response. The Writing Studies scholarship on transfer has emphasized writing and reading assignment design (Adler-Kassner et al. 2012; Anson and Moore 2016; Beaufort 2007; Carillo 2014; Downs and Robertson 2015; Moore and Bass 2017; Wardle 2009; Yancey et al. 2014; Yancey et al. 2018; Yancey at al. 2019), but perhaps because this research has focused mostly on designing curriculum, the transfer scholarship has not delved into the role of responding in writing transfer. The Teaching for Transfer literature has had little to say about responding for transfer. International literature on response does explore the concept of feedforward, but this concept has tended to focus on response that can be applied by the student to the next assignment within a course, rather than response aimed at more far-reaching transfer (Carless 2006; Duncan 2007; Martini and DiBattista 2014; Orsmond and Merry 2011; Pokorny and Pickford 2010; Vardi 2012).

Influential models of response, such as Brian Huot’s (2002) Theory of Response in his seminal book (Re)Articulating Writing Assessment for Teaching and Learning, and Lynne Goldstein’s (2005) response model in Teacher Written Commentary in Second Language Writing Classrooms, frame response as a dialogue between teacher and student but do not put student self-assessment at the center of the response construct. Goldstein argues that the key to response is the effectiveness of the commentary provided and the quality of the communication between teachers and students about the students’ revisions (4). Huot encourages teachers to involve students in all stages of the evaluation of their work in a process he refers to as instructive evaluation (69). But Huot’s chapter focused on building a theory of response emphasizes the central role of the teacher as responder, even as Huot argues that the teacher must remain in dialogue with the student. Thanks in large part to the scholarship of David Boud and Nancy Falchikov, self-assessment and self-reflection have been more integral to recent research on response and assessment in international scholarship. Self-assessment is integrated in John Hattie and Helen Timperley’s (2007) Model of Feedback to Enhance Learning, and student self-assessment is central to Charles Juwah and coauthors’ (2004) Model of Formative Assessment and Feedback.

As a tool for researchers to capture the social and cognitive contexts of response and for teachers across the curriculum to design more sophisticated response constructs that invite students to play a more central role in their own learning and assessment, I introduce a constructivist response heuristic (figure 1.1). The heuristic is built around fundamental questions that researchers can ask in studying response and that teachers can ask in constructing response for their classes. The heuristic distills, organizes, and synthesizes fifty years of empirical research on response in Writing Studies and Teaching English to Speakers of Other Languages (TESOL). Additionally, the heuristic organizes my analysis of the data in my corpus. Because the heuristic is informed by constructivist educational theory and research in response that points to the value of peer response and student self-assessment, the heuristic involves a conceptual reframing of response research and shifts the focus of researching response and of designing response constructs from teachers to students. In answering the questions who should respond, what should response focus on, and what contexts should be considered when responding, the heuristic emphasizes the student: student self-assessment, students’ literacy histories, and students’ ability to transfer knowledge to future writing contexts. The heuristic consists of six interrelated questions researchers can consider when studying response.

Figure 1.1. A constructivist heuristic for response.

The heuristic is informed by the results of my national study of response to college writing and a comprehensive review of the literature on teacher and peer response and student self-assessment, including research from Writing Studies, English as a Second Language/ English as a Foreign Language (ESL/EFL), Writing across the Curriculum (WAC), and international scholarship published in English. I discuss my approach to both my primary and secondary research in more detail in the following section.

The Research Design

Data Collection

When I set out to study response to college writing, I did not want to solicit responses directly from teachers, in the fear that they would share only what they considered to be their best comments. I also wanted richer data than just teacher comments, and I especially wanted to include student voices, which were missing from Assignments across the Curriculum and from much response research. At the time that I was considering a project on response, feedback to student writing was not accessible online in the same way I was easily able to collect a large corpus of writing assignments via Internet searches for my research in Assignments across the Curriculum. I postponed the project on response and spent the next few years collaborating with colleagues on a different project focused on a methodology for developing sustainable WAC programs. When that project was completed, I revisited my idea of a national study of response, and by this time—2018—I discovered that it was possible to collect a large corpus of response via the Internet. The key was to focus my search on ePortfolios.

Portfolio assessment is a pedagogical approach that involves students collecting their work for the class in a portfolio and the teacher typically assessing the compiled portfolio rather than individual assignments. Contents included in a portfolio of student work vary, but most portfolios include rough and final drafts of student writing and a culminating self-reflection memo/letter/essay (Calfee and Perfumo 1996; Yancey 1996, 2009; Yancey and Weiser 1997). Portfolio reflection essays have been of special interest to researchers focused on student self-assessment of their writing (Bower 2003; Emmons 2003; Yancey 1998b). ePortfolios have become popular in first-year writing courses and are becoming more common in courses across disciplines and as a tool for students to collect and reflect on their work throughout their academic career, thanks in part to the availability of robust ePortfolio platforms such as Digication and Mahara. Through Internet key term searches such as teacher comments, peer feedback, and reflection essay combined with the term portfolio, I was able to locate student ePortfolios that collect work from individual courses as well as institutional portfolios that gather work from students’ entire undergraduate careers. Most of the ePortfolios I collected include multiple artifacts of both peer and teacher feedback as well as student self-assessment in the form of process memos and portfolio reflection essays. Common platforms students used to create the ePortfolios in my corpus include Digication, WordPress, and Weebly. Because there were few ePortfolios available published in courses outside of the United States, and because I considered this study to be the second part of the work I began with a study of writing assignment in US institutions of higher education, I focused only on courses at US institutions.

Carol Rutz (2004) argues that piles of student papers may bear thousands of fascinating teacher comments, but at least half of the story remains untold as long as student writers are not part of the conversation (122). I was especially interested in studying how students react to response from their teachers and peers, and what I found to be extremely useful qualitative data available in the ePortfolios were the many cases in which students reflect on teacher and peer responses and on their own writing processes in process memos, introductions to portfolios and to individual web pages, midterm reflections, and final portfolio reflection essays. Most of the portfolios in my corpus contain at least some student reflection on peer and teacher comments, and a little over half of the portfolios (128) include extended portfolio reflection essays that reference peer and/or teacher feedback. Throughout Reconstructing Response to Student Writing, student voices are predominant.

In 2018 and 2019, I searched online for as many