Research reveals the advantages and the limitations of GenAI feedback, providing insights for its application in L2 writing instruction

Generative artificial intelligence (GenAI) tools, which mimic human-like conversations have the capacity to respond to questions, admit mistakes, and provide instantaneous linguistic feedback. A new study among college students in China has revealed the technological, educational, and social applications, as well as the limitations, of GenAI feedback. This study highlights the potential of GenAI feedback to enhance L2 writing outcomes, despite its current shortcomings.

A generative artificial intelligence (GenAI) tool named ERNIE Bot represents a valuable tool for students by assisting them in language corrections, generating ideas, evaluating content clarity, providing feedback on a draft’s organization, changing the tone from informal to academic, and adding cohesive elements like discourse markers. Integration of GenAI tools into writing feedback processes may help to provide information in a deep and personal manner for students and enhance its capability as a powerful educational tool in a supportive language learning environment.

In a study published online on January 22, 2025 in the ECNU Review of Education, a team of researchers from Huaihua University, Central South University, and the National University of Defense Technology (NUDT) examined the use of GenAI feedback among Chinese university students. Led by Professor Xuchuan Chen from NUDT, the study explored students’ practices and perceptions of this technology.

For this study 12 students were selected from a group of 200 students attending a 16-week English course at a mid-sized public university in China. Volunteers were selected to ensure a mix of different genders and varying levels of language proficiency. Throughout the course of the study, students were tasked with composing four distinct essays ranging from 300 to 500 words on topics designed to challenge various writing skills. After the completion of each essay, the students were asked to exchange their work for peer feedback. Subsequently, the students also sought GenAI feedback by submitting their original essays to the ERNIE Bot AI using suitable prompts. After receiving feedback from their peers and ERNIE Bot, the students were asked to make the necessary revisions to their writing.

The data for this study was collected from multiple sources, including the writing drafts of each participant, the reviews from peers, the chat logs with ERNIE Bot, and the semi-structured interviews. Each time, after receiving the reviews from the peers and the GenAI, the participant was interviewed for 20 to 30 minutes. The interview transcripts were used as the primary source of data, which was then correlated with the other data sources. The data was then analyzed using both inductive and deductive thematic analysis.

The researchers were able to identify key advantages of GenAI feedback in technical, educational, and social aspects. They were able to specifically highlight the technological benefits, including the timeliness and personalization of feedback; the educational advantages, including its roles as both an essay editor and a responsive tutor; and the social benefits, including its ability to provide a supportive environment and an engaging process. Furthermore, the participating students highlighted the comprehensive educational function of GenAI feedback. They shared that their interactions with the GenAI provided them with greater learning opportunities to improve their grammar and vocabulary.

“Instead of prohibiting GenAI completely, teachers can seek methods to collaborate with these AI-driven tools and harness their benefits,” suggests Researcher Prof. Mi Rong.

The researchers also reported that the integration of these technologies in L2 writing learning has its own challenges. They revealed issues such as the limited AI skills of end users, the risk of dependency that could hinder self-learning, and GenAI’s limitations in understanding human emotions.

“Our findings suggest that peer review can still be used alongside GenAI feedback. As a valuable resource for providing feedback on emotional aspects of writing, peer review should be kept for providing feedback on emotional expression,” Researcher Prof. Yudan Mi says.

In conclusion, the insights gained from this study may provide answers to the broader discussions on the viability of using GenAI in the field of educational technology. The study may also provide practitioners, and policymakers with valuable perspectives aimed at leveraging the capabilities of GenAI to enrich L2 writing outcomes while maintaining educational equity and empathy.

***

Reference
Title of original paper: Exploring the Affordances and Challenges of GenAI Feedback in L2 Writing Instruction: A Comparative Analysis With Peer Feedback

Journal: ECNU Review of Education

DOI: 10.1177/20965311241310883