A generative AI writing tutor that produces personalized feedback for English learners

Generative AI tools—artificial intelligence models that can create images, texts, videos, or other content in short periods of time—have recently become increasingly popular, especially after the public release of ChatGPT. ChatGPT, the popular chatbot by OpenAI, is now being used by countless users worldwide to source information and even produce creative texts or academic essays.

The use of ChatGPT by students is already sparking considerable debate, as it could lead to students being graded on assignments that they did not produce themselves and potentially even dishonestly advancing in classes that they did not study for. A team of researchers at Columbia University recently developed Scraft, a generative AI-based system that can support students' learning, instead of helping them to cheat on their assignments.

This AI-based tool, introduced in a paper pre-published on arXiv, can generate useful grammatical feedback on English writing assignments. Scraft could be particularly helpful for foreign students who are trying to improve their English writing skills, helping them to spot mistakes that they are making often and understand how to fix them.

"We naturally came to this idea as an ESL student who grew up struggling with writing education and as a current ESL writing tutor interacting with other ESL students," Tyler Kim, one of the researchers who carried out the study, told Tech Xplore. "We realized that writing feedback is hard, expensive, and [time consuming to] deliver, which prevents students from becoming better writers. This is a serious problem in Korea where English, despite being a second language, is critical for educational and job prospects, as more than half of job postings require English writing proficiency."

Scraft, the generative AI-based system created by the researchers, was designed to offer useful feedback based on the Socratic questioning method. Socratic questioning, named after the Greek philosopher Socrates, is a well-known approach for producing insightful questions that prompt reflection and learning.

"Scraft is an AI writing tutor for English learners that provides real-time, personalized writing feedback and targeted writing exercises that pinpoint the writing weaknesses of individual students," Kim explained. "It uses generative AI to understand writing context, provide actionable feedback, and thought-provoking questions to promote critical thinking that fosters language acquisition."

Kim and his colleagues already produced a working prototype of Scraft and evaluated it in a preliminary student involving 15 university students. These students were asked to use the AI-based tool for two weeks while working on a writing assignment and then share their thoughts and feedback during 15-minute interviews.

The interviewed students said that they had found Scraft very useful and that it helped them to improve their English writing skills. They were particularly pleased with the insightful questions it produced and said that it pin-pointed mistakes or aspects of their writing that they had not been aware of.

"We were able to use generative AI to produce educational feedback, rather than for the most intuitive use-case: text/essay generation," Kim said. "As students, we realized using an AI text generator is basically mindlessly watching AI generate content which doesn't help us become better writers. We wanted to use the same technology in an educational manner that helps us become better writer, instead of just making our writing better."

The researchers are planning to release a public beta version of Scraft in early June 2023 and hope that it will help many English learners worldwide to improve their writing skills. Meanwhile, they will conduct further studies aimed at further evaluating and improving their AI writing tool.

"We are continuing our research in two main directions," Kim added. "Firstly, we wish to explore how users are interacting with AI writing feedback and our tool. Then we plan to test whether AI feedback helps language acquisition and quantifying its results."