Students submitted more than 22 million papers that may have used generative AI last year, new data from plagiarism detection company Turnitin shows.
A year ago, Turnitin launched an AI font recognition tool trained on student-written essays and other AI-generated texts. Since then, more than 200 million essays have been checked by the detector, most of which were written by high school and college students. Turnitin found that 11 percent may contain AI written language in 20 percent of their content, with 3 percent of total works reviewed flagged as having 80 percent or more of AI writing. (Turnitin is owned by Advance, which also owns WIRED publisher Condé Nast.) Turnitin says its detector has a false positive rate of less than 1 percent when analyzing complete documents.
The launch of ChatGPT sparked knee-jerk fears that the essay would die for English classes. The chatbot can synthesize and distill information almost instantly – but that doesn’t mean it always gets everything right. Generative AI has been known to hallucinate by creating its own facts and citing academic references that do not exist in reality. Generative AI chatbots have also been caught spitting out biased texts about gender and race. Despite these shortcomings, students have used chatbots for research, organizing ideas, and as ghostwriters. Traces of chatbots have even been found in peer-reviewed, published academic papers.
Understandably, teachers want to hold their students accountable for using generative AI without permission or disclosure. However, this requires a reliable method to demonstrate the use of AI in a specific task. Teachers have at times tried to find their own solutions to detecting AI in writing, using messy, untested methods of enforcing rules and causing distress to students. To further complicate the issue, some teachers are even using generative AI in their grading processes.
It is difficult to identify the use of genetic AI. It is not that easy to report plagiarism because the generated text is still the original text. Additionally, there are nuances to how students use genetic AI. Some may ask chatbots to write most or all of their work for them, while others may use the tools as a resource or brainstorming partner.
Students aren’t just seduced by ChatGPT and similar large language models either. So-called word spinners are another type of AI software that rewrites text, potentially making it less obvious to a teacher that the work was plagiarized or AI-generated. Turnitin’s AI detector has also been updated to detect word spinners, says Annie Chechitelli, the company’s chief product officer. It can also flag work rewritten by services like spell checker Grammarly, which now has its own generative AI tool. As well-known software increasingly adds generative AI components, it becomes increasingly unclear what students can and cannot use.
Detection tools themselves carry the risk of bias. English learners are more likely to trigger them; A 2023 study found a false positive rate of 61.3 percent when evaluating TOEFL (Test of English as a Foreign Language) exams with seven different AI detectors. The study did not examine Turnitin’s version. The company says it has trained its detector on the writing of English learners and native English speakers. A study published in October found that Turnitin was among the most accurate of 16 AI speech detectors in a test in which the tool examined coursework and AI-generated papers.