ChatGPT in schools: What is it and how to detect it

A year 8 student is asked to write an assignment on the effects of the black death and opens with: 

The pestilential scourge of the Black Death wrought a cataclysmic impact upon mediaeval society, precipitating a profound demographic shift, intensifying social upheaval, and catalysing a fundamental restructuring of economic and cultural institutions that reverberated throughout the ages.

Welcome to the new age of AI-assisted assignments. 

It’s no secret that with the ever increasing technological advances, comes many challenges in the education industry. Teachers deserve a lot of credit for always adapting to these advances in technology: modifying teaching techniques and often dealing with a remodelling of the system. 

What comes with these developments, is students finding new and innovative ways to cheat on their assignments and exams. Teachers and educators alike have to stay on their toes with the new methods of cheating and as I’m sure you are aware the most advanced one yet has just burst onto the scene - ChatGPT. 

What is GPT?

ChatGPT is an intelligent AI software that can write virtually anything and has the ability to interact in a conversational way. The language model can answer questions, generate paragraphs of texts, write essays, letters, emails and more with impressive language. It can even further iteratively refine text based on additional inputs.

Why does it pose a risk to education in Australia? 

The ability for ChatGPT to write sophisticated responses to questions almost immediately is unprecedented. Students in schools and universities are capitalising on this and it’s important to look out for the signs.

ChatGPT can provide information quickly and efficiently, but it may not always provide the depth of understanding and critical thinking skills that are necessary for effective learning. In addition, ChatGPT's responses are generated based on the data it has been trained on, which may not always be accurate or up-to-date.

Furthermore, over-reliance on ChatGPT as a primary source of information could lead to a lack of critical thinking and analysis skills among students. It is important for students to learn how to assess the credibility and reliability of sources, including digital sources like ChatGPT.

It’s crucial to set clear expectations for students up front, so they understand what is and is not allowed on a given assignment, and know the potential consequences of using model generated content in their work.

How to detect ChatGPT 

ChatGPT poses a real challenge for teachers and there are concerns around the ethics of the AI model. Whilst there is no clear-cut method of detecting the use of ChatGPT, there are many tips and tools which can be used to help identify. While it may be difficult to detect, there are several strategies that educators can use to identify students who are using ChatGPT to cheat.

One method is the recently developed AI text classifier which tells the likelihood of the work submitted being generated by AI technology. This is not a fool proof way of detecting it as it can produce both false negatives and false positives. Therefore it should only be used in addition to other indicating factors when investigating a holistic assessment of academic dishonesty or plagiarism.

That being said here are some tips that may help you identify that a student has used ChatGPT for their assignment:

• Similar topics/points of view from students

One warning sign is if you notice that multiple students in your class have submitted essays with strikingly similar topics or points of view. Since ChatGPT is an automated service, it could be possible that more than one student has accessed and used it to complete their assignments.

• Monitor the writing style of the student

Another way to detect students who are using ChatGPT to write their assignments is to monitor their writing style. ChatGPT is designed to mimic human writing, but it may have distinct patterns or quirks that can be identified. Furthermore, ChatGPT is a powerful language model. If a primary school student starts producing complex and sophisticated writing/language styles it may be an indicator that it is not their own work.

• Ask specific questions during grading

Another strategy is to ask specific questions during grading that require the student to demonstrate an understanding of the material. If a student has used ChatGPT to write their assignment, they may struggle to answer these questions or may provide incorrect answers. 

• Analyse the complexity of the writing

Educators can use this information to investigate further and determine if the student is using ChatGPT to write their assignments.

• American Language:

Chat GPT was developed in America and therefore gives its responses in American language. For example, if you notice that some words such as ‘prioritise’ are spelt ‘prioritize,’ then there is a good chance that this student has been scraping information from alternative sources such as ChatGPT to write an answer. 

To summarise, there is no set way of determining whether or not a student has used ChatGPT but there are numerous indicators that can help you identify its use. It’s important not to deem the student guilty immediately without a sophisticated investigation. Keep an eye out for the signs and act appropriately. 

Ultimately, as with any developing technology the rest of the world will need to find new ways to adapt. This might mean fewer “homework” assignments and a greater curriculum reliance on different ways to test and measure those essential educational skills.