ChatGPT or human text? Here’s one surprising way to tell the difference

ChatGPT’s ability to flawlessly compose a paper within seconds has raised concerns about the future of education. Cheating has never been easier. Issues with the AI chatbot have created demand for AI-generated text detectors. However, a new study shows there are some key characteristics that can help distinguish between text created by ChatGPT and text written by humans.

For the study, researchers built an ML model that looked for patterns and characteristics in ChatGPT responses that could help distinguish them from human-generated text. Two experiments prompted ChatGPT to generate restaurant reviews and also prompted it to rephrase the original human-generated text reviews, according to the study. 

The observations showed that ChatGPT describes experiences rather than shares feelings. It avoids personal pronouns, uses some unusual words and, interestingly, never uses aggressive or rude language. For example, it used the atypical word “inattentive,” as well as exaggerating language like “absolutely delicious” and “overly polite”. 

Also: How to get started using ChatGPT

The study’s results indicate that ChatGPT’s writing style is extremely polite. And unlike humans, it cannot produce responses that include metaphors, irony, or sarcasm. Does that mean the true differentiator between artificial and human intelligence is our rudeness? 

“It is extremely polite, aiming to please different types of requests from various domains fairly well mimicking humans, but that still does not have the profoundness of human language (e.g. irony, metaphors,…),” the study said. 

Other indicators of ChatGPT written text included a lack of detail. For example, in the reviews, ChatGPT used general information it knew about restaurants instead of describing restaurant specifics a human who dined in a restaurant would include. The study also found that ChatGPT repeated itself a lot, constantly using the word “restaurant” within its text. 

Also: What is ChatGPT and why does it matter? Here’s everything you need to know

If you are attempting to distinguish between human or ChatGPT written text, looking out for these characteristics may be your best bet. AI-generated text detection tools exist, but none of them perform nearly as accurately as needed. 

On Wednesday, OpenAI, the research company behind ChatGPT, released a free ChatGPT text-identifying tool; however, the tool can not be relied on. OpenAI’s “classifier” tool can only correctly identify 26% of AI-written text with a “likely AI-written” designation and provides false positives 9% of the time. 

ZDNET tested other AI text-detecting tools, including GPT-2 Output Detector, Writer AI Content Detector, and Content at Scale AI Content Detection. Results showed that these tools were unreliable as well.