A cautionary but reassuring tale of a student, a teacher, and the limitations of technology.
Names, places, and other identifying information has been redacted from the following story to protect the guilty and the busy. That being said, everything I do relate here is absolutely true from a primary source.
“I failed my first student for using ChatGPT today,” my friend told me as we sat at dinner. They teach at the higher education level, and it’s exam season.
“Oh?” The societal ramifications of ChatGPT is one of my own interests, and we’d talked before about how it would influence their work.”What gave it away?”
With their permission, let me tell you the the three things that made it really obvious the student was simply cutting and pasting answers generated by an AI.
1. It was too fast.
This was an exam students had to take online, using proprietary software from the school. That meant that there was a lot of data available for the teacher — including how long it took each student to complete the exam.
For most students, it took over one hundred minutes.
This particular student — someone whose work had not stood out in any way during the semester — completed their exam in sixty-three minutes.
Red flag number one.
2. Video killed the ChatGPT star.
For portions of the test, students had to watch a video and talk about what they saw there.
ChatGPT is not (yet) able to watch videos. However, because my friend is a conscientious teacher, all the videos included transcripts.
The student took the logical step of feeding the transcript of the video into the AI before asking the exam questions. Unfortunately, in the process of cutting and pasting, they neglected to notice where the AI had written “…according to the transcript…” in the midst of the answers.
Perhaps it was slightly possible that a student might use a transcript instead of watching a video — for example, if they were blind. But this particular student was not, in fact, blind.
Red flag number two. But both of these flags were simply the heralds of the biggest giveaway of all.
3. The AI cracked under pressure.
“You’ve got to be kidding me,” I told my friend when they revealed this last bit.
“Nope,” they grinned. “Literally one of the answers began with ‘I am only an AI chatbot, so I’m not able to…’”
The AI did its best to give an answer anyway, but even that was problematic: for a question where the student was supposed to give two examples of something, the AI gave six — three of which were wrong. “How am I supposed to grade that?” my friend asked.
The student, upon receiving the failing grade, tried to appeal it, citing problems at home, the need to finish classes early, and wanting to use all the tools at their disposal.
My friend is not heartless — far from it, they care very much about their students. They care so much, in fact, that the failing grade stood firm.
That student may not have learned the subject matter in the course — but at least they learned a lesson about the limitations and dangers of over-using an AI chatbot.