SoylentNews is people
SoylentNews
Last week, OpenAI published tips for educators in a promotional blog post that shows how some teachers are using ChatGPT as an educational aid, along with suggested prompts to get started. In a related FAQ, they also officially admit what we already know: AI writing detectors don't work, despite frequently being used to punish students with false positives.
In a section of the FAQ titled "Do AI detectors work?", OpenAI writes, "In short, no. While some (including OpenAI) have released tools that purport to detect AI-generated content, none of these have proven to reliably distinguish between AI-generated and human-generated content."
In July, we covered in depth why AI writing detectors such as GPTZero don't work, with experts calling them "mostly snake oil."
[...]
That same month, OpenAI discontinued its AI Classifier, which was an experimental tool designed to detect AI-written text. It had an abysmal 26 percent accuracy rate.
(Score: 3, Insightful) by VLM on Tuesday September 12 2023, @02:58PM
I have a modest proposal which extends my remarks in another comment.
In my experience Fing around with the current marketing push of AI, its unimaginably good at distilling English prose into something smaller like an essay or definition. Its also unimaginably bad at anything it can't distill 1000:1 or that requires reasoning or counting or visuospatial talents or really anything other than eating lots of lit and spitting out a little lit.
If you notice carefully, examples of the "staggering power of AI" focus on stuff that is a summary of gigabytes of rehashed topics. There must be a million FizzBuzz online, so its pretty good at distilling a million of them into a working example. Or there must be a thousand essays both online and in books and whatever else AI eats, legally or illegally, about "Euthyphro vs Crito" by Socrates so it can upon request give you a great essay about E vs C or rephrase it into an epic rap battle or a sad country western song or whatever other output filter you'd like.
However, if you push the limits into realms never discussed, its lost. So AI give me an essay about some new meme from /b/ vs Phaedo's dialogue and its just lost. "So there's this green frog and an ancient greek philosopher and ..." Another good laugh is any mathematics. Or counting and for extra AI torture mix in ordinal vs cardinal numbers. In my experience AI is lost when doing topology beyond word-chopping definitions and similar rephrasing.
So essentially its a plagiarism machine. If its been talked about online or written in a book, it'll average together what it's seen and give the correct ish answer.
Two educational strategies:
Adjust the curriculum to focus on higher level topics. We're always going to have AI around to give definitions so no need to memorize them anyway. Its assumed if you're a math major you don't need to start the undergrads with 2+2=4. Well, apply this to the humanities and engineering and science. No fizzbuzz for you, you walk in first day of freshman CS and bam first day its database theory and filesystems and hard RTOS concurrency.
Adjust the curriculum to be research focused instead of stack ranking vocational candidates. Before continental drift was accepted, asking AI about it would get you canned anti-conspiracy theorist responses. However, hypothetically, if you assigned 25 undergrads to work with Dr Wegener a century ago or whatever, you could grade them on actual human work. The AI will refuse to work on it so this is perfect for grading humans. If you argued with an AI in 2017 about corona viruses originating from China the AI political filter would shut you down, the government filtering would shut you down, the anti-conspiracy theorist filters would shut you down, etc. (BTW that's how you know something is true in the 2020s LOL) but humans were indeed working on novel lab-created corona viruses, obviously, and students could be graded on attempting to help research. Not sure if throwing half the undergrad class into research would net speed things up or net slow things down. Remember some of those kids would have had their credentials in a couple years in the old days, so they'll be of help. Of course the deadweight will be deadweight, but that's been the case for all of human history so maybe no loss?
So, in summary, delete the lower level classes from higher education for the freshmen and sophomores, and toss all the juniors and seniors directly into cutting edge research.