Stories
Slash Boxes
Comments

SoylentNews is people

SoylentNews is powered by your submissions, so send in your scoop. Only 10 submissions in the queue.
posted by hubie on Tuesday September 12 2023, @07:31AM   Printer-friendly
from the burn-it-all-down dept.

https://arstechnica.com/information-technology/2023/09/openai-admits-that-ai-writing-detectors-dont-work/

Last week, OpenAI published tips for educators in a promotional blog post that shows how some teachers are using ChatGPT as an educational aid, along with suggested prompts to get started. In a related FAQ, they also officially admit what we already know: AI writing detectors don't work, despite frequently being used to punish students with false positives.

In a section of the FAQ titled "Do AI detectors work?", OpenAI writes, "In short, no. While some (including OpenAI) have released tools that purport to detect AI-generated content, none of these have proven to reliably distinguish between AI-generated and human-generated content."

In July, we covered in depth why AI writing detectors such as GPTZero don't work, with experts calling them "mostly snake oil."
[...]
That same month, OpenAI discontinued its AI Classifier, which was an experimental tool designed to detect AI-written text. It had an abysmal 26 percent accuracy rate.


Original Submission

 
This discussion was created by hubie (1068) for logged-in users only, but now has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 2) by sjames on Tuesday September 12 2023, @01:57PM (4 children)

    by sjames (2882) on Tuesday September 12 2023, @01:57PM (#1324202) Journal

    This is just an extension of the problem of workplace metrics in a technical setting. Bosses love their metrics even though they produce little meaningful information. But poll the others in the department and they easily identify the key players and deadwood, often based on things the metrics don't look at.

    Like the star who produces 3 lines of code a day but solves 5 other people's problems that were blocking progress. Or the deadwood who cranks out 1000 mostly useless or buggy lines of code.

    The teachers may have to, God forbid, talk to the students one on one to evaluate them. It need not be in person, video or voice will do.

    Starting Score:    1  point
    Karma-Bonus Modifier   +1  

    Total Score:   2  
  • (Score: 2) by VLM on Tuesday September 12 2023, @02:24PM (3 children)

    by VLM (445) on Tuesday September 12 2023, @02:24PM (#1324207)

    The teachers may have to, God forbid, talk to the students one on one to evaluate them. It need not be in person, video or voice will do.

    Grade entirely on presentation performance, or extensive in class quiz/test.

    Its interesting to think about how presentations used to be graded entirely on demonstration of sophistry, but in the future, presentations might be graded mostly on demonstrated skills and knowledge.

    There's little point anymore in grading on take home homework or written at home essays.

    Interesting idea to think about: When identity politics was weak in the old days, we needed a "demographic blind" way to stack rank and hire the correct people. But we're entering a new era of identity politics where all that matters is the color of a person's skin, their sex act preferences, etc, so if we're going to base stack ranking and hiring on what amounts to genetics, do we really need a complicated and expensive higher education system? HR doesn't need an "education" section on the resume anymore, all they need is a brown paper bag to compare to skin color and a self identification of gender for their hiring process. The whole thing of "cheating" in higher ed might be a moot point soon enough.

    • (Score: 2) by aafcac on Tuesday September 12 2023, @04:13PM (2 children)

      by aafcac (17646) on Tuesday September 12 2023, @04:13PM (#1324244)

      The point of grading homework is that if you don't do it, nobody will do the homework and there isn't time to get the practice in during school. That's not to say that it needs to be a lot of work, but a bit every day does make a difference, rather than allowing students to just cram for an exam or presentation.

      It also allows teachers to see what's going on before they get to the exam and for students to know whether to ask questions or not.

      • (Score: 2) by VLM on Tuesday September 12 2023, @07:47PM (1 child)

        by VLM (445) on Tuesday September 12 2023, @07:47PM (#1324293)

        I got out of ChemEng right before the enforced problem class where we would have sat in a room and worked chem eng problems as a class.

        I would imagine it would be possible to have "enforced pair programming" in CS classes at a scheduled time as a "lab". We did that in middle and high school, never post-high school but it should be possible.

        The "lab" concept is interesting, hard science has been doing that awhile and it should be possible to enforce a "lab" concept for humanities essay writing classes and similar.

        • (Score: 2) by aafcac on Wednesday September 13 2023, @02:20AM

          by aafcac (17646) on Wednesday September 13 2023, @02:20AM (#1324349)

          The last time I took a science class we had a neat virtual lab to work with. I probably would've hated that aspect of science classes so much if I could get the relevant details when I needed them. I remember failing an assignment because I couldn't tell the fruit flies apart because they weren't labeled and I couldn't get the professor you explain what I was supposed to be looking for.