If you feed America's most important legal document—the US Constitution—into a tool designed to detect text written by AI models like ChatGPT, it will tell you that the document was almost certainly written by AI. But unless James Madison was a time traveler, that can't be the case. Why do AI writing detection tools give false positives? We spoke to several experts—and the creator of AI writing detector GPTZero—to find out.
[...] In machine learning, perplexity is a measurement of how much a piece of text deviates from what an AI model has learned during its training. As Dr. Margaret Mitchell of AI company Hugging Face told Ars, "Perplexity is a function of 'how surprising is this language based on what I've seen?'"
So the thinking behind measuring perplexity is that when they're writing text, AI models like ChatGPT will naturally reach for what they know best, which comes from their training data. The closer the output is to the training data, the lower the perplexity rating. Humans are much more chaotic writers—or at least that's the theory—but humans can write with low perplexity, too, especially when imitating a formal style used in law or certain types of academic writing. Also, many of the phrases we use are surprisingly common.
Let's say we're guessing the next word in the phrase "I'd like a cup of _____." Most people would fill in the blank with "water," "coffee," or "tea." A language model trained on a lot of English text would do the same because those phrases occur frequently in English writing. The perplexity of any of those three results would be quite low because the prediction is fairly certain.
Related Stories
In a notable shift toward sanctioned use of AI in schools, some educators in grades 3–12 are now using a ChatGPT-powered grading tool called Writable, reports Axios. The tool, acquired last summer by Houghton Mifflin Harcourt, is designed to streamline the grading process, potentially offering time-saving benefits for teachers. But is it a good idea to outsource critical feedback to a machine?
[...]
"Make feedback more actionable with AI suggestions delivered to teachers as the writing happens," Writable promises on its AI website. "Target specific areas for improvement with powerful, rubric-aligned comments, and save grading time with AI-generated draft scores." The service also provides AI-written writing-prompt suggestions: "Input any topic and instantly receive unique prompts that engage students and are tailored to your classroom needs."
[...]
The reliance on AI for grading will likely have drawbacks. Automated grading might encourage some educators to take shortcuts, diminishing the value of personalized feedback. Over time, the augmentation from AI may allow teachers to be less familiar with the material they are teaching. The use of cloud-based AI tools may have privacy implications for teachers and students. Also, ChatGPT isn't a perfect analyst. It can get things wrong and potentially confabulate (make up) false information, possibly misinterpret a student's work, or provide erroneous information in lesson plans.
[...]
there's a divide among parents regarding the use of AI in evaluating students' academic performance. A recent poll of parents revealed mixed opinions, with nearly half of the respondents open to the idea of AI-assisted grading.As the generative AI craze permeates every space, it's no surprise that Writable isn't the only AI-powered grading tool on the market. Others include Crowdmark, Gradescope, and EssayGrader. McGraw Hill is reportedly developing similar technology aimed at enhancing teacher assessment and feedback.
On Wednesday, Reuters reported that OpenAI is working on a plan to restructure its core business into a for-profit benefit corporation, moving away from control by its nonprofit board. The shift marks a dramatic change for the AI company behind ChatGPT, potentially making it more attractive to investors while raising questions about its commitment to sharing the benefits of advanced AI with "all of humanity," as written in its charter.
A for-profit benefit corporation is a legal structure that allows companies to pursue both financial profits and social or environmental goals, ostensibly balancing shareholder interests with a broader mission to benefit society. It's an approach taken by some of OpenAI's competitors, such as Anthropic and Elon Musk's xAI.
[...] Bloomberg reports that OpenAI is discussing giving Altman a 7 percent stake, though the exact details are still under negotiation. This represents a departure from Altman's previous stance of not taking equity in the company, which he had maintained was in line with OpenAI's mission to benefit humanity rather than individuals.
[...] The proposed restructuring also aims to remove the cap on returns for investors, potentially making OpenAI more appealing to venture capitalists and other financial backers. Microsoft, which has invested billions in OpenAI, stands to benefit from this change, as it could see increased returns on its investment if OpenAI's value continues to rise.
(Score: 3, Touché) by legont on Wednesday July 19 2023, @02:09AM (8 children)
There is not much difference between an AI and a psycho,
"Wealth is the relentless enemy of understanding" - John Kenneth Galbraith.
(Score: 5, Funny) by Anonymous Coward on Wednesday July 19 2023, @02:37AM (5 children)
a psycho like somebody who ends their sentence with a comma?
(Score: 1, Touché) by Anonymous Coward on Wednesday July 19 2023, @05:47AM (4 children)
No, more like one who forgets to capitalize the first letter of a sentence,
(Score: 1, Touché) by Anonymous Coward on Wednesday July 19 2023, @06:03AM (3 children)
joke's on you; it wasn't a sentence to begin with
(Score: 2, Funny) by Anonymous Coward on Wednesday July 19 2023, @06:17AM (2 children)
...and you must be postmenopausal.
(Score: 1) by khallow on Wednesday July 19 2023, @12:34PM
(Score: 2, Touché) by Anonymous Coward on Wednesday July 19 2023, @04:03PM
Parent was not off-topic. It was indicating that GP poster had lost his period.
If you don't get the joke, don't think you are qualified to moderate it.
(Score: 4, Insightful) by Freeman on Wednesday July 19 2023, @03:20PM (1 child)
Actually, I posit that a psycho is less disturbing than an AI. A psycho usually is at the least, self-interested and people can relate to that. An AI can't feel and literally will only do what it's programmed to do. An AI programmed by humans better be on a short leash.
Joshua 1:9 "Be strong and of a good courage; be not afraid, neither be thou dismayed: for the Lord thy God is with thee"
(Score: 3, Interesting) by legont on Thursday July 20 2023, @01:30PM
The biggest issue with modern AI is that they are not programmed per se. They are trained on an unknown data set. Unknown because sets are too big to be known.
One may say that a human is trained similarly. Yes, but most humans have mamas and friends who find time to stimulate or punish every learned action. Those "teachers" are mostly not psycho. AI luck such a back loop. We can, off course, fix it but it would require generations of training until moral AI are grown and could be left by themselves to train new generations. That's assuming exponential smartness of AI would stop or slow to a reasonable rate.
"Wealth is the relentless enemy of understanding" - John Kenneth Galbraith.
(Score: 2) by MostCynical on Wednesday July 19 2023, @02:34AM (2 children)
has any AI been trained on English-as-spoken 300 to 200 years ago?
if not, how surprising is it that language from 'back then' doesn't match current style?
Few people, even even erudite people, writing for fun, use multiple clauses in a sentence, and even simple predicate clauses are almost defunct.
clear and simple paragraphs are written very differently to how they would be written now..
e.g.
"I guess once you start doubting, there's no end to it." -Batou, Ghost in the Shell: Stand Alone Complex
(Score: 5, Insightful) by ikanreed on Wednesday July 19 2023, @03:45AM
You have it exactly backwards. It's not oddity that its hitting on. It's commonality.
These detectors are neural nets and LLMs themselves. Which typically means they hone in on local minima for their detection job. One might imagine one of the local minima could very easily be "made of fragments repeated often on the internet and documents". And just guess what one of the most cited documents of all time is?
(Score: 2) by Freeman on Wednesday July 19 2023, @03:18PM
No, that's what "English" majors do. Think of redneck hill billy guy that never graduated elementary. He can probably string together sentences and spell better than most "English-as-spoken" back then.
Here's an account by a former member of the House of Representatives of the United States of America, less than 200 years ago.
https://www.gutenberg.org/files/37925/37925-h/37925-h.htm [gutenberg.org]
Joshua 1:9 "Be strong and of a good courage; be not afraid, neither be thou dismayed: for the Lord thy God is with thee"
(Score: 3, Insightful) by darkfeline on Wednesday July 19 2023, @05:00AM (1 child)
The whole idea of AI detectors is bogus. How do you tell the difference between a copy of the US Constitution written by AI and an identical copy of the US Constitution written by monkeys banging on typewriters? The end result has effectively zero artifacts from the production process (as digital text).
Join the SDF Public Access UNIX System today!
(Score: 3, Funny) by Anonymous Coward on Wednesday July 19 2023, @05:50AM
In the monkey-written one you'll frequently find the word "banana".
(Score: 2) by Mojibake Tengu on Wednesday July 19 2023, @05:54AM
What about Establishing a Congress 2.0 assembled from all existing AI implementations, as a new Law prescribing Entity?
As humans already proved themselves much more chaotic than machines, a complete redesign of Legal System by machines would be a significant improvement of civilization, for the benefit of both Humans and AI Language Models.
Rust programming language offends both my Intelligence and my Spirit.
(Score: 2) by ledow on Wednesday July 19 2023, @07:11AM (3 children)
"AI merely a statistical model trying to blindly extrapolate from its training data alone, with zero powers of inference."
News at 11.
(Score: 2, Touché) by shrewdsheep on Wednesday July 19 2023, @08:32AM (2 children)
In all fairness, that perfectly describes the majority of humans, too.
(Score: 3, Insightful) by ledow on Thursday July 20 2023, @07:04AM (1 child)
It absolutely doesn't.
Inference is literally still missing from AI, always has been, and there's no evidence that the human brain is operating statistically at all. Hell, people don't operate statistically, they barely understand what it would mean to do so.
In fact, the majority of humans don't act in any predictable, rational or logical way at all.
(Score: 0) by Anonymous Coward on Thursday July 20 2023, @07:07AM
Statistical, then?
(Score: 2) by looorg on Wednesday July 19 2023, @09:22AM
> But unless James Madison was a time traveler, that can't be the case.
Have they explored this path further? It could explain a lot of things. No stone left unturned!
(Score: 2) by istartedi on Wednesday July 19 2023, @09:23AM (3 children)
Since this "perplexity" appears to be a measure of unfamiliarity, it's most likely detecting archaic language.
If you fed it some slang used in only a few localities, I gather it would come to the same conclusion.
Appended to the end of comments you post. Max: 120 chars.
(Score: 4, Informative) by requerdanos on Wednesday July 19 2023, @01:59PM (1 child)
Yes, but perplexity (and burstiness) are signs of human, not AI, authorship, as I understood the article. The US constitution probably looks ordinary, normal, and boring -- low perplexity -- because it's in the training set.
(Score: 3, Funny) by istartedi on Wednesday July 19 2023, @07:06PM
What are you doing reading the article? Are you a bot? /sarcasm.
Appended to the end of comments you post. Max: 120 chars.
(Score: 4, Interesting) by aafcac on Wednesday July 19 2023, @04:52PM
Possibly, but I'd be curious if the same AI says the same if other documents created through group negotiation. There's a lot of ideas in the constitution from disparate sources like the writing of AI.