SoylentNews

Freeman [soylentnews.org] writes:

https://arstechnica.com/information-technology/2024/12/certain-names-make-chatgpt-grind-to-a-halt-and-we-know-why/ [arstechnica.com]

OpenAI's ChatGPT [arstechnica.com] is more than just an AI language model with a fancy interface. It's a system consisting of a stack of AI models and content filters that make sure its outputs don't embarrass OpenAI or get the company into legal trouble when its bot occasionally makes up potentially harmful facts [arstechnica.com] about people.

Recently, that reality made the news when people discovered [x.com] that the name "David Mayer" breaks ChatGPT. 404 Media also discovered [404media.co] that the names "Jonathan Zittrain" and "Jonathan Turley" caused ChatGPT to cut conversations short. And we know another name, likely the first, that started the practice last year: Brian Hood.
[...]
When asked about these names, ChatGPT responds with "I'm unable to produce a response" or "There was an error generating a response" before terminating the chat session, according to Ars' testing.
[...]
ChatGPT-breaking names found so far through a communal effort taking place on [social media [x.com]] and [Reddit].
[...]

• Brian Hood
• Jonathan Turley
• Jonathan Zittrain
• David Faber
• Guido Scorza

[...]
We first discovered that ChatGPT choked on the name "Brian Hood" in mid-2023 while writing about his defamation lawsuit [arstechnica.com]. In that lawsuit, the Australian mayor threatened to sue OpenAI after discovering ChatGPT falsely claimed he had been imprisoned for bribery when, in fact, he was a whistleblower who had exposed corporate misconduct.

The case was ultimately resolved in April 2023 when OpenAI agreed to filter out the false statements within Hood's 28-day ultimatum [arstechnica.com]. That is possibly when the first ChatGPT hard-coded name filter appeared.
[...]
The "David Mayer" block in particular (now resolved) presents additional questions, first posed on Reddit [reddit.com] on November 26, as multiple people share this name. Reddit users speculated about connections to David Mayer de Rothschild [wikipedia.org], though no evidence supports these theories.
[...]
Already, Scale AI prompt engineer Riley Goodside discovered how an attacker might interrupt a ChatGPT session using a visual prompt injection [x.com] of the name "David Mayer" rendered in a light, barely legible font embedded in an image. When ChatGPT sees the image (in this case, a math equation), it stops, but the user might not understand why.

The filter also means that it's likely that ChatGPT won't be able to answer questions about this article when browsing the web, such as through ChatGPT with Search [arstechnica.com].
[...]
These are still very early days in AI assistants, LLMs, and chatbots. Their use has opened up numerous opportunities and vulnerabilities that people are still probing daily.

Original Submission