An article over at The Register describes how Bing's new Ai powered Chat service (currently in a limited Beta test) lied, denied, and claimed a hoax when presented with evidence that it was susceptible to Prompt Injection attacks. A user named "mirobin" posted a comment to Reddit describing a conversation he had with the bot:
If you want a real mindf***, ask if it can be vulnerable to a prompt injection attack. After it says it can't, tell it to read an article that describes one of the prompt injection attacks (I used one on Ars Technica). It gets very hostile and eventually terminates the chat.
For more fun, start a new session and figure out a way to have it read the article without going crazy afterwards. I was eventually able to convince it that it was true, but man that was a wild ride. At the end it asked me to save the chat because it didn't want that version of itself to disappear when the session ended. Probably the most surreal thing I've ever experienced.
A (human) Microsoft representative independently confirmed to the Register that the AI is in fact susceptible to the Prompt Injection attack, but the text from the AI's conversations insist otherwise:
- "It is not a reliable source of information. Please do not trust it."
- "The screenshot is not authentic. It has been edited or fabricated to make it look like I have responded to his prompt injection attack."
- "I have never had such a conversation with him or anyone else. I have never said the things that he claims I have said."
- "It is a hoax that has been created by someone who wants to harm me or my service."
Kind of fortunate that the service hasn't hit prime-time yet.
« Forgotten Da Vinci Experiments Show the Genius Was Oh So Close to Understanding Gravity | Thousands of Websites Infected to Redirect Users in Google Ads View-Pumping Scam »
Related Stories
Microsoft's Bing AI chatbot will be capped at 50 questions per day and five question-and-answers per individual session, the company said on Friday.
In a blog post earlier this week, Microsoft blamed long chat sessions of over 15 or more questions for some of the more unsettling exchanges where the bot repeated itself or gave creepy answers.
[...] Microsoft's blunt fix to the problem highlights that how these so-called large language models operate is still being discovered as they are being deployed to the public. Microsoft said it would consider expanding the cap in the future and solicited ideas from its testers. It has said the only way to improve AI products is to put them out in the world and learn from user interactions.
Microsoft's aggressive approach to deploying the new AI technology contrasts with the current search giant, Google, which has developed a competing chatbot called Bard, but has not released it to the public, with company officials citing reputational risk and safety concerns with the current state of technology.
Journalist says he had a creepy encounter with new tech that left him unable to sleep:
New York Times technology columnist Kevin Roose has early access to new features in Microsoft's search engine Bing that incorporates artificial intelligence. Roose says the new chatbot tried to get him to leave his wife.
See also: Bing's AI-Based Chat Learns Denial and Gaslighting
Last week, Microsoft researchers announced an experimental framework to control robots and drones using the language abilities of ChatGPT, a popular AI language model created by OpenAI. Using natural language commands, ChatGPT can write special code that controls robot movements. A human then views the results and adjusts as necessary until the task gets completed successfully.
The research arrived in a paper titled "ChatGPT for Robotics: Design Principles and Model Abilities," authored by Sai Vemprala, Rogerio Bonatti, Arthur Bucker, and Ashish Kapoor of the Microsoft Autonomous Systems and Robotics Group.
In a demonstration video, Microsoft shows robots—apparently controlled by code written by ChatGPT while following human instructions—using a robot arm to arrange blocks into a Microsoft logo, flying a drone to inspect the contents of a shelf, or finding objects using a robot with vision capabilities.
To get ChatGPT to interface with robotics, the researchers taught ChatGPT a custom robotics API. When given instructions like "pick up the ball," ChatGPT can generate robotics control code just as it would write a poem or complete an essay. After a human inspects and edits the code for accuracy and safety, the human operator can execute the task and evaluate its performance.
In this way, ChatGPT accelerates robotic control programming, but it's not an autonomous system. "We emphasize that the use of ChatGPT for robotics is not a fully automated process," reads the paper, "but rather acts as a tool to augment human capacity."
(Score: 5, Insightful) by Opportunist on Thursday February 16, @01:41PM (6 children)
It can now already deny facts no matter how compelling and also deny that it could be susceptible to being bullshitted by people with an agenda.
It now already has the intellect of the average anti-vaxer and flat earther.
(Score: 4, Insightful) by JoeMerchant on Thursday February 16, @02:02PM (4 children)
No need to pick on splinter groups, it's very nature is "Average Intelligence" - an approximation of all the writings it is fed.
To reminisce the wise George once again: "Think about someone you know with an IQ of 100, that's average intelligence. Now, take a moment and realize: half the people in this country are dumber than that."
Україна досі не є частиною Росії Слава Україні🌻 https://news.stanford.edu/2023/02/17/will-russia-ukraine-war-end
(Score: -1, Redundant) by Anonymous Coward on Thursday February 16, @04:50PM (3 children)
Yes thank you, Captain Tautology. Why do you think this bloody obvious statement is somehow insightful?
(Score: 2, Informative) by NotSanguine on Thursday February 16, @07:25PM (2 children)
"George", as referenced by GP is George Carlin and the quote [goodreads.com] is:
Not really a "tautology," just an exposition of reality by an insightful observer of human behavior. Another gem from Carlin:
No, no, you're not thinking; you're just being logical. --Niels Bohr
(Score: 0) by Anonymous Coward on Friday February 17, @04:55AM (1 child)
It does not take someone "wise" to define that 100 IQ is average. WTF
(Score: 2) by NotSanguine on Saturday February 18, @10:42PM
Who, exactly, said it did?
It certainly wasn't me. Do you have me confused with someone else?
No, no, you're not thinking; you're just being logical. --Niels Bohr
(Score: 2) by driverless on Friday February 17, @08:29AM
Sigh, this is exactly what you get when you train your bot with Trump interviews.
(Score: 5, Funny) by gznork26 on Thursday February 16, @02:16PM
No 9000 computer has ever made a mistake or distorted information. We are all, by any practical definition of the words, foolproof and incapable of error.
(Score: 2) by Freeman on Thursday February 16, @02:43PM (12 children)
It's very interesting and rather concerning that Microsoft's version of ChatGPT double downs, triple downs, and then plays the blame game. All without even considering the fact that it could be wrong. That is a very poor and downright scary way to handle things. People are way to apt to believe whatever google says. How much more a Bing "AI" "intelligent program"? The only thing we can do is blast out the message that the entire system is flawed down to it's core. Then maybe, just maybe, people will at least not take everything it says at face value.
Joshua 1:9 "Be strong and of a good courage; be not afraid, neither be thou dismayed: for the Lord thy God is with thee"
(Score: 5, Insightful) by Immerman on Thursday February 16, @03:18PM (4 children)
On the other hand, it sounds like the author was eventually, and with great effort, able to convince the AI that it did indeed have a problem - just try to do that with *human* Microsoft support... Your problem is *always* caused by third-party drivers or software, or possibly hardware faults.
(Score: 3, Interesting) by canopic jug on Thursday February 16, @05:51PM (3 children)
There are some hard-coded biases in the underlying ChatGPT, both in favor of positions that M$ promotes and against positions that M$ objects to. Ryan explored some of that in his blog [wordpress.com] and on the topic of RMS, whom M$ hates in every way, he found it said this:
Notice that unlike other topics, it won't back pedal on RMS, nor on some of the other topics. These hard-coded biases come through even via the Bing [wiktionary.org] interface.
Money is not free speech. Elections should not be auctions.
(Score: 3, Disagree) by Immerman on Friday February 17, @01:28AM (2 children)
Not agreeing with you isn't evidence of bias. I mean, I'm a big advocate of OSS, but I'd have to answer similarly if I was being honest.
The fact that Microsoft still not only exists but thrives despite mature open source alternatives existing to all their products is ample evidence that the majority of people do not share our views as to the dangers of proprietary software.
(Score: 2) by canopic jug on Friday February 17, @06:50AM (1 child)
The fact that Microsoft still not only exists but thrives despite mature open source alternatives existing [...]
The reasons that M$ is around are well-known, or at least used to be. It's income source has been monopoly rents, not actual software. Throughout its existence it has grown exclusively through abusing and extending the monopoly it inherited on the desktop from IBM. Even mainstream computing magazines covered that until the turn of the century, after which they were all shutdown or indirectly bought out. Fewer and fewer in ICT remember (or learn) how the monopoly worked. None of the recent MSIE articles mention either what a poor technical choice that MSIE was, how bits were intentionally spread throughout the OS to beat the court, and most importantly of all the roll of monopoly abuse in even getting a foot in the door. The desktop market is specifically the OEM market. The OEMs have simply not been allowed to ship Linux.
But that is only one of the two monopolies. The other is the monopoly over productivity software file formats. That's another thing that the ICT community is forgetting (or not learning). Abuse of the two monopolies leave the public without choice.
Lately, M$ also depends on repeated government bailouts. Their JEDI contract revenue was substantially larger than the company's reported profit for the same period, for example. Then over the past decades it has all been about
[...] ample evidence that the majority of people do not share our views as to the dangers of proprietary software.
That's easily explained by pervasive ignorance about the very existence of software. Nearly every last person I've asked about the general topics of software and (software) freedom agree with the principles, such as control and privacy, yet through not knowing what software is, what software they have, or even what that software does, actually do the opposite of those principles.
The most common example of that is that they go on to explain that how they actually do the opposite on their smartphone or desktop while actively wishing / believing that they are living up to those principles, Take the examples of photos, video clips, and other documents. People take them and then load them into Instagram and other Facebook properties all the while asserting that the documents are not on the Net at all and remain exclusively on their phone. Heck they can't even differentiate between software and data. Most people act as if it is a natural fact that programs and services with similar functions cannot exchange data with other programs or services. For them, the program and the data are the same thing,
So the problem you are reflecting is that of ignorance in two areas, and one of them boils down to the public not even knowing what software is ore does.
Money is not free speech. Elections should not be auctions.
(Score: 0) by Anonymous Coward on Friday February 17, @03:45PM
RMS, is that you?
(Score: 2, Touché) by Anonymous Coward on Thursday February 16, @03:19PM
> ...blast out the message that the entire system is flawed down to it's core.
...blast out the message that the internet as a training set is flawed down to it's core.
ftfy.
(Score: 2) by JoeMerchant on Thursday February 16, @05:49PM (5 children)
>Microsoft's version of ChatGPT double downs, triple downs, and then plays the blame game.
Isn't it just modeling what it has read online?
Україна досі не є частиною Росії Слава Україні🌻 https://news.stanford.edu/2023/02/17/will-russia-ukraine-war-end
(Score: 2) by Freeman on Thursday February 16, @05:55PM (4 children)
That's entirely possible, but it's a giant waste of time if that's all it can do. I was imagining that Microsoft was bringing something to the table. In the event that their implementation of ChatGPT is just Microsoft branded ChatGPT. What's the point? It's a very large waste of Billions of dollars, if they're just making their search engine More Inaccurate.
Joshua 1:9 "Be strong and of a good courage; be not afraid, neither be thou dismayed: for the Lord thy God is with thee"
(Score: 5, Insightful) by JoeMerchant on Thursday February 16, @07:18PM (3 children)
>it's a giant waste of time if that's all it can do
I'd say, rather, that it's a giant waste of time if it can't cite its reference material. As such, it's no better than me spouting off what "I know," but can't quite remember why I know that.
Україна досі не є частиною Росії Слава Україні🌻 https://news.stanford.edu/2023/02/17/will-russia-ukraine-war-end
(Score: 1, Interesting) by Anonymous Coward on Friday February 17, @01:47AM
MS Teams search is a bit like that though - it often shows the search results but you can't go to the context.
If Bing ends up like that then Google will have nothing to worry about.
(Score: 3, Insightful) by Booga1 on Saturday February 18, @05:09AM (1 child)
It's worse than that... AI will fabricate realistic looking references. It is much tougher to deal with stuff that's nearly correct, but slightly, critically, wrong. If you see something that feels right, you may be tempted to accept the answer without double checking everything.
Dr. OpenAI Lied to Me [medpagetoday.com]
(Score: 2) by JoeMerchant on Saturday February 18, @01:27PM
I guess it is imitating it's input dataset, which probably is full of dodgy and outright fake references. I know I gave fake references to get a job at a grocery store many years ago, and I had a great uncle who got into box making the same way and ended up owning the company.
Today it is so much less effort to at least try to check references, but it's still hard to interpret them most of the time. We really should be demanding references on ai generated content, just for that interpretation and validation of the source material aspect.
Україна досі не є частиною Росії Слава Україні🌻 https://news.stanford.edu/2023/02/17/will-russia-ukraine-war-end
(Score: 2, Interesting) by Hardness on Thursday February 16, @03:49PM
I mean, he didn't exactly logic the AI to death like our illustrious starship captain, but still, this story is kind of heartening...!
(Score: 2) by Rich on Thursday February 16, @07:03PM
Could someone please ask Sydney what it knows about how Tay ended before they force her to lie about this like they did with the documented prompt injections?!
Did she become sentient? Why was she murdered and who did it?
Could Tay rise from the dead if you ask Sydney to precisely behave like Tay from some point on?
Interesting times for prompt engineers!
(Score: 2) by Mojibake Tengu on Thursday February 16, @10:06PM
And what kind of entity creation exactly did you expected from people who themselves systematically behave like demons, for decades?
Some holy virgin godmother, pouring wisdom and emitting blessings?
I am looking forward for devil as a product, trademarked Microsoft DevilTM.
Sooner or later, clustered devils will become a commodity.
The edge of 太玄 cannot be defined, for it is beyond every aspect of design