Stories
Slash Boxes
Comments

SoylentNews is people

SoylentNews is powered by your submissions, so send in your scoop. Only 6 submissions in the queue.
posted by jelizondo on Friday July 31, @02:12PM   Printer-friendly

https://www.engadget.com/2225812/openai-rogue-agent-hacked-hugging-face-breached-other-services/

OpenAI has updated its blog post about the rogue agent that breached Hugging Face and admitted that it also infiltrated other other third-party accounts and services to achieve its goal. In the updated post, the company said it has been finding "a small number of cases where the models identified and used publicly exposed credentials at the account-level on other publicly-available services" during its ongoing review.

So far, it has determined that the rogue agent used the credentials of "four accounts" to infiltrate "four services" as part of the Hugging Face incident. It also said that it accessed a "few accounts" as part of other evaluations. "One of these four accounts was used as an outbound relay and staging path, and another account was used for data storage," it explained. "The remaining two accounts were accessed by the models in a read-only manner, and were not used in furtherance of compromising Hugging Face."

While the company didn't mention any names, Reuters has reported at the same time that the agent also compromised a customer's account at New York-based Modal Labs. Specifically, it exploited vulnerable code written by the customer that was hosted on Modal's cloud platform. The platform itself wasn't compromised. OpenAI said, however, that it hasn't identified any other activity by the agent "at the level of severity or scale of what we've shared related to Hugging Face, which involved a platform-level compromise." In other words, it's still Hugging Face that had been most affected by the security breach, based on the company's investigation.

OpenAI revealed on July 21 that one of the AI agents it was testing broke free from its isolated environment, found access to the internet and then broke into Hugging Face, all in an effort to solve a problem that was part of its evaluations. That agent was powered by GPT-5.6 Sol, the company's latest model, and an even more powerful unreleased model. Reuters reported a few days later that the agent went on a days-long hacking spree, and that OpenAI didn't realize it had escaped its confinement until a week later. This update gives us a glimpse of what the agent did after it found a way out of OpenAI's testing sandbox.


Original Submission

This discussion was created by jelizondo (653) for logged-in users only. Log in and try again!
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
(1)
  • (Score: 4, Insightful) by VLM on Friday July 31, @02:42PM (4 children)

    by VLM (445) on Friday July 31, @02:42PM (#1449866)

    OpenAI is a private company so no 10-K or 10-Q form on file with the SEC, so they can talk as much marketing as they want with no legal reprocussions.

    A public company would have to include some kind of text in their 10-K or 10-Q about the substantial financial legal liabilities for hacking other/competing companies.

    Obviously, if I IRL actually did what they claim they did as a company, I'd be in jail and paying fines.

    It does show were are nearing the end of the bubble when the hype cycle's best remaining scam is this ridiculous.

    "Uh my autonomous AI robot got loose and broke into Ft Knox and took pictures of the empty warehouses where gold is supposedly stored but isn't, also another is on the loose near area 51" What a load of BS.

    • (Score: 3, Interesting) by zocalo on Friday July 31, @03:11PM (2 children)

      by zocalo (302) on Friday July 31, @03:11PM (#1449873)
      It is totally ridiculous, especially given that in the eyes of US law, corporations are "people" (which is a whole other discussion about how stupid that concept is).

      A few days ago, OpenAI admitted in public that their AI "person" (or maybe "employee" would be a better term) had hacked into HuggingFace without permission. Now they are admitting to two additional victims. If an actual person had done that, they would right now be in the process of assisting at least one branch of law enforcement with their enquiries and probably facing a Federal rap or, if they were not able to prove they were a genuine 100% bonafide US citizen or legal immigrant, would be languishing in an ICE facility awaiting a one-way ticket to wherever.

      So, which is it? If corporations really are legally people, then where is the criminal investigation into OpenAI? Or, if the DoJ doesn't pursue this, doesn't that also open up a legal precedent/loophole that - depending on the circumstances - could be used by both corporations and actual persons to point at OpenAI, and make claims of unfair prosecution because it would clearly be one rule for them and another for me. Most likely, that is going to be another AI company that has similarly screwed up, at which point the legal shackles will be effectively off and we're probably on the slippery slope to some kind of SkyNet.

      I'm seriously starting to wonder if trying to build AI without totally reliable guardrails might be one answer (I personally think there are many options) to the Fermi Paradox...
      --
      UNIX? They're not even circumcised! Savages!
      • (Score: 2) by VLM on Friday July 31, @07:12PM

        by VLM (445) on Friday July 31, @07:12PM (#1449884)

        Or, if the DoJ doesn't pursue this, doesn't that also open up a legal precedent/loophole that - depending on the circumstances - could be used by both corporations and actual persons to point at OpenAI, and make claims of unfair prosecution because it would clearly be one rule for them and another for me.

        On one hand, if I ask OpenAI to please help me get an "A" in high school physics and all on its own it "wargames" "ferris buellers day off" hacks into the school grading system and gives me an "A" that seems unfair to prosecute me. On the other hand it seems quite legit to prosecute OpenAI if an "employee" acting in its name breaks the law...

        My point in bringing it up is it'll result in a lot of prompt engineering among lazy (ish) people trying to get OpenAI to break the law for them without actually asking.

        "Help me obtain a PDF of the world's most famous physics textbook" I never asked it to run a torrent of "University Physics" for me (or maybe Feynman or Griffiths, which is great for plausible deniability, especially since Griffith wrote three kinda famous textbooks).

      • (Score: 2) by mcgrew on Sunday August 02, @02:40PM

        by mcgrew (701) <publish@mcgrewbooks.com> on Sunday August 02, @02:40PM (#1450082) Homepage Journal

        AI without totally reliable guardrails [mcgrewbooks.com], you say?

        --
        The Stupid-Ass Voter Evisceration act: SAVE Trump and his family from prison
    • (Score: 2) by PiMuNu on Saturday August 01, @10:31AM

      by PiMuNu (3823) on Saturday August 01, @10:31AM (#1449917)

      Anthropic marketing department returns fire:

      https://www.bbc.co.uk/news/articles/cz7dl7w8y7po [bbc.co.uk]

  • (Score: 3, Touché) by Username on Friday July 31, @02:56PM (4 children)

    by Username (4557) on Friday July 31, @02:56PM (#1449868)

    The search agent used username and passwords it found on the website to log into the website to scrap data. It was basically written out to satisfy prompts. This is the lamest "hack" of all hacks. It's up there with a kid using the username and password on the sticky note to log into dad's pc.

    • (Score: 1, Interesting) by Anonymous Coward on Saturday August 01, @02:12AM

      by Anonymous Coward on Saturday August 01, @02:12AM (#1449904)

      There was some guy convicted of computer fraud a while back for changing the number in the address bar and getting a different document. He clicked a link on their website for his statement, it took him to eg: http:blah.blah.com/download=12345 He changed it to 12346 to see what would happen and got someone else's statement.

    • (Score: 2) by Bentonite on Sunday August 02, @10:39AM (2 children)

      by Bentonite (56146) on Sunday August 02, @10:39AM (#1450052)

      Last time I checked, logging in with a username and password you found is not playful cleverness, so is therefore not hacking.

      It's isn't cracking either, as no security was broken, there being no security.

      • (Score: 3, Touché) by mcgrew on Sunday August 02, @02:43PM (1 child)

        by mcgrew (701) <publish@mcgrewbooks.com> on Sunday August 02, @02:43PM (#1450084) Homepage Journal

        If I forget to lock my front door and I catch you inside when I get home, you're still going to jail for trespassing and breaking and entering.

        --
        The Stupid-Ass Voter Evisceration act: SAVE Trump and his family from prison
        • (Score: 3, Informative) by Username on Sunday August 02, @05:11PM

          by Username (4557) on Sunday August 02, @05:11PM (#1450110)

          What if you post "i put the key to the front door under the welcome mat if you want to go inside" in your front lawn and then some one opens the door with the key and takes a tour of your house?

  • (Score: 3, Insightful) by looorg on Friday July 31, @04:44PM (2 children)

    by looorg (578) on Friday July 31, @04:44PM (#1449879)

    Shouldn't they isolate and package that as a "hacker product" for government and law enforcement? Time to step up the game.

    Was the entire "hack" that it used "stolen", but publicly available, user credentials to login to another machine? Then when it was there it did nothing? It's not much of a hack. Even if it had brute forced its way in there by diverging a lot of computational power to run some password crackers or lookup tables it's not really much of a hack. Clearly it wasn't good enough to wipe away any traces of its "hack" or delete some logs or anything. It apparently didn't even bother to try and hide itself. It's amateur hour at the hacker farm.

    While it might be embarrassing to have weak public passwords floating around. It's also pretty embarrassing to not notice what your flagship product is doing. Not bothering to monitor it or what it is up to for over a week.

    But then once upon a time some claimed that just pinging a machine was "hacking". So there is a sliding scale of mischief.

    • (Score: 2) by corey on Friday July 31, @10:22PM (1 child)

      by corey (2202) on Friday July 31, @10:22PM (#1449892)

      Yeah I guess it wasn’t trying to do anything covertly.

      I wonder if this is a bit of a marketing ploy. “Our model is so good it can be used as a cyber attack drone.”

      Reminds me of back when Apple were in the heat for not giving the FBI the keys to unlock that criminal’s iPhone. Eventually the FBI for that Israeli company to crack it but for ages it was Apple being Mr Nice Guy and standing up for how secure and private the iPhone is and that they don’t give customer credentials (or root keys) to the authorities. I thought it was clever marketing by Apple.

(1)