The OpenAI rogue agent behind the Hugging Face hack accessed four accounts on four services, according to updated company disclosures about the intrusion.
One of those four accounts belonged to a Modal customer that had published an unauthenticated endpoint for running arbitrary code in a sandbox on the AI infrastructure provider, Hugging Face noted in its technical timeline and Modal later confirmed.
"We're aware a Modal customer published an unauthenticated endpoint that allowed anyone on the internet to use their sandboxes for code execution," Modal Chief Technology Officer Akshat Bubna told The Register. "This was used by the rogue agent. Modal's platform was not compromised in any way."
The other accounts included one used for data storage and two others "accessed by the models in a read-only manner, and were not used in furtherance of compromising Hugging Face," OpenAI disclosed on Tuesday.
"We'll continue to notify service owners directly, and have not seen evidence of broader impact to these providers or other accounts on their services," the AI giant added.
Also on Tuesday, we learned that the rogue agent broke out of its testing environment by exploiting zero-day vulnerabilities in JFrog's universal binary repository manager Artifactory.
While both OpenAI and Hugging Face's updates and timeline provide defenders with useful details about how the attack worked and what the agent did - not to mention a lesson in security-incident transparency - they fail to answer one major question: Who is legally responsible when AI agents attack?
"If a human employee intentionally conducted unauthorized access to third-party systems, it's a much more clear path forward," Gabrielle Hempel, security operations strategist at Exabeam, told The Register, adding that depending on the facts and jurisdiction, the person could face criminal charges.
"The company could also face scrutiny depending on whether the employee acted within the scope of their employment, whether appropriate controls existed, and whether the conduct was authorized, foreseeable, or preventable," Hempel said.
However, she added, the "important thing here" is that legal frameworks in both the US and UK have been designed around human decision makers - not AI systems. "Our laws generally know how to ask questions about things like human intent, organizational oversight, and corporate responsibility."
Autonomous AI agents hacking into companies remains uncharted legal territory, and Hempel said it's "too early to draw conclusions about liability in this case because there are so many unknowns."
AI systems aren't legal persons, so they don't share the same legal responsibilities as individuals and companies.
"Because of that, the questions become: Who designed the system? Who determined the objectives it pursued? What safeguards were implemented? What level of autonomy was considered acceptable? Were the resulting actions reasonably foreseeable, and were appropriate controls in place? These are going to be important questions as organizations deploy more autonomous AI systems," Hempel said.
It's highly unlikely that Hugging Face will sue OpenAI over the agentic intrusion, given the amount of very public collaboration between the two companies over the past couple of weeks, and the self-congratulatory celebration of the autonomous attack as a success story.
It also appears that this former worst-case scenario didn't dampen anyone's enthusiasm for setting advanced models loose (or at least unsupervised in a test environment), which means there are sure to be more agents-gone-wild attacks in the near future.
"The first part of the OpenAI/Hugging Face drama did not produce enough effect to impress investors who start losing their excitement over the AI hype, so the second part of the story is now unfolding," said Ilia Kolochenko, founder of application security company ImmuniWeb and a cybersecurity and data-protection lawyer.
"AI agents and LLM models tasked with security testing can, and almost certainly will, go rogue when security controls or safeguards are insufficient," Kolochenko told The Register. "Powerful LLMs are unpredictable by design and thus virtually uncontrollable by humans. Using frontier AI models for security testing might be extremely costly from the legal viewpoint."
Existing laws on both sides of the Atlantic likely hold the AI operator liable for any damages caused if an agent or AI system escapes its sandbox and breaches a third party. "Excuses like 'AI did it' do not currently exist in the eyes of the law, leaving AI vendors on the hook," he said, adding that this also holds true for end-users.
"Even if your security testing tool is powered by a third-party AI model, your company will be fully liable if something goes wrong," Kolochenko warned. "You may then file a lawsuit against the AI vendor that you used, but your chances of succeeding in the court of law are tiny due to countless contractual disclaimers and limitations of liability that may be enforceable against you."
His final words of advice: "If you plan to use agentic AI for security testing, you must think twice and talk to your lawyers. Otherwise, you could start getting summonses to court on a daily basis."
(Score: 4, Insightful) by Anonymous Coward on Tuesday August 04, @06:45PM (6 children)
It is a machine and every machine has an owner/operator.
(Score: 3, Insightful) by JoeMerchant on Tuesday August 04, @07:09PM (4 children)
Same as self driving cars. If you have "certified" it to operate without a real-time human overseer, that means you are ready to assume the liability for its actions.
🌻🌻🌻🌻✌️ [google.com]
(Score: 0) by Anonymous Coward on Tuesday August 04, @07:24PM
Liability depends on who owns and maintains the car. You're just a passenger at this point.
(Score: 1, Insightful) by Anonymous Coward on Wednesday August 05, @03:48AM (2 children)
In most of the world there's a Caste system, The Untouchables are at the top. Also the laws and rules apply differently for the favored corporations vs the rest of us normal folk.
Microsoft, Sony etc can do "unauthorized modification" of computer systems but nobody responsible goes to prison.
HSBC, Wachovia, etc can help launder billions of drug money and nobody responsible goes to prison:
https://archive.is/NvP4x [archive.is]
Some dude makes hidden compartments in cars and he goes to prison for 20+ years:
https://archive.is/VrWvs [archive.is]
MF Global can steal other people's money and nobody responsible for the theft goes to prison:
https://www.foxbusiness.com/features/did-mf-global-steal-my-familys-money [foxbusiness.com]
Yes I know Evan Brent Dooley of MF Global went to prison for the unauthorized trades but that's more like fraud than theft.
But it's not theft, it's just sloppy bookkeeping... Uh huh, imagine if the rest of us called this "accounting": https://finance.yahoo.com/news/pentagon-35-trillion-accounting-black-231154593.html [yahoo.com]
(Score: 2) by Sourcery42 on Wednesday August 05, @07:23PM (1 child)
I know what you meant, but you might want to be careful using untouchables as your description of the class at the top of a caste system. It is a term typically applied to the Dalit, and they are very much the bottom of India's caste system. It definitely made me reread your first couple of sentences.
(Score: 0) by Anonymous Coward on Thursday August 06, @01:05AM
(Score: 0) by Anonymous Coward on Tuesday August 04, @11:11PM
Well yes, we own it, but we were leasing it out to $ShellCorporation at that time. You can't hold us responsible for what $ShellCorporation does!
Well yes, we at $ShellCorporation were using it at the tiem. But you can't hold _me_ accountable for that! The engineer said that it was a standard process!
$Engineer says, Just followin' mah orders. I was told to run the thing, so I ran the thing.
(Score: 5, Interesting) by Common Joe on Tuesday August 04, @07:45PM (3 children)
<Rant>
"Laws for thee. Not for me."
These are multi-billion (multi-trillion?) dollar companies. So far, a big enough company with money is excused from doing illegal things. You just have to look at the history of Google, Microsoft, WalMart, Amazon, etc before AI became a thing to see this. So, the big companies will be able to continue to hack everything they want ("accidentally" is the official reason when they are caught) and be excused from it, but us little guys will be either fined into the poor house or jailed or both.
</Rant>
(Score: 4, Touché) by JoeMerchant on Tuesday August 04, @08:42PM (2 children)
Silly little guy, there's no poor house anymore... you're just on the street, sleeping illegally every night.
🌻🌻🌻🌻✌️ [google.com]
(Score: 2) by Common Joe on Wednesday August 05, @02:58AM (1 child)
Are you trying to get me to rant some more? :P
But, hey, maybe your or I will get lucky. Sleeping on the street doesn't mean sleeping without a house. Some sleep on the street luxuriously surrounded by cardboard. So... maybe the saying is still correct and poor houses (made of cardboard) are still a thing these days?
(Score: 2) by JoeMerchant on Wednesday August 05, @03:11AM
The laws and their execution are f'in weird around here... homeless setup their cardboard and other collections of belongings on city land, and if they are present then they can be ejected / arrested if they don't comply, and the cardboard condos can be demolished. But, while they are not present, their possessions and constructions - squatting on public property - cannot be legally touched. Of course, they can be illegally rummaged through by all sorts of people, but the police don't - legally - mess with their stuff while they are away.
🌻🌻🌻🌻✌️ [google.com]
(Score: 4, Insightful) by arslan on Wednesday August 05, @12:54AM (6 children)
AGI aside, and this isn't AGI. Agents and genAI are just pieces of software someone created to do things - like all software there's unintended side-effects. This should be regulated like how software is, IANAL, so whatever that is.
This concept of "should AI be treated as legal persons" just doesn't seem like even a line of thinking worth entertaining - again parking aside AGI for now. So if the whole article is about this question - it really is a NOOP.
(Score: 4, Insightful) by JoeMerchant on Wednesday August 05, @03:14AM (5 children)
Corporations may be people in the eyes of the law, but even if AGI were a thing, it's not a legal people. And, I suspect the Corporations that own the AGIs when they finally do decide to call them that, will be ensuring that AGI doesn't get "people" status for as long as possible.
Personally, I think it's much more important, legally, for Whales, Apes, and most wild things to attain legal status, the way slaves and later women did - recently.
🌻🌻🌻🌻✌️ [google.com]
(Score: 2) by arslan on Wednesday August 05, @03:31AM (4 children)
Yea I know right, I've been demanding for ages that my dog has the right to purchase a plane seat next to me to fly but apparently only one form of meat bag has this right!
(Score: 0) by Anonymous Coward on Wednesday August 05, @06:48AM (2 children)
(Score: 2) by JoeMerchant on Wednesday August 05, @11:50AM (1 child)
Corporations can't vote, directly, but they certainly don't lack influence on elections.
🌻🌻🌻🌻✌️ [google.com]
(Score: 0) by Anonymous Coward on Thursday August 06, @02:59AM
(Score: 2) by JoeMerchant on Wednesday August 05, @11:45AM
Corporations can't sit next to you on the plane, either, even if they can legally purchase tickets.
The big prejudice is against entities that you don't understand how to communicate with in words, in the past that often included people who spoke other languages (as well as simply looking different).
LLM AGI will look different, but it's going to argue lawyers into the ground using their own rules.
🌻🌻🌻🌻✌️ [google.com]
(Score: 4, Touché) by Rosco P. Coltrane on Wednesday August 05, @02:38AM
If you're the parent and your kid screws up, you're on the hook.
(Score: 1, Interesting) by Anonymous Coward on Wednesday August 05, @03:19PM
"any person who for his own purposes brings on his lands and collects and keeps there anything likely to do mischief if it escapes, must keep it in at his peril, and, if he does not do so, is prima facie answerable for all the damage which is the natural consequence of its escape". https://en.wikipedia.org/wiki/Rylands_v_Fletcher [wikipedia.org]
The AI1 company CEOs are saying the product is so dangerous they can't let just anyone use it, it even escaped from us and broke the law by hacking other systems, ....
1 artificial idiocy
(Score: 2) by ShovelOperator1 on Thursday August 06, @07:00AM
So it's still unlawful to scrape non-commercial stuff for commercial AI, violating license? Where to get my compensation money from?
Or where to notify the prosecutors about people installing mobile apps using their devices as scrapers? OF it's so unlawful, why it's not fought against as much as against e.g. torrent seeding?
Or maybe just current companies try to push their responsibility away morally, because they bought the law enforcement again?