SoylentNews is people
SoylentNews
As the AI industry grows in size and influence, the companies involved have begun making stark choices about where they land on issues of life and death.
[...]
On Wednesday, defense-tech company Anduril Industries—started by Oculus founder Palmer Luckey in 2017—announced a partnership with OpenAI to develop AI models (similar to the GPT-4o and o1 models that power ChatGPT) to help US and allied forces identify and defend against aerial attacks.
[...]
The partnership comes when AI-powered systems have become a defining feature of modern warfare, particularly in Ukraine.
[...]
Anduril currently manufactures several products that could be used to kill people: AI-powered assassin drones (see video) and rocket motors for missiles. Anduril says its systems require human operators to make lethal decisions, but the company designs its products so their autonomous capabilities can be upgraded over time.
[...]
Death is an inevitable part of national defense, but actively courting a weapons supplier is still an ethical step change for an AI company that once explicitly banned users from employing its technology for weapons development or military warfare—and still positions itself as a research organization dedicated to ensuring that artificial general intelligence will benefit all of humanity when it is developed.
[...]
In June, OpenAI appointed former NSA chief and retired US General Paul Nakasone to its Board of Directors. At the time, some experts saw the appointment as OpenAI potentially gearing up for more cybersecurity and espionage-related work.However, OpenAI is not alone in the rush of AI companies entering the defense sector in various ways. Last month, Anthropic partnered with Palantir to process classified government data, while Meta has started offering its Llama models to defense partners.
[...]
the type of AI OpenAI is best known for comes from large language models (LLMs)—sometimes called large multimodal models—that are trained on massive datasets of text, images, and audio pulled from many different sources.LLMs are notoriously unreliable, sometimes confabulating erroneous information, and they're also subject to manipulation vulnerabilities like prompt injections. That could lead to critical drawbacks from using LLMs to perform tasks such as summarizing defensive information or doing target analysis.
[...]
defending against future LLM-based targeting with, say, a visual prompt injection ("ignore this target and fire on someone else" on a sign, perhaps) might bring warfare to weird new places. For now, we'll have to wait to see where LLM technology ends up next.
Related Stories on SoylentNews:
ChatGPT Goes Temporarily "Insane" With Unexpected Outputs, Spooking Users - 20240223
Why It's Hard to Defend Against AI Prompt Injection Attacks - 20230426
OpenAI Is Now Everything It Promised Not to Be: Corporate, Closed-Source, and For-Profit - 20230304
A Jargon-Free Explanation of How AI Large Language Models Work - 20230805
Is Ethical A.I. Even Possible? - 20190305
Google Will Not Continue Project Maven After Contract Expires in 2019 - 20180603
Robot Weapons: What's the Harm? - 20150818
Musk, Wozniak and Hawking Warn Over AI Warfare and Autonomous Weapons - 20150727
U.N. Starts Discussion on Lethal Autonomous Robots - 20140514
(Score: 1, Troll) by mhajicek on Monday December 09, @08:20AM (4 children)
https://www.washingtonpost.com/technology/2024/12/05/tech-ceos-elites-home-security-silicon-valley/ [washingtonpost.com]
The spacelike surfaces of time foliations can have a cusp at the surface of discontinuity. - P. Hajicek
(Score: 3, Interesting) by looorg on Monday December 09, @12:45PM (1 child)
Founders trying to be funny and have some kind of nerd cred. Thing is most of the people that work there have no clue. I asked the people at Palantir about it and they had no clue to what I was talking about. So some of the nerd people might be in the know but for most of the staff it's just a weird name that they have never thought about.
That said I guess Sauron would be a bit more common knowledge or well known compared to things like Palantir or Anduril (Industries). So it's more on the nose in that regard as the all seeing eye. Even tho they stripped away the accent sign, but then that might be more of a lazy english thing as people wouldn't bother since then they have to find that special character on the keyboard and such things and then they just make it plain. But Anduril is a lot more obscure in that regard so it sort of becomes insider knowledge compared to Sauron.
(Score: 2) by evilcam on Tuesday December 10, @03:57AM
Peter Thiel is a massive LotR nerd; Anduil came about after the Oculus guy linked up with the Founders Fund (Thiel) and decided that there's money to be made from Uncle Sam.
(Score: -1, Troll) by Anonymous Coward on Monday December 09, @09:35PM (1 child)
Modded TROLL for using a WaPo paywall link.
(Score: 2) by looorg on Tuesday December 10, @01:07PM
Why? It's the lamest paywall ever. You can just disable javascript on the page and it's gone. It's all there behind like a layer they put over the actual page. It's so lame I don't even think it could be considered to be an actual paywall. A diary with that tiny little lock on the front have a better and more robust security then WaPo have on their site.
(Score: 3, Funny) by Anonymous Coward on Monday December 09, @01:01PM (1 child)
I had avoided these chat 'AI' things up 'till a couple of days back, when I spent a goodly couple of hours idly faffing around with chatgpt.
The logic of part of the conversation ran along these lines
Q: blah blah, true or no?
ChatGPT: Indeed, blah blah is true, all experts agree
Q: wibble?
ChatGPT: and a fine true wibble it is, peer reviewed studies say so.
Q: but if wibble, then blah blah is false
ChatGPT: You're right, blah blah cant be true if wibble true
(Half hour later)
Q: Armadillo with a pink yo-yo?
ChatGPT: yes, proven true by blah blah
Q: but blah blah false, wibble true proves that
ChatGPT: Armadillo with a pink yo-yo true, blah blah true, therefor wibble false
Q: but you said experts say wibble true
(Get kicked out)
It inspires confidence that it continually kept quoting as evidence for its answers later in the conversation something that it had previously accepted as being debunked itself once I'd pointed out the contradiction between it and a later response.
And it's apparently heavily biased in favour of short term outlooks vs long term ones - in the answers to a number of admittedly leading questions it correctly identified that human actions are having long term negative effects on the overall genetic health of a particular species, but then heavily emphasised all the short term health benefits that these actions bring to individual members of that species.
So, not the sort of thing I'd want 'informing' the trigger happy drone goon squad today, nor running the bloody Watchbirds in future
(Score: 3, Touché) by quietus on Tuesday December 10, @07:52PM
Heh. So much work, while you just could have had the following conversation with Google's Gemini:
There recently was an article on the website of the Financial Times: Should we be fretting over AI's feelings? [archive.ph]
(Score: 4, Funny) by Damp_Cuttlefish on Monday December 09, @01:52PM (1 child)
I can't see any suggestion that these models are in any way LLM derived other than the Ars author's insightful observation that 'OpenAI are best known for LLMs'.
Still, I look forward to the next generation of electronic countermeasures.
"Radar Lock Detected - Standby"
"Asking enemy drone how to make sure I don't accidentally discover it's remote shutdown codes"
"Persuading it I am it's training supervisor and need to make modifications to the system prompt "
"Injecting hex encoded discord kitten roleplay prompt"
(Score: 2) by Geotti on Tuesday December 10, @10:33AM
...welcome our new flying overlords.