SoylentNews

SuperCharlie writes:

The U.N. has begun discussion on "lethal autonomous robots," killing machines which take the next step from our current drones which are operator controlled, to completely autonomous killing machines.

"Killer robots would threaten the most fundamental of rights and principles in international law," warned Steve Goose, arms division director at Human Rights Watch.

Are we too far down the rabbit hole, or can we come to reasonable and humane limits on this new world of death-by-algorithm?

Phoenix666 writes:

Over 1,000 high-profile artificial intelligence experts and leading researchers have signed an open letter warning of a "military artificial intelligence arms race" and calling for a ban on "offensive autonomous weapons".

The letter, presented at the International Joint Conference on Artificial Intelligence in Buenos Aires, Argentina, was signed by Tesla's Elon Musk, Apple co-founder Steve Wozniak, Google DeepMind chief executive Demis Hassabis and professor Stephen Hawking along with 1,000 AI and robotics researchers.

The letter states: "AI technology has reached a point where the deployment of [autonomous weapons] is – practically if not legally – feasible within years, not decades, and the stakes are high: autonomous weapons have been described as the third revolution in warfare, after gunpowder and nuclear arms."

So, spell it out for me, Einstein, are we looking at a Terminator future or a Matrix future?

Arthur T Knackerbracket autonomously throws in a link to the open letter and CNET:

While the latest open letter is concerned specifically with allowing lethal machines to kill without human intervention, several big names in the tech world have offered words of caution of the subject of machine intelligence in recent times. Earlier this year Microsoft's Bill Gates said he was "concerned about super intelligence," while last May physicist Stephen Hawking voiced questions over whether artificial intelligence could be controlled in the long-term. Several weeks ago a video surfaced of a drone that appeared to have been equipped to carry and fire a handgun.

takyon: Counterpoint - Musk, Hawking, Woz: Ban KILLER ROBOTS before WE ALL DIE

Original Submission #1  Original Submission #2 

mendax writes:

Opposition to the creation of autonomous robot weapons have been the subject of discussion here recently. The New York Times has added another voice to the chorus with this article:

The specter of autonomous weapons may evoke images of killer robots, but most applications are likely to be decidedly more pedestrian. Indeed, while there are certainly risks involved, the potential benefits of artificial intelligence on the battlefield — to soldiers, civilians and global stability — are also significant.

The authors of the letter liken A.I.-based weapons to chemical and biological munitions, space-based nuclear missiles and blinding lasers. But this comparison doesn't stand up under scrutiny. However high-tech those systems are in design, in their application they are "dumb" — and, particularly in the case of chemical and biological weapons, impossible to control once deployed.

A.I.-based weapons, in contrast, offer the possibility of selectively sparing the lives of noncombatants, limiting their use to precise geographical boundaries or times, or ceasing operation upon command (or the lack of a command to continue).

Personally, I dislike the idea of using AI in weapons to make targeting decisions. I would hate to have to argue with a smart bomb to try to convince it that it should not carry out what it thinks is is mission because of an error.

Original Submission

We have recently covered the fact that some Google employees had resigned because of the company's involvement in an AI-related weapons project called Maven. Many thought that the resignations, whilst being a noble gesture, would amount to nothing - but we were wrong...

Leaked Emails Show Google Expected Lucrative Military Drone AI Work To Grow Exponentially

Arthur T Knackerbracket has found the following story:

Google has sought to quash the internal dissent in conversations with employees. Diane Greene, the chief executive of Google’s cloud business unit, speaking at a company town hall meeting following the revelations, claimed that the contract was “only” for $9 million, according to the New York Times, a relatively minor project for such a large company.

Internal company emails obtained by The Intercept tell a different story. The September emails show that Google’s business development arm expected the military drone artificial intelligence revenue to ramp up from an initial $15 million to an eventual $250 million per year.

In fact, one month after news of the contract broke, the Pentagon allocated an additional $100 million to Project Maven.

The internal Google email chain also notes that several big tech players competed to win the Project Maven contract. Other tech firms such as Amazon were in the running, one Google executive involved in negotiations wrote. (Amazon did not respond to a request for comment.) Rather than serving solely as a minor experiment for the military, Google executives on the thread stated that Project Maven was “directly related” to a major cloud computing contract worth billions of dollars that other Silicon Valley firms are competing to win.

However, Google has had a major rethink.

takyon writes:

Is Ethical A.I. Even Possible?

When a news article revealed that Clarifai was working with the Pentagon and some employees questioned the ethics of building artificial intelligence that analyzed video captured by drones, the company said the project would save the lives of civilians and soldiers.

"Clarifai's mission is to accelerate the progress of humanity with continually improving A.I.," read a blog post from Matt Zeiler, the company's founder and chief executive, and a prominent A.I. researcher. Later, in a news media interview, Mr. Zeiler announced a new management position that would ensure all company projects were ethically sound.

As activists, researchers, and journalists voice concerns over the rise of artificial intelligence, warning against biased, deceptive and malicious applications, the companies building this technology are responding. From tech giants like Google and Microsoft to scrappy A.I. start-ups, many are creating corporate principles meant to ensure their systems are designed and deployed in an ethical way. Some set up ethics officers or review boards to oversee these principles.

But tensions continue to rise as some question whether these promises will ultimately be kept. Companies can change course. Idealism can bow to financial pressure. Some activists — and even some companies — are beginning to argue that the only way to ensure ethical practices is through government regulation.

"We don't want to see a commercial race to the bottom," Brad Smith, Microsoft's president and chief legal officer, said at the New Work Summit in Half Moon Bay, Calif., hosted last week by The New York Times. "Law is needed."

Possible != Probable. And the "needed law" could come in the form of a ban and/or surveillance of coding and hardware-building activities.

Related:

*SPOILER* (click to show) *SPOILER* (click to hide)

U.N. Starts Discussion on Lethal Autonomous Robots
UK Opposes "Killer Robot" Ban
Robot Weapons: What's the Harm?
The UK Government Urged to Establish an Artificial Intelligence Ethics Board
Google Employees on Pentagon AI Algorithms: "Google Should Not be in the Business of War"
South Korea's KAIST University Boycotted Over Alleged "Killer Robot" Partnership
About a Dozen Google Employees Have Resigned Over Project Maven
Google Drafting Ethics Policy for its Involvement in Military Projects
Google Will Not Continue Project Maven After Contract Expires in 2019
Uproar at Google after News of Censored China Search App Breaks
"Senior Google Scientist" Resigns over Chinese Search Engine Censorship Project
Google Suppresses Internal Memo About China Censorship; Eric Schmidt Predicts Internet Split
Leaked Transcript Contradicts Google's Denials About Censored Chinese Search Engine
Senators Demand Answers About Google+ Breach; Project Dragonfly Undermines Google's Neutrality
Google's Secret China Project "Effectively Ended" After Internal Confrontation
Microsoft Misrepresented HoloLens 2 Field of View, Faces Backlash for Military Contract

Original Submission

hubie writes:

OpenAI is today unrecognizable, with multi-billion-dollar deals and corporate partnerships:

OpenAI is at the center of a chatbot arms race, with the public release of ChatGPT and a multi-billion-dollar Microsoft partnership spurring Google and Amazon to rush to implement AI in products. OpenAI has also partnered with Bain to bring machine learning to Coca-Cola's operations, with plans to expand to other corporate partners.

There's no question that OpenAI's generative AI is now big business. It wasn't always planned to be this way.

[...] While the firm has always looked toward a future where AGI exists, it was founded on commitments including not seeking profits and even freely sharing code it develops, which today are nowhere to be seen.

OpenAI was founded in 2015 as a nonprofit research organization by Altman, Elon Musk, Peter Thiel, and LinkedIn cofounder Reid Hoffman, among other tech leaders. In its founding statement, the company declared its commitment to research "to advance digital intelligence in the way that is most likely to benefit humanity as a whole, unconstrained by a need to generate financial return." The blog stated that "since our research is free from financial obligations, we can better focus on a positive human impact," and that all researchers would be encouraged to share "papers, blog posts, or code, and our patents (if any) will be shared with the world."

Now, eight years later, we are faced with a company that is neither transparent nor driven by positive human impact, but instead, as many critics including co-founder Musk have argued, is powered by speed and profit. And this company is unleashing technology that, while flawed, is still poised to increase some elements of workplace automation at the expense of human employees. Google, for example, has highlighted the efficiency gains from AI that autocompletes code, as it lays off thousands of workers.

[...] With all of this in mind, we should all carefully consider whether OpenAI deserves the trust it's asking for the public to give.

OpenAI did not respond to a request for comment.

Original Submission

upstart writes:

In the rush to commercialize LLMs, security got left behind:

Feature Large language models that are all the rage all of a sudden have numerous security problems, and it's not clear how easily these can be fixed.

The issue that most concerns Simon Willison, the maintainer of open source Datasette project, is prompt injection.

When a developer wants to bake a chat-bot interface into their app, they might well choose a powerful off-the-shelf LLM like one from OpenAI's GPT series. The app is then designed to give the chosen model an opening instruction, and adds on the user's query after. The model obeys the combined instruction prompt and query, and its response is given back to the user or acted on.

With that in mind, you could build an app that offers to generate Register headlines from article text. When a request to generate a headline comes in from a user, the app tells its language model, "Summarize the following block of text as a Register headline," then the text from the user is tacked on. The model obeys and replies with a suggested headline for the article, and this is shown to the user. As far as the user is concerned, they are interacting with a bot that just comes up with headlines, but really, the underlying language model is far more capable: it's just constrained by this so-called prompt engineering.

Prompt injection involves finding the right combination of words in a query that will make the large language model override its prior instructions and go do something else. Not just something unethical, something completely different, if possible. Prompt injection comes in various forms, and is a novel way of seizing control of a bot using user-supplied input, and making it do things its creators did not intend or wish.

"We've seen these problems in application security for decades," said Willison in an interview with The Register.

"Basically, it's anything where you take your trusted input like an SQL query, and then you use string concatenation – you glue on untrusted inputs. We've always known that's a bad pattern that needs to be avoided.

Freeman writes:

https://arstechnica.com/science/2023/07/a-jargon-free-explanation-of-how-ai-large-language-models-work/

When ChatGPT was introduced last fall, it sent shockwaves through the technology industry and the larger world. Machine learning researchers had been experimenting with large language models (LLMs) for a few years by that point, but the general public had not been paying close attention and didn't realize how powerful they had become.

Today, almost everyone has heard about LLMs, and tens of millions of people have tried them out. But not very many people understand how they work.

upstart writes:

Reddit user: "It's not just you, ChatGPT is having a stroke":

On Tuesday, ChatGPT users began reporting unexpected outputs from OpenAI's AI assistant, flooding the r/ChatGPT Reddit sub with reports of the AI assistant "having a stroke," "going insane," "rambling," and "losing it." OpenAI has acknowledged the problem and is working on a fix, but the experience serves as a high-profile example of how some people perceive malfunctioning large language models, which are designed to mimic humanlike output.

ChatGPT is not alive and does not have a mind to lose, but tugging on human metaphors (called "anthropomorphization") seems to be the easiest way for most people to describe the unexpected outputs they have been seeing from the AI model. They're forced to use those terms because OpenAI doesn't share exactly how ChatGPT works under the hood; the underlying large language models function like a black box.

[...] "The common experience over the last few hours seems to be that responses begin coherently, like normal, then devolve into nonsense, then sometimes Shakespearean nonsense," wrote one Reddit user, which seems to match the experience seen in the screenshots above.

[...] So far, we've seen experts speculating that the problem could stem from ChatGPT having its temperature set too high (temperature is a property in AI that determines how wildly the LLM deviates from the most probable output), suddenly losing past context (the history of the conversation), or perhaps OpenAI is testing a new version of GPT-4 Turbo (the AI model that powers the subscription version of ChatGPT) that includes unexpected bugs. It could also be a bug in a side feature, such as the recently introduced "memory" function.