SoylentNews is people
SoylentNews
Microsoft's AI text-to-image generator, Copilot Designer, appears to be heavily filtering outputs after a Microsoft engineer, Shane Jones, warned that Microsoft has ignored warnings that the tool randomly creates violent and sexual imagery, CNBC reported.
Jones told CNBC that he repeatedly warned Microsoft of the alarming content he was seeing while volunteering in red-teaming efforts to test the tool's vulnerabilities. Microsoft failed to take the tool down or implement safeguards in response, Jones said, or even post disclosures to change the product's rating to mature in the Android store.
[...] Bloomberg also reviewed Jones' letter and reported that Jones told the FTC that while Copilot Designer is currently marketed as safe for kids, it's randomly generating an "inappropriate, sexually objectified image of a woman in some of the pictures it creates." And it can also be used to generate "harmful content in a variety of other categories, including: political bias, underage drinking and drug use, misuse of corporate trademarks and copyrights, conspiracy theories, and religion to name a few."
[...] Jones' tests also found that Copilot Designer would easily violate copyrights, producing images of Disney characters, including Mickey Mouse or Snow White. Most problematically, Jones could politicize Disney characters with the tool, generating images of Frozen's main character, Elsa, in the Gaza Strip or "wearing the military uniform of the Israel Defense Forces."
Ars was able to generate interpretations of Snow White, but Copilot Designer rejected multiple prompts politicizing Elsa.
If Microsoft has updated the automated content filters, it's likely due to Jones protesting his employer's decisions. [...] Jones has suggested that Microsoft would need to substantially invest in its safety team to put in place the protections he'd like to see. He reported that the Copilot team is already buried by complaints, receiving "more than 1,000 product feedback messages every day." Because of this alleged understaffing, Microsoft is currently only addressing "the most egregious issues," Jones told CNBC.
Related stories on SoylentNews:
Cops Bogged Down by Flood of Fake AI Child Sex Images, Report Says - 20240202
New "Stable Video Diffusion" AI Model Can Animate Any Still Image - 20231130
The Age of Promptography - 20231008
AI-Generated Child Sex Imagery Has Every US Attorney General Calling for Action - 20230908
It Costs Just $400 to Build an AI Disinformation Machine - 20230904
US Judge: Art Created Solely by Artificial Intelligence Cannot be Copyrighted - 20230824
"Meaningful Harm" From AI Necessary Before Regulation, says Microsoft Exec - 20230514 (Microsoft's new quarterly goal?)
the Godfather of AI Leaves Google Amid Ethical Concerns - 20230502
Stable Diffusion Copyright Lawsuits Could be a Legal Earthquake for AI - 20230403
AI Image Generator Midjourney Stops Free Trials but Says Influx of New Users to Blame - 20230331
Microsoft's New AI Can Simulate Anyone's Voice With Three Seconds of Audio - 20230115
Breakthrough AI Technique Enables Real-Time Rendering of Scenes in 3D From 2D Images - 20211214
(Score: 4, Insightful) by krishnoid on Saturday March 09 2024, @05:19PM (2 children)
I mean, Microsoft's had problems with their artificial intelligence [theverge.com] ventures before. Maybe it says more about Microsoft's roots -- after all, Bill Gates wasn't the most collaborative of businesspeople, and I bet it suffuses their whole corporate culture and direction today.
(Score: 1, Touché) by Anonymous Coward on Saturday March 09 2024, @06:20PM
It's probably copyright and/or trademark infringement if MS's AI is producing stuff that strongly resembles Disney's. More so if they're charging some people etc.
(Score: 4, Interesting) by Spamalope on Sunday March 10 2024, @12:31AM
I wonder if any of this is driving by competitors deliberately trying to cause this. (as with the long standing practice of posting offensive content on competing platforms or politically opposed forum then complaining). That wouldn't absolve anyone, but would be a mitigating factor. At some point they're going to need a 'kid/worksafe adult/NSFW adult' switch that actually works for all of them.
(Score: 2, Interesting) by VLM on Saturday March 09 2024, @05:26PM (2 children)
So it's Hollywood. Not impressed.
Does this mean AI is Hollywood or Hollywood is AI or the same people with the same values run both?
(Score: 5, Insightful) by choose another one on Saturday March 09 2024, @05:50PM (1 child)
> Does this mean AI is Hollywood or Hollywood is AI or the same people with the same values run both?
Probably the last one.
Also this got me: "inappropriate, sexually objectified image of a woman"
So, not a man, or maybe he's sure the image is not a man because he asked for woman (be careful in Thailand)?
It's AI Gen dude, it's not of man or woman, _it's_ _not_ _real_ (be nice if AI gen images could come with non removable warning label to that effect).
Oh, and inappropriate by whose standards? Taliban, Iranian Morality Police?
(Score: 2) by Freeman on Monday March 11 2024, @03:25PM
Microsoft engineer warns company’s AI tool creates violent, sexual images, ignores copyrights [cnbc.com] (From the CNBC Article.)
Joshua 1:9 "Be strong and of a good courage; be not afraid, neither be thou dismayed: for the Lord thy God is with thee"
(Score: 4, Informative) by looorg on Saturday March 09 2024, @07:12PM (3 children)
I thought their tool knew what it was doing, State-of-the-art and all of that. To have it randomly spew out things -- code, text or profanity shouldn't exactly be inspiring future customers with confidence. If this is what it does randomly, what else is it doing randomly? If it randomly just does things for shits and giggles then as a product it's a failure that can't be trusted to do anything.
(Score: 2) by ElizabethGreene on Saturday March 09 2024, @10:24PM
The tool does not know what it's doing. It's a black box really. You can do content filtering on the prompts and run the output through a "is this smut?" detector, but things will fall through the cracks. It's the same problem you face when trying to write net-nanny software.
(Score: 4, Informative) by number11 on Sunday March 10 2024, @01:18AM
Let's face it, if you can develop an AI that will not violate any human laws, it's going to be comatose. A human can't do that, why would you expect a simulacrum to do it?
(Score: 2) by SomeGuy on Sunday March 10 2024, @02:54PM
The entire problem with "AI" since DAY ONE has been that one never exactly knows what an IA system will do or why. But idiots still expect they can just plug it in for some application and have it do exactly what they think it will. Then they are shocked when it goes all sediment shaped sediment on them.
(Score: 4, Interesting) by ElizabethGreene on Saturday March 09 2024, @10:16PM
Bias warning, I work for Microsoft and I've fiddled with AI.
I've heard anecdotally that when Disney hires a new animator, they offer them access to their honest-to-god **stacks** of Rule 34 (and worse) content they've pulled off the internet. This is so they understand what will happen to characters they create. Their legal team pulls stuff off the internet every day, but it's a Sisyphean task. There is a never-ending fountain of smut even Pre-AI. How does AI make that worse? It's a people problem.
Candidly, I've yet to see any AI images worse than what's on the 4chan/b thumbnail catalog on a slow day.
A few gratitude sidebars here:
1. Thank you so much to the Genshin Impact and Overwatch Rule34 contributors. Your contributions to Blender have made it an amazing piece of open-source software.
2. Also, many thanks to the Furry community. I have an artist/costume designer relative that has scraped through some hard times with the help of "creative" commissioned pieces.
(Score: 3, Touché) by Snotnose on Sunday March 10 2024, @01:06AM (1 child)
You're telling me no paintbrush or paint maker has never sold their tools to someone who used them to create violent and/or sexual imagery they sold to kids?
How do I know you've never been to a Comic-con, nor watched Saturday morning cartoons back in the day.
Somewhere in America a little girl is hiding in an attic writing about ICE.
(Score: 3, Insightful) by boltronics on Sunday March 10 2024, @03:39AM
AI in these cases is less of a tool to facilitate the creation of art, but more akin to a piece of equipment that generates art for you.
Perhaps a better analogy is a common car vs a self-driving car. Self driving cars are illegal in most parts of the world (at least without human supervision ensuring they are working as intended). At some point they will likely prove to be safer than a human driver, legal frameworks will be created and self-driving cars will become authorised for their ultimately intended use of removing human involvement from the driving process, aside from the person inputting where to go. A human won't be able to deliberately use the car to run down people or other illegal acts as partial responsibility would be on the manufacturer. The car only needs to know the destination.
AI is like a self-driving car, on the road today in all countries, without legal frameworks in place dictating how they need to operate, and without even being close to demonstrating what humans can do in terms of being fit for purpose — or at the very least, not doing something that is illegal in some way, be it the output, or even how it was trained. That is the issue.
It's GNU/Linux dammit!