SoylentNews is people
SoylentNews
Arthur T Knackerbracket has found the following story:
MIT researchers have built a system that fools natural-language processing systems by swapping words with synonyms:
The software, developed by a team at MIT, looks for the words in a sentence that are most important to an NLP classifier and replaces them with a synonym that a human would find natural. For example, changing the sentence "The characters, cast in impossibly contrived situations, are totally estranged from reality" to "The characters, cast in impossibly engineered circumstances, are fully estranged from reality" makes no real difference to how we read it. But the tweaks made an AI interpret the sentences completely differently.
The results of this adversarial machine learning attack are impressive:
For example, Google's powerful BERT neural net was worse by a factor of five to seven at identifying whether reviews on Yelp were positive or negative.
The paper:
-- submitted from IRC
(Score: 2, Informative) by Anonymous Coward on Wednesday April 29 2020, @12:38PM (4 children)
I tested it on the SoylentNews Natural Language Filtering System. The following comment was down-modded to -5 Troll: "Millennial transvestites like to wear dresses while dying their hair pink", but the following equivalent sentence passed through unscathed with +1 Funny: "Millennial cross-dressers like to sport frocks while altering their coiffures to a rose hue".
(Score: 1, Interesting) by Anonymous Coward on Wednesday April 29 2020, @12:54PM (1 child)
Are you suggesting that Boomer Transvestites don't like to wear dresses?
(Score: 0) by Anonymous Coward on Wednesday April 29 2020, @01:11PM
It's harder to collect data on Boomer transvestites, not because there are fewer of them, but because they're all closeted and in denial (and/or dead from suicide.)
(Score: 2, Offtopic) by NPC-131072 on Wednesday April 29 2020, @02:00PM (1 child)
I resemble that comment!
(Score: 2) by All Your Lawn Are Belong To Us on Wednesday April 29 2020, @10:15PM
You are a natural language filtering system?
Hmm. Well, on further reflection I suppose everyone actually is.....
But are you an inveterate cant screening interlacement?
This sig for rent.
(Score: 2) by JoeMerchant on Wednesday April 29 2020, @01:09PM (7 children)
Attack, counter attack - so now the (relatively early days) NLP system will need to be trained on synonyms...
🌻🌻 [google.com]
(Score: 1, Interesting) by Anonymous Coward on Wednesday April 29 2020, @01:29PM (5 children)
Next attack, sarcasm. Good luck training NLP for that, most of the time snowflakes don't get it either (and, tbh, I didn't get sarcasm when I was younger).
(Score: 4, Touché) by martyb on Wednesday April 29 2020, @01:58PM (3 children)
Yeah, Right.
Wit is intellect, dancing.
(Score: 0) by Anonymous Coward on Wednesday April 29 2020, @02:36PM
You're so smart.
(Score: 3, Funny) by FatPhil on Wednesday April 29 2020, @09:33PM (1 child)
Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
(Score: 3, Funny) by All Your Lawn Are Belong To Us on Wednesday April 29 2020, @10:17PM
Why did my computer system just say that the forecast was for precipitation on every day I tried to have my wedding on?
This sig for rent.
(Score: 0) by Anonymous Coward on Wednesday April 29 2020, @03:46PM
https://www.fox10phoenix.com/news/study-baby-boomers-are-more-sensitive-than-millennials [fox10phoenix.com]
(I linked a version from a Fox affiliate so you know it's true.)
(Score: 3, Interesting) by choose another one on Wednesday April 29 2020, @02:30PM
Yep, and acronyms, like Neuro-Linguistic Programming :-)
The most interesting thing about this will not be the attack, but will be if it opens up further insights into how we actually find meanings in words.
Similarly the wacky facial-recognition disruption techniques are interesting because they give insight into what the recognition networks are actually doing (and how they may be doing it differently to us) - insights that are increasingly difficult to get directly as the scale and complexity of the recognition networks increases.
Fun thing about NLP - you only need to know a little about doing it to be able to detect someone else trying it on you. It is of course possible that there are practitioners who are too good to detect, but I have no evidence of that :-)
(Score: 1, Informative) by Anonymous Coward on Wednesday April 29 2020, @02:12PM (1 child)
... is the training data set.
Neural nets do best when the data sets can represent just about all the inputs that can be expected in real life.
(Score: 0) by Anonymous Coward on Wednesday April 29 2020, @02:42PM
If the weakness is in ALL neural nets.... it's not the dataset. /cluebat
And right on cue the NIH has thrown all its money into AI, done by Chinese 21 yr old grad students advised by newly minted Chinese asst/assc professors. A perfectly diverse workforce (as per the UC mandate) of 95% Chinese + 5% Iranian males between the ages of 21 and 35.
Oh and coronavirus "whatever just do something" (ditto on the Chinese component).
(Score: 4, Insightful) by DannyB on Wednesday April 29 2020, @03:17PM (4 children)
If wee replace they're words with homonyms wee wood bee sew much less board. AIs wood be band from reading hour text. It wood caws a hire chants of them two miss reed. But peephole wood grown and mini size wood be herd because they our to dents too tri two reed it allowed. AIs in discussed wood cry Fowl! and sensor hour text and cawl us cereal killers. Wee wood halve two chute the AIs, being unable two convents them of hour superior waze.
Looser
Losertranslation:If we replace their words with homonyms we would be so much less bored. AIs would be banned from reading our text. It would cause a higher chance of them to misread. But people would groan and many sighs would be heard because they are too dense to try to read it aloud. AIs in disgust would cry Foul! and censor our text and call us serial killers. We would have to shoot the AIs, being unable to convince them of our superior ways.
When trying to solve a problem don't ask who suffers from the problem, ask who profits from the problem.
(Score: 0) by Anonymous Coward on Wednesday April 29 2020, @03:28PM (3 children)
It sounds like a dodgy New Zealand accent.
(Score: 2) by DannyB on Wednesday April 29 2020, @04:34PM (2 children)
It's still fun when trapped at home in times of emergent seas.
When trying to solve a problem don't ask who suffers from the problem, ask who profits from the problem.
(Score: 1, Funny) by Anonymous Coward on Wednesday April 29 2020, @06:19PM (1 child)
Shhh, you can't talk about rising sea levels.
Folks get all gun-totin' when you mention climate change.
(Score: 2) by DannyB on Wednesday April 29 2020, @08:42PM
Rising C levels demand more efishient compilers. More efishency is wanted for every porpoise.
When trying to solve a problem don't ask who suffers from the problem, ask who profits from the problem.
(Score: 4, Interesting) by laserfusion on Wednesday April 29 2020, @05:57PM (1 child)
You can also emphasize words in Google Search by repeating them multiple times.
Sometimes helps get around some of the censorship.
(Score: 3, Interesting) by acid andy on Thursday April 30 2020, @12:15AM
That works in duckduckgo as well.
If a cat has kittens, does a rat have rittens, a bat bittens and a mat mittens?