Arthur T Knackerbracket has found the following story:
MIT researchers have built a system that fools natural-language processing systems by swapping words with synonyms:
The software, developed by a team at MIT, looks for the words in a sentence that are most important to an NLP classifier and replaces them with a synonym that a human would find natural. For example, changing the sentence "The characters, cast in impossibly contrived situations, are totally estranged from reality" to "The characters, cast in impossibly engineered circumstances, are fully estranged from reality" makes no real difference to how we read it. But the tweaks made an AI interpret the sentences completely differently.
The results of this adversarial machine learning attack are impressive:
For example, Google's powerful BERT neural net was worse by a factor of five to seven at identifying whether reviews on Yelp were positive or negative.
The paper:
-- submitted from IRC
(Score: 3, Interesting) by choose another one on Wednesday April 29 2020, @02:30PM
Yep, and acronyms, like Neuro-Linguistic Programming :-)
The most interesting thing about this will not be the attack, but will be if it opens up further insights into how we actually find meanings in words.
Similarly the wacky facial-recognition disruption techniques are interesting because they give insight into what the recognition networks are actually doing (and how they may be doing it differently to us) - insights that are increasingly difficult to get directly as the scale and complexity of the recognition networks increases.
Fun thing about NLP - you only need to know a little about doing it to be able to detect someone else trying it on you. It is of course possible that there are practitioners who are too good to detect, but I have no evidence of that :-)