Stories
Slash Boxes
Comments

SoylentNews is people

posted by Fnord666 on Wednesday April 29 2020, @11:40AM   Printer-friendly
from the fun-with-words dept.

Arthur T Knackerbracket has found the following story:

MIT researchers have built a system that fools natural-language processing systems by swapping words with synonyms:

The software, developed by a team at MIT, looks for the words in a sentence that are most important to an NLP classifier and replaces them with a synonym that a human would find natural. For example, changing the sentence "The characters, cast in impossibly contrived situations, are totally estranged from reality" to "The characters, cast in impossibly engineered circumstances, are fully estranged from reality" makes no real difference to how we read it. But the tweaks made an AI interpret the sentences completely differently.

The results of this adversarial machine learning attack are impressive:

For example, Google's powerful BERT neural net was worse by a factor of five to seven at identifying whether reviews on Yelp were positive or negative.

The paper:

-- submitted from IRC


Original Submission

 
This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 3, Interesting) by choose another one on Wednesday April 29 2020, @02:30PM

    by choose another one (515) Subscriber Badge on Wednesday April 29 2020, @02:30PM (#988190)

    Yep, and acronyms, like Neuro-Linguistic Programming :-)

    The most interesting thing about this will not be the attack, but will be if it opens up further insights into how we actually find meanings in words.

    Similarly the wacky facial-recognition disruption techniques are interesting because they give insight into what the recognition networks are actually doing (and how they may be doing it differently to us) - insights that are increasingly difficult to get directly as the scale and complexity of the recognition networks increases.

    Fun thing about NLP - you only need to know a little about doing it to be able to detect someone else trying it on you. It is of course possible that there are practitioners who are too good to detect, but I have no evidence of that :-)

    Starting Score:    1  point
    Moderation   +1  
       Interesting=1, Total=1
    Extra 'Interesting' Modifier   0  
    Karma-Bonus Modifier   +1  

    Total Score:   3