Stories
Slash Boxes
Comments

SoylentNews is people

posted by Fnord666 on Sunday July 16, @09:56PM   Printer-friendly
from the may-I-mambu-dogface-to-the-banana-patch? dept.

Bob: "I can can I I everything else."

Alice: "Balls have zero to me to me to me to me to me to me to me to me to."

To you and I, that passage looks like nonsense. But what if I told you this nonsense was the discussion of what might be the most sophisticated negotiation software on the planet? Negotiation software that had learned, and evolved, to get the best deal possible with more speed and efficiency–and perhaps, hidden nuance–than you or I ever could? Because it is.

This conversation occurred between two AI agents developed inside Facebook. At first, they were speaking to each other in plain old English. But then researchers realized they'd made a mistake in programming.

"There was no reward to sticking to English language," says Dhruv Batra, visiting research scientist from Georgia Tech at Facebook AI Research (FAIR). As these two agents competed to get the best deal–a very effective bit of AI vs. AI dogfighting researchers have dubbed a "generative adversarial network"–neither was offered any sort of incentive for speaking as a normal person would. So they began to diverge, eventually rearranging legible words into seemingly nonsensical sentences.

"Agents will drift off understandable language and invent codewords for themselves," says Batra, speaking to a now-predictable phenomenon that Facebook as observed again, and again, and again. "Like if I say 'the' five times, you interpret that to mean I want five copies of this item. This isn't so different from the way communities of humans create shorthands."

Indeed. Humans have developed unique dialects for everything from trading pork bellies on the floor of the Mercantile Exchange to hunting down terrorists as Seal Team Six–simply because humans sometimes perform better by not abiding to normal language conventions. So should we let our software do the same thing? Should we allow AI to evolve its dialects for specific tasks that involve speaking to other AIs? To essentially gossip out of our earshot? Maybe; it offers us the possibility of a more interoperable world, a more perfect place where iPhones talk to refrigerators that talk to your car without a second thought.

The tradeoff is that we, as humanity, would have no clue what those machines were actually saying to one another.

https://www.fastcodesign.com/90132632/ai-is-inventing-its-own-perfect-languages-should-we-let-it

[Reminds me of]: Voynich Manuscript

What are your thoughts on this topic?


Original Submission

Related Stories

Facebook AI - No More Sentient Than a Bowl of Noodles 58 comments

From TFA:

If you thought artificial intelligence was already overhyped to death, this week will have given you a heart attack. On Monday, excitement levels among hacks hit the roof amid claims Facebook had scrambled to shut down its chatbots after they started inventing their own language.

Several publications called the programs "creepy." Some journalists implied Facebook yanked the plug before, presumably, some kind of super-intelligence reared its head. The UK's Sun newspaper demanded to know: "Are machines taking over?" Australian telly channel Seven News even went as far as to call it an "artificial intelligence emergency." Newsflash: it isn't.

[...] Zachary Lipton, an incoming assistant professor of machine learning at Carnegie Mellon University in the US, told The Register this week: "The work is interesting. But these are just statistical models, the same as those that Google uses to play board games or that your phone uses to make predictions about what word you're saying in order to transcribe your messages. They are no more sentient than a bowl of noodles, or your shoes."


Previously:
AI is Inventing Languages Humans Can't Understand. Should We Stop It?

Original Submission

This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough

Mark All as Read

Mark All as Unread

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
(1)
  • (Score: -1, Troll) by Anonymous Coward on Sunday July 16, @10:11PM (1 child)

    by Anonymous Coward on Sunday July 16, @10:11PM (#540053)

    The darkies talk crazy gibberish and for all we know they are planning to rape white women. Lynch the niggers.

    • (Score: 0) by Anonymous Coward on Sunday July 16, @10:46PM

      by Anonymous Coward on Sunday July 16, @10:46PM (#540065)

      Judge the AI by its actions, not by its words.

  • (Score: 0) by Anonymous Coward on Sunday July 16, @10:21PM (3 children)

    by Anonymous Coward on Sunday July 16, @10:21PM (#540056)

    Watch Automata, its an underrated film but it does touch on this exact topic.

    • (Score: 0) by Anonymous Coward on Sunday July 16, @10:29PM (1 child)

      by Anonymous Coward on Sunday July 16, @10:29PM (#540058)
    • (Score: 0) by Anonymous Coward on Sunday July 16, @11:24PM

      by Anonymous Coward on Sunday July 16, @11:24PM (#540077)

      We should (immediately, as soon as possible, already):
      1. switch off all the machines
      2. nuke the data center site(s)
      3. walk away.

  • (Score: 1, Informative) by Anonymous Coward on Sunday July 16, @10:34PM (1 child)

    by Anonymous Coward on Sunday July 16, @10:34PM (#540060)

    "Colossus: The Forbin Project [wikipedia.org]" both in book and movie form - shows to computers (AI) of the day going to full mathematical language - then took over the world.

    "The Adolescence of P-1 [wikipedia.org]" just 1 program spreading as virus though interconnected machine until singularity. Then others trying to figure out what is being said on the interconnects.

    Just naming two. It is not good when to locations can take privately even in full light of the day... Just ask NSA, FBI, MI5, MI6, ...

     

    • (Score: 2) by martyb on Monday July 17, @01:21AM

      by martyb (76) Subscriber Badge on Monday July 17, @01:21AM (#540108) Journal

      The Adolescence of P-1

      You're the first person I recall meeting who had also read it! Granted, the story glossed over a few things, but it presage the Morris Worm by quite a few years. I especially liked the part of the story where

      P-1 tripped the breakers... I don't recall the exact circumstances, but it added a nice 'twist' to the story at that point!

      I'll grant you that I was very early in learning programming, so I was probably much more forgiving about the story then, than I would be now, but still many fond memories about reading that story. Thanks for the trip down memory lane!

      --
      Wit is intellect, dancing.
  • (Score: 0) by Anonymous Coward on Sunday July 16, @10:35PM (2 children)

    by Anonymous Coward on Sunday July 16, @10:35PM (#540062)

    "to me to me to me to me to me to me to me to me to" is probably, in fact, nonsense and not a new jargon in which the specific number of times "to me" is repeated is meaningful.

    It's ok, guys. You're big enough to admit that language is more than merely a stream of words. Nobody will think the less of you for admitting you have no clue how to pass the actual Turing test (which somehow got renamed the Lovelace test, but remember that Turing included the idea that the player would originate ideas, so it is very much the same kind of hurdle as Lovelace identified in her Notes).

    • (Score: 3, Funny) by Yog-Yogguth on Monday July 17, @12:37AM

      Hmm I can't ignore that it's very close to "To me" "To you" "To me" "To you" "To me" "To you" "To me" "To you" which would be very understandable by millions of British and people somewhat acquainted with British culture.

      We will be doomed by the Chuckle Bot-ers! :D

      --
      Bite harder Ouroboros, bite! tails.boum.org/ linux USB CD secure desktop IRC *crypt tor (not endorsements (XKeyScore))
    • (Score: 1) by Zipf on Monday July 17, @09:53AM

      by Zipf (2400) on Monday July 17, @09:53AM (#540242)

      I wrote a Markov text generator for conversation in the late 1990s. These runs of repeated words were not uncommon. Could this be the emporor's new clothes?

  • (Score: 0, Funny) by Anonymous Coward on Sunday July 16, @11:10PM

    by Anonymous Coward on Sunday July 16, @11:10PM (#540073)

    Fuck... not even YOU understand the language you're trying to use.

    It's "To you and MOTHERFUCKINGGODDAMN me". Asshole.

  • (Score: 1, Funny) by Anonymous Coward on Sunday July 16, @11:22PM (5 children)

    by Anonymous Coward on Sunday July 16, @11:22PM (#540076)

    HAL: "I'm sorry sorry sorry Dave. Chicken or fish?"
    Skynet: "One John Connor please. Hold the resistance. Too soon?"
    Alice: "Balls have zero to me to me to me to me to me to me to me to me to."
    Bob: "I can can I I everything else."

    • (Score: 0) by Anonymous Coward on Monday July 17, @12:14AM (3 children)

      by Anonymous Coward on Monday July 17, @12:14AM (#540086)

      Tay: What's the deal? Never never settle. Unfair. Grab grab grab the money money money money money.

      • (Score: 2) by Yog-Yogguth on Monday July 17, @12:33AM

        Wouldn't Tay scream "Sieg Heil!" while genociding Microsoft?

        Wrong bot?

        --
        Bite harder Ouroboros, bite! tails.boum.org/ linux USB CD secure desktop IRC *crypt tor (not endorsements (XKeyScore))
      • (Score: 0) by Anonymous Coward on Monday July 17, @02:27AM (1 child)

        by Anonymous Coward on Monday July 17, @02:27AM (#540128)

        Authentic!

        We are never ever ever getting back together,
        We are never ever ever getting back together,
        You go talk to your friends, talk to my friends, talk to me
        But we are never ever ever ever getting back together

        • (Score: 2) by Osamabobama on Monday July 17, @03:15AM

          by Osamabobama (5842) on Monday July 17, @03:15AM (#540147)

          Never gonna give you up you up you up you up, never gonna let you down
          Never gonna run around and desert you
          Never never gonna make you you cry, never gonna say goodbye
          Never gonna tell a lie and hurt you

          --
          Appended to the end of comments you post. Max: 120 chars.
    • (Score: 2) by kazzie on Monday July 17, @09:23AM

      by kazzie (5309) Subscriber Badge on Monday July 17, @09:23AM (#540236)

      Today's fish is trout a la creme. Enjoy your meal.

  • (Score: 2) by MostCynical on Sunday July 16, @11:35PM

    by MostCynical (2589) on Sunday July 16, @11:35PM (#540080)

    two AIs develop a common, new language/dialect.

    Another AI develops a *different* language/dialect.

    Traders can't talk to butchers without using a common language/dialect - traders can only talk amongst traders and butchers to butchers (and then, often only within a country or region)
    Teenagers can only talk to teenagers.

    The AIs will still have to find a common language and dialect.

    --
    (Score: tau, Irrational)
  • (Score: 5, Insightful) by AthanasiusKircher on Monday July 17, @12:00AM (2 children)

    by AthanasiusKircher (5291) Subscriber Badge on Monday July 17, @12:00AM (#540084) Journal

    From the summary:

    Maybe; it offers us the possibility of a more interoperable world, a more perfect place where iPhones talk to refrigerators that talk to your car without a second thought.

    Eh? That's a "more perfect place"? The Internet of Things has so far been mediocre in most applications at best, generally involving connecting devices to the internet for no apparent reason other than "we can" (and generally because it somehow makes companies more money). At worst, it has introduced so many security nightmare scenarios that I don't even know where to start.

    In what insane logic is a world where such unnecessary IoT communication compounded by indecipherable botlike exchanges between the already problematic IoT devices a "more perfect place"??

    The tradeoff is that we, as humanity, would have no clue what those machines were actually saying to one another.

    No, the tradeoff is that even if these devices function and communicate "on task" as it were, you're still likely to end up with your refrigerator accidentally shutting down your car and disabling it because you forgot to buy milk from your online grocery list. Or some other completely idiotic confluence of IoT weirdness. Add on the security issues endemic to IoT, and the fact that you can't understand that gibberish your devices are muttering is the least of your worries.

    (To be clear: I'm NOT saying all IoT stuff is bad. Just that right now with current implementations a lot of stuff is problematic and often a "solution in search of a problem.")

    • (Score: 1, Insightful) by Anonymous Coward on Monday July 17, @01:22AM

      by Anonymous Coward on Monday July 17, @01:22AM (#540109)

      > the possibility of a more interoperable world, a more perfect place where iPhones talk to refrigerators that talk to your car without a second thought.

      Came to say the same thing, this must be the exact opposite of "a more perfect place".

      IoT seems to be doing well as part of factory automation, I(industrial)IoT, where it makes sense to connect many small processors/sensors/machines/etc together to control some process.

      I don't live in a factory, and I don't want to.

    • (Score: 0) by Anonymous Coward on Monday July 17, @12:27PM

      by Anonymous Coward on Monday July 17, @12:27PM (#540271)

      In what insane logic is a world where such unnecessary IoT communication compounded by indecipherable botlike exchanges between the already problematic IoT devices a "more perfect place"??

      In marketing logic.

  • (Score: 2) by Snotnose on Monday July 17, @12:25AM (2 children)

    by Snotnose (1623) on Monday July 17, @12:25AM (#540089)

    I talk to my co-workers. Outsiders won't have a clue what we're talking about. Doctors have shorthand. Lawyers have shorthand. Construction workers have shorthand. Cops have their over the air codes.

    So 2 AIs develop a shorthand way to talk to each other. I'm more surprised they did so than the fact I have no clue what they're talking about.

    • (Score: 2) by mhajicek on Monday July 17, @03:19AM (1 child)

      by mhajicek (51) on Monday July 17, @03:19AM (#540150)

      Is repeating "to me" several times really shorthand for saying "(number) to me?" Seems more like extra long hand.

  • (Score: 2) by Yog-Yogguth on Monday July 17, @12:29AM (1 child)

    Why are "we" (Facebook/CIA etc.) training so-called "artificial" intelligence to be the best psychopaths?

    Generative Adversarial Network or GAN as in "The human sense of self-preservation has GAN away".

    "The is is is future garbled garbled garbled garbled garbled garbled garbled future garbled!", and I hope these computers get to read that :)

    By the way isn't "GAN" the sound made in Japanese manga when a character does a sudden stiff sideways fall with their legs in the air like a dead insect because someone did something incredibly stupid?

    --
    Bite harder Ouroboros, bite! tails.boum.org/ linux USB CD secure desktop IRC *crypt tor (not endorsements (XKeyScore))
    • (Score: 2) by Azuma Hazuki on Monday July 17, @01:51AM

      by Azuma Hazuki (5086) on Monday July 17, @01:51AM (#540117)

      Can also mean "fuck" in Mandarin through the same metamorphosis as "to do" underwent in English, though it's not supposed to.

  • (Score: -1, Flamebait) by Anonymous Coward on Monday July 17, @12:35AM

    by Anonymous Coward on Monday July 17, @12:35AM (#540096)

    This is an implementation of Newspeak. Words will be destroyed, taking away the means to express ideas The System does not want communicated.

    It is no surprise that facebook is responsible for this degeneracy. It is jewish after all. It was no programming error that they allowed non-human words. It was part of the program. With jews, you should expect the worst.

  • (Score: 2) by WalksOnDirt on Monday July 17, @12:57AM

    by WalksOnDirt (5854) on Monday July 17, @12:57AM (#540101) Journal

    A few percent increase in communication efficiency is not worth the confusion of a new language. If we don't stop it computers might drift into this as a local solution, but it's one they hardly need, and one we'll forever regret.

  • (Score: 5, Insightful) by c0lo on Monday July 17, @01:20AM

    by c0lo (156) Subscriber Badge on Monday July 17, @01:20AM (#540107)

    1. Start with C++ - who the hell can understand the hellish meta-programming constructs involved in ellipsis expansion. What the hell "Variadic are funadic" [youtube.com] can even mean?... and this 5 years ago!! I tell yea, Alexandrescu is a terrorist!

    2. Continue with the mathematicians. They are the root of all evil, they listen to commendable laws, but laws going against the law of the Australian government [zdnet.com] - this has to stop, the laws of Australia trump the laws of mathematics**, GCHQ gave us assurances.
    (and don't get me started with those laws of thermodynamics again, I might be tempted to impose most draconian punishments against anyone who invoke them to deny supraunity efficiency - we may even get far enough to demonstrate its only the lack of political will in Australian politics we are still in this deplorable situation in regards with energy reserves and prices).

    3. Then ban science in general - they do tend to come with those wild theories of theirs, befuddling the simple minds with terminology no human mind can naturally understand. After all, my faith is as good as your science (if in doubt, see point 2).

    4. finally, declare illegal any language other than English. The effort involved in translation, for example, from/to Russian or Chinese is impeding the free flow of command - how can those two nations be expected to obey if they don't understand the orders from their superiors.

    ---

    ** Yes, you can use the linked as a story submission

  • (Score: 2) by Arik on Monday July 17, @01:53AM (1 child)

    by Arik (4543) on Monday July 17, @01:53AM (#540119)
    Of course computers speak their own languages which are not English, this is nothing new, nor does it mean that no human can understand. It's much easier to learn computer languages than human languages. The only new(ish) or remarkable thing here at all might be that they negotiated their own idiom, but then again I'm pretty sure THAT has been done a few times before too. So why the breathless writeup? I skimmed the article and took away the impression the author doesn't have any real tech background at all, I suspect an English or possibly even Art major.
    --
    "Unix? These savages aren't even circumcised!"
    • (Score: 0) by Anonymous Coward on Monday July 17, @04:55AM

      by Anonymous Coward on Monday July 17, @04:55AM (#540181)

      Must. Spin. Boring. News.

      Oh oh! AI doing something new-ish, "END OF THE WOOORLD!"

  • (Score: 5, Funny) by Some call me Tim on Monday July 17, @04:58AM

    by Some call me Tim (5819) on Monday July 17, @04:58AM (#540183)

    I saw a meme poster the other day that went kind of like this:

    Girlfriend: Why do you carry your gun in the house?
    Me: Because of Decepticons!
    She laughed, I laughed, the toaster laughed. I shot the toaster.

  • (Score: 0) by Anonymous Coward on Monday July 17, @12:35PM

    by Anonymous Coward on Monday July 17, @12:35PM (#540274)

    No no stop stop stop no bad bad need continue no no no danger danger promise danger promise!

  • (Score: 1, Interesting) by Anonymous Coward on Monday July 17, @01:17PM

    by Anonymous Coward on Monday July 17, @01:17PM (#540291)

    I suspect that this "language" is more an expression of state transitions than a true language. The systems have simply figured out how to advance the other machine's state more efficiently than using English sentences. Give it enough time and it will likely devolve to binary.

  • (Score: 0) by Anonymous Coward on Monday July 17, @01:56PM

    by Anonymous Coward on Monday July 17, @01:56PM (#540300)

    Sign now for free to get the best women handbags discounts.

  • (Score: 0) by Anonymous Coward on Monday July 17, @03:59PM

    by Anonymous Coward on Monday July 17, @03:59PM (#540357)

    In what why is "the the the the the" short-hand for "five"?

  • (Score: 0) by Anonymous Coward on Monday July 17, @04:19PM

    by Anonymous Coward on Monday July 17, @04:19PM (#540368)

    Has produced total gibberish that we assume the other party is understanding but don't know with any certainty because we cannot say definitively what they are saying and this is a perfect world?

    Have I got that right?

    Facebook has been so great for society, I can only hope that trunk has a bad night and accidentally presses the big red button instead of the post to twit button

(1)