Stories
Slash Boxes
Comments

SoylentNews is people

posted by Fnord666 on Friday November 29 2019, @01:55AM   Printer-friendly
from the fighting-back dept.

Machines' ability to learn by processing data gleaned from sensors underlies automated vehicles, medical devices and a host of other emerging technologies. But that learning ability leaves systems vulnerable to hackers in unexpected ways, researchers at Princeton University have found.

In a series of recent papers, a research team has explored how adversarial tactics applied to artificial intelligence (AI) could, for instance, trick a traffic-efficiency system into causing gridlock or manipulate a health-related AI application to reveal patients' private medical history. As an example of one such attack, the team altered a driving robot's perception of a road sign from a speed limit to a "Stop" sign, which could cause the vehicle to dangerously slam the brakes at highway speeds; in other examples, they altered Stop signs to be perceived as a variety of other traffic instructions.


Original Submission

 
This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
(1)
  • (Score: 0) by Anonymous Coward on Friday November 29 2019, @02:16AM (3 children)

    by Anonymous Coward on Friday November 29 2019, @02:16AM (#925862)

    > the team altered a driving robot's perception of a road sign from a speed limit to a "Stop" sign,

    That sounds hard, have to get into the training set somehow(?)

    Easy way--make a cardboard "Stop" sign and tape it to the speed limit sign.

    • (Score: 2) by rigrig on Friday November 29 2019, @11:33AM (2 children)

      by rigrig (5129) Subscriber Badge <soylentnews@tubul.net> on Friday November 29 2019, @11:33AM (#925982) Homepage

      Not the actual training set, just the output.
      What they did was basically:

      1. Feed a speed limit sign image to the AI, which outputs a (high) "this is a speed limit sign" confidence value and a (low) "this is a stop sign" value
      2. Add random noise to the input and see what happens to the confidences
      3. Optimize the noise for high "stop sign" and low "speed sign" confidence, and low human noticeability
      4. Spray-paint the noise on a speed sign
      --
      No one remembers the singer.
      • (Score: 0) by Anonymous Coward on Friday November 29 2019, @06:23PM (1 child)

        by Anonymous Coward on Friday November 29 2019, @06:23PM (#926081)

        Thanks for the more complete explanation. It still sounds hard, even if it doesn't require getting into the training set.

        Someone still has to physically alter the speed sign, but it might stay altered longer if the mods truly were difficult for a human to see.

        • (Score: 0) by Anonymous Coward on Saturday November 30 2019, @02:00AM

          by Anonymous Coward on Saturday November 30 2019, @02:00AM (#926231)

          Assuming it is visible to humans at all it probably just looks like dirt smudges. I don't know about where you live but nobody washes street signs around here so something like that could easily go unnoticed.

  • (Score: 2) by legont on Friday November 29 2019, @06:06AM (7 children)

    by legont (4179) on Friday November 29 2019, @06:06AM (#925941)

    Slamming on breaks on highway because of a stop sign is artificial stupidity. If so called AI does it, we should revoke it's driving privileges, period.

    More general, no AI should be allowed on the streets until all the AI's are better than the best human.

    --
    "Wealth is the relentless enemy of understanding" - John Kenneth Galbraith.
    • (Score: 3, Insightful) by maxwell demon on Friday November 29 2019, @09:09AM (4 children)

      by maxwell demon (1608) on Friday November 29 2019, @09:09AM (#925971) Journal

      More general, no AI should be allowed on the streets until all the AI's are better than the best human.

      With that standard, you should also only allow humans on the road if they are better than the best human. Congratulations, you've just revoked all driving licenses. ;-)

      --
      The Tao of math: The numbers you can count are not the real numbers.
      • (Score: 2) by rigrig on Friday November 29 2019, @11:25AM (1 child)

        by rigrig (5129) Subscriber Badge <soylentnews@tubul.net> on Friday November 29 2019, @11:25AM (#925981) Homepage

        You mean all except mine. — 83% of all drivers

        --
        No one remembers the singer.
        • (Score: 3, Informative) by maxwell demon on Friday November 29 2019, @02:33PM

          by maxwell demon (1608) on Friday November 29 2019, @02:33PM (#926007) Journal

          Sorry, the post said better than the best human. So even if you are the absolutely best human driver of the world, you still get your license revoked, because you're not better than yourself.

          --
          The Tao of math: The numbers you can count are not the real numbers.
      • (Score: 2) by legont on Friday November 29 2019, @11:27PM (1 child)

        by legont (4179) on Friday November 29 2019, @11:27PM (#926176)

        No, AI's don't have human rights; not yet anyway.
        One can own, and once owned make, torture, and kill AI's as one pleases.

        --
        "Wealth is the relentless enemy of understanding" - John Kenneth Galbraith.
        • (Score: 2) by maxwell demon on Saturday November 30 2019, @07:05AM

          by maxwell demon (1608) on Saturday November 30 2019, @07:05AM (#926310) Journal

          I'm not aware that having a driving license has been declared a human right.

          Indeed, it is possible to lose your driving license because of repeated traffic violations. I don't think anyone has ever claimed that to be a human rights violation.

          --
          The Tao of math: The numbers you can count are not the real numbers.
    • (Score: 0) by Anonymous Coward on Friday November 29 2019, @06:29PM (1 child)

      by Anonymous Coward on Friday November 29 2019, @06:29PM (#926085)

      > no AI should be allowed on the streets until all the AI's are better than the best human.

      Interesting. I set the standard a little lower, wanting the AI to be as good as my demographic (which is very good--don't drive impaired, no smart/cell phone distraction, manual transmission, etc, etc).

      The highways are remarkably safe if you are a decent & alert driver. Given that, it's easy to be lulled into false security. I try to not kid myself that the stats are on my side. I could make a mistake, or get taken out by someone else's mistake.

      • (Score: 2) by legont on Friday November 29 2019, @11:33PM

        by legont (4179) on Friday November 29 2019, @11:33PM (#926179)

        My favorite argument, which nobody dares to take so far, is how AI driver should priotarise between a child in the car and a child running across the street in a life and death situation. Will it be disclosed by manufacturer? Would AI owners (car owners) are able to adjust this setting? Will hacking it be legal?
        Currently human behind the wheel makes this decision and accepts consequences. For example, when I see a small "animal" running across the road I typically brake. When I have a child inside my car, I run the animal over.

        --
        "Wealth is the relentless enemy of understanding" - John Kenneth Galbraith.
(1)