Hackers can trick a Tesla into accelerating by 50 miles per hour:
This demonstration from the cybersecurity firm McAfee is the latest indication that adversarial machine learning can potentially wreck autonomous driving systems, presenting a security challenge to those hoping to commercialize the technology.
Mobileye EyeQ3 camera systems read speed limit signs and feed that information into autonomous driving features like Tesla's automatic cruise control, said Steve Povolny and Shivangee Trivedi from McAfee's Advanced Threat Research team.
The researchers stuck a tiny and nearly imperceptible sticker on a speed limit sign. The camera read the sign as 85 instead of 35, and in testing, both the 2016 Tesla Model X and that year's Model S sped up 50 miles per hour.
This is the latest in an increasing mountain of research showing how machine-learning systems can be attacked and fooled in life-threatening situations.
[...] Tesla has since moved to proprietary cameras on newer models, and Mobileye EyeQ3 has released several new versions of its cameras that in preliminary testing were not susceptible to this exact attack.
There are still a sizable number of Tesla cars operating with the vulnerable hardware, Povolny said. He pointed out that Teslas with the first version of hardware cannot be upgraded to newer hardware.
"What we're trying to do is we're really trying to raise awareness for both consumers and vendors of the types of flaws that are possible," Povolny said "We are not trying to spread fear and say that if you drive this car, it will accelerate into through a barrier, or to sensationalize it."
So, it seems this is not so much that a particular adversarial attack was successful (and fixed), but that it was but one instance of a potentially huge set. Obligatory xkcd.
Protecting Smart Machines From Smart Attacks
A New Clothing Line Confuses Automated License Plate Readers
A Simple Sticker Tricked Neural Networks Into Classifying Anything as a Toaster
3D Printed Turtles Fool Google Image Classification Algorithm
Slight Street Sign Modifications Can Completely Fool Machine Learning Algorithms
(Score: 2) by stretch611 on Thursday February 20 2020, @03:35AM (3 children)
This hack is unlikely to get people to drive 85mph.
First, before we say that the tesla really f'd up, the article I read earlier [bloomberg.com] mentioned how this can even fool human drivers to think the speed limit is 85.
If you think of how this hack worked, and the actual numbers, there are not that many options. After all other than turning things into 8's, how many numbers can be changed in this way? Maybe a 5 into a 6 (which for speeding won't even get cops to notice you in most places. Another possibility is turning a 1 into a 7, but how many speed limits are there in places that are only 10 or 15mph... the only ones I can think of are parking lots.
While changing 2, 3, 5, or 6 into an 8 is possible, the fact is that only a single state (TX) allows 85mph anywhere. (only 7 other states allow 80mph) That is an obvious red flag for this hack. (reference: https://en.wikipedia.org/wiki/Speed_limits_in_the_United_States [wikipedia.org] )
If you are not on a rural freeway, it is a dead giveaway that the sign is wrong as even the 8 states that allow 80mph or more, the only place that happens are on rural freeways.
Even if a person/computer is fooled into thinking that this speed limit is correct, the fact is that you can't go faster than the car in front of you. So traffic will not allow you to go this fast.
Another point brough up by the article I read is:
i.e. GPS and mp software will know the actual speed limit regardless of what the signs say. If you have used any GPS system in the past 20 years you would realize this too. Most roads in the US have there associated speed limit included in GPS/Map databases.
So while this "hack" can fool a camera/sensor, in many cases, common sense and/or computer databases would dispute it quite effectively.
Now with 5 covid vaccine shots/boosters altering my DNA :P
(Score: 2) by dry on Thursday February 20 2020, @07:10AM (2 children)
In Canada, where we also have Tesla's, common speed limits are 50 in town and 80 on many highways with quite a few rural or major roads being 60.
All speeds are in km/h.
(Score: 2) by maxwell demon on Thursday February 20 2020, @01:41PM (1 child)
Which raises another question: Will those Teslas always reliably know when to interpret a road sign as MPH or km/h?
The Tao of math: The numbers you can count are not the real numbers.
(Score: 2) by dry on Thursday February 20 2020, @04:10PM
'twas another thing I was wondering. I'd think their mapping software is good enough to know where the border is but we still get the odd American who crosses the border, sees the 80 speed sign and goes flying down the highway at 140 km/h.