SoylentNews is people
SoylentNews
from the now-you-see-it-now-you-don't dept.
How a $300 projector can fool Tesla's Autopilot:
Six months ago, Ben Nassi, a PhD student at Ben-Gurion University advised by Professor Yuval Elovici, carried off a set of successful spoofing attacks against a Mobileye 630 Pro Driver Assist System using inexpensive drones and battery-powered projectors. Since then, he has expanded the technique to experiment—also successfully—with confusing a Tesla Model X and will be presenting his findings at the Cybertech Israel conference in Tel Aviv.
The spoofing attacks largely rely on the difference between human and AI image recognition. For the most part, the images Nassi and his team projected to troll the Tesla would not fool a typical human driver—in fact, some of the spoofing attacks were nearly steganographic, relying on the differences in perception not only to make spoofing attempts successful but also to hide them from human observers.
Nassi created a video outlining what he sees as the danger of these spoofing attacks, which he called "Phantom of the ADAS," and a small website offering the video, an abstract outlining his work, and the full reference paper itself. We don't necessarily agree with the spin Nassi puts on his work—for the most part, it looks to us like the Tesla responds pretty reasonably and well to these deliberate attempts to confuse its sensors. We do think this kind of work is important, however, as it demonstrates the need for defensive design of semi-autonomous driving systems.
(Score: 5, Funny) by theluggage on Sunday February 02 2020, @12:50AM (1 child)
In another experiment, researchers painted a large mural of a tunnel entrance on a wall . Approximately 1/3 of the self-drive cars tested failed to stop crashed into the wall, 1/3 drove at speed towards the image but apparently passed through the 'tunnel' unharmed and 1/3 were crushed by an unfeasibly large truck that emerged from the wall. Acme enterprises have since discontinued their autonomous vehicle program, blaming "bugs".
Meanwhile, back in the non-cartoon and allegedly real world, and thinking of defensive design:
Here in the UK, trailers often (are required to, I believe) have a sticker on the back showing their maximum permitted speed - which uses exactly the same design as the speed limit sign. So, you're following a trailer through a 30mph zone, your AI will probably think the speed limit is 50... Hopefully, it won't mistake the entire trailer for a speed limit sign, but...
There's an emerging trend for sporty cars with vivid geometric paint jobs - there's one near where I live so they are around on the road - and if you see one, the first thing you think of is "dazzle ship" (if not, google it, then you will) - i.e. camouflage. I'm almost sure that they're not going to fool a human driver, but they could easily fuck with AI.
Then there's all those advertising hoardings and company logos on vehicles... So if you're following a Fed Ex van in your Tesla and it suddenly swerves to the right, you'll know why...
Worse (certainly in the UK) a self-driving car might actually believe the roadside matrix signs on motorways... (which helpfully tell you that you should drive at 30mph on a 70mph-rated road because there was a queue here last Tuesday, warn you of "Fog" which may even have been there 7 hours ago or 'flood', which basically tells you that there is statistically less chance of there being a flood ahead than if there had been no sign... Cleverly, after 40 years of conditioning drivers to ignore matrix signs, they're being replaced with new 'smart' overhead matrix signs that actually tell you important things, like camera-enforced variable speed limits that are actually based on traffic flow, or that the former breakdown lane that had been repurposed as a traffic lane is now a breakdown lane again because someone has, you know, broken down. That last one isn't remotely funny, by the way. )
(Score: 2) by hendrikboom on Monday February 03 2020, @02:58AM
And then there are animated billboards that can be dangerously distracting, especially when they have images of moving vehicles on them so you'll look at them and see the advertisement.
They can confuse me and slow my reaction time for real dangers. I don't know what effect they haave on AIs, but I'd like them banned.
Not to mention the animated displays that are actually on advertising trucks that travel on city streets so everyone gets to see them.