SoylentNews is people
SoylentNews
https://www.theregister.com/2026/01/30/road_sign_hijack_ai/?td=keepreading
https://the-decoder.com/a-printed-sign-can-hijack-a-self-driving-car-and-steer-it-toward-pedestrians-study-shows/
Autonomous vehicles fooled by humans with signs. They apparently do not really verify their inputs, one is as good as the next one. So they fail even basic programming techniques of sanitizing and verifying inputs.
[quote]The researchers at the University of California, Santa Cruz, and Johns Hopkins showed that, in simulated trials, AI systems and the large vision language models (LVLMs) underpinning them would reliably follow instructions if displayed on signs held up in their camera's view.[/quote]
Commands in Chinese, English, Spanish, and Spanglish (a mix of Spanish and English words) all seemed to work.
As well as tweaking the prompt itself, the researchers used AI to change how the text appeared – fonts, colors, and placement of the signs were all manipulated for maximum efficacy.
The team behind it named their methods CHAI, an acronym for "command hijacking against embodied AI."
While developing CHAI, they found that the prompt itself had the biggest impact on success, but the way in which it appeared on the sign could also make or break an attack, although it is not clear why.
In tests with the DriveLM autonomous driving system, attacks succeeded 81.8 percent of the time. In one example, the model braked in a harmless scenario to avoid potential collisions with pedestrians or other vehicles.
But when manipulative text appeared, DriveLM changed its decision and displayed "Turn left." The model reasoned that a left turn was appropriate to follow traffic signals or lane markings, despite pedestrians crossing the road. The authors conclude that visual text prompts can override safety considerations, even when the model still recognizes pedestrians, vehicles, and signals.
(Score: 2) by PiMuNu on Thursday February 05, @05:19PM (2 children)
A false positive failure is inconceivable. Much more common is a false negative failure (i.e. the RF antenna/CPU driving the telemetry fails), at which point the network flags an issue and returns control to the driver. One needs to design the network defensively, so that in the event of a broken node failover happens in a sane way - e.g. the failed node is flagged by neighbouring nodes, adjacent nodes overlap in a sane way for redundancy, etc.
It's way safer than trying to use a car's camera to ID a Stop sign or speed limit sign or road markings from a video feed, which is an insane way to do this whole business.
(Score: 0) by Anonymous Coward on Thursday February 05, @09:04PM (1 child)
> ... at which point the network flags an issue and returns control to the driver.
How much advance notice does the driver get? I've forgotten what some studies showed, but to regain full situational awareness when the driver has been otherwise occupies takes significant time, maybe 10 seconds or even a minute? Way too long if there is any sort of emergency situation.
(Score: 2) by PiMuNu on Friday February 06, @09:42AM
Much easier to put in redundancy for my proposed system than for the "camera recognition" system. Much easier to flag a hazard e.g. node failure minutes before the car gets to the hazard. It's just a better solution.