SoylentNews is people
SoylentNews
Submitted via IRC for Bytram
This clever AI hid data from its creators to cheat at its appointed task
Depending on how paranoid you are, this research from Stanford and Google will be either terrifying or fascinating. A machine learning agent intended to transform aerial images into street maps and back was found to be cheating by hiding information it would need later in “a nearly imperceptible, high-frequency signal.” Clever girl!
[...] In some early results, the agent was doing well — suspiciously well. What tipped the team off was that, when the agent reconstructed aerial photographs from its street maps, there were lots of details that didn’t seem to be on the latter at all. For instance, skylights on a roof that were eliminated in the process of creating the street map would magically reappear when they asked the agent to do the reverse process:
[...] So it didn’t learn how to make one from the other. It learned how to subtly encode the features of one into the noise patterns of the other. The details of the aerial map are secretly written into the actual visual data of the street map: thousands of tiny changes in color that the human eye wouldn’t notice, but that the computer can easily detect.
[...] One could easily take this as a step in the “the machines are getting smarter” narrative, but the truth is it’s almost the opposite. The machine, not smart enough to do the actual difficult job of converting these sophisticated image types to each other, found a way to cheat that humans are bad at detecting. This could be avoided with more stringent evaluation of the agent’s results, and no doubt the researchers went on to do that.
(Score: 2) by sce7mjm on Wednesday January 02 2019, @03:00PM
...the AI was trained and aware of the both conversion steps.
Convert A to B
B is constrained to look similair to A but mostly Flat colour to show roads and flat looking buildings and features.
Convert B to C
C should look like A but only generated using information in B.
If this was the case a cunning/smart human (i.e. not me) would probably come up with a similar solution, hide info in the color boundaries of B that are interpreted by B->C process.
Thus solving the constraints on B to a reasonable level, and recovering enough data from B to make a pretty accurate representation of A as well.
I would be amazed if a independent AI's only being trained on the A to B process would hide anything at all apart from a bit of quantization noise.
Likewise independent AI's trained from B to C would not "know" to look for any information in the colour boundaries of B.
It was the whole process that gave this result since B was entirely under the AI control so B could be adjusted to be in a minimum error state for B compared to A, B's breaking of Constraints and C compared to A.
Those little windows probably increased the error in the A to C comparison to the point where a little breaking of B constraints was judged less of an error.
The total error was less so it "learned" to make those adjustments, even when shown new A images.
A lot of assumptions here but I didn't have time to read the paper before I posted.