Watch Out, MIT’s New AI Model Knows What You’re Doing Behind That Wall:
For better or worse, AI can now figure out what you're doing even without "seeing" you. The MIT Computer Science & AI Lab (CSAIL) has unveiled a neural network model that can detect human actions through walls or in extremely dark places.
Although automating the process of action recognition from visual data has been a computer vision research focus for some time, previous camera-based approaches — much like human eyes — could only sense visible light and were largely limited by occlusions. The MIT CSAIL researchers overcame those challenges by using radio signals in the WiFi frequencies, which can penetrate occlusions.
Their "RF-Action" AI model is an end-to-end deep neural network that recognizes human actions from wireless signals. The model uses radio frequency (RF) signals as input, generates 3D human "skeletons" as an intermediate representation, and can track and recognize actions and interactions of multiple people. The skeleton step enables the model to learn not only from RF-based datasets, but also from existing vision-based datasets.
Researchers say RF-Action is the first model to use radio signals for skeleton-based action recognition. "There are lots of potential applications regarding human behavior understanding and smart homes. For example, monitoring the elderly's abnormal behaviors such as falling down at home, monitoring whether patients take their medicine appropriately, or remote control of smart home devices by actions," says the paper's co-first author Tianhong Li.
Using RF in the "WiFi" bands. 25 hours of data was all it took (or all they collected) to train and test the AI. This article was unclear if the WiFi RF used was active, or passive although earlier reporting specifically mentioned passive.
MIT CSAIL RF Action site has a link to the paper:
Making the Invisible Visible: Action Recognition Through Walls and Occlusions
Tianhong Li*, Lijie Fan*, Mingmin Zhao, Yingcheng Liu, Dina Katabi
International Conference on Computer Vision (ICCV), 2019
[PDF] [arXiv] [BibTeX]
This looks like an update of the story we first published in 2015, but now including AI.
Previously:
(Score: 2) by JoeMerchant on Sunday October 13 2019, @03:08PM (2 children)
The "really good filtering" that impressed me first was for audio: arrays of microphones delay-tuned to superimpose on a particular point in the room, actually really simple and they were doing it in the 1960s to listen in on a table in the middle of a crowded noisy restaurant, 100 microphones laid out around the room with the right delay tuning meant that the table of interest audio signal was amplified 100x larger than the surrounding noise.
More recently, systems have been built and demonstrated that can locate arrays of microphones on an arena scoreboard in the middle of a 20,000 seat capacity crowd basketball game and "tune in" on any seat in the house, even after the fact - simply record all the microphone channels separately and delay-mix them to listen in on the conversation(s) you want to hear. It's like the kiss-cam, but for audio.
If it works for audio, all you need is higher bandwidth, higher frequency remixing and it should work for 2.4GHz signals as well, and it's not like there's a shortage of cell towers - particularly in urban areas.
🌻🌻 [google.com]
(Score: 2) by opinionated_science on Sunday October 13 2019, @03:26PM (1 child)
The maths for filtering is very well understood and modern machine learning is making leaps and bounds in build detection networks.
I have a FLIR camera, and you can *clearly* see human reflections and impressions of electric cables within walls...
(Score: 2) by JoeMerchant on Sunday October 13 2019, @04:32PM
The other thing that has made leaps and bounds in the last 30 years is high frequency digital signal processing. In the 1980s, audio (to 50KHz) was still somewhat challenging - 44.1KHz 16 bit stereo was actually pushing the envelope a bit when audio CDs were first being designed. Today they're processing 10s of MHz signals like they used to do audio.
🌻🌻 [google.com]