Using the watch's built-in motion sensors, more specifically data from the accelerometer and gyroscope, researchers were able to create a 3D map of the user's hand movements while typing on a keyboard
The researchers then created two algorithms: one detected when the hand dipped (to create a heat map of keystrokes) and the second used the data from the first to detect duration of pauses to guess how many keystrokes were performed by the other hand. Combining this information with a simple dictionary lookup allowed the researchers to reproduce what was typed on the keyboard.
The blog references this report at SoftPedia:
Romit Roy Choudhury, Associate Professor at ECE Illinois, together with a group of students, worked on a project called Motion Leaks (MoLe), funded by the National Science Foundation, set to be presented during this week at the MobiCon 2015 conference in Paris.
Their research consisted of a homegrown app which they installed on a Samsung Gear Live smartwatch.
Original Submission
(Score: 0) by Anonymous Coward on Saturday September 19 2015, @05:28PM
Now google will can know everything I type so as to better show me ads for products I am interested in!
Oh the wonders of the modern world!
(Score: 2) by takyon on Saturday September 19 2015, @07:25PM
Judging by market share, Apple will be the one to ad that feature.
[SIG] 10/28/2017: Soylent Upgrade v14 [soylentnews.org]
(Score: 0) by Anonymous Coward on Saturday September 19 2015, @07:56PM
I think google sells at least a couple of orders of magnitude more ads than apple does.
(Score: 2) by takyon on Saturday September 19 2015, @08:58PM
Apple controls 75% of the wristjob market.
[SIG] 10/28/2017: Soylent Upgrade v14 [soylentnews.org]
(Score: 2) by darkfeline on Saturday September 19 2015, @11:04PM
>wristjob
Is that a thing now? 10 bucks says it originated in Japan.
Join the SDF Public Access UNIX System today!
(Score: 1) by Snospar on Saturday September 19 2015, @06:24PM
Now why couldn't they have come up with something good before turning this technology to evil?
I'm amazed they can get such accuracy from the simple motions of the watch as you type, couldn't they use this type of "motion mapping" to turn the smart watch into a simple controller for Virtual Reality? It would be a huge boost to the immersion concept if you could just use your hands as normal and this was interpreted correctly and displayed within the VR environment.
I'm guessing the typing motion is so constrained that it's an easy target for detection - anything else is going to be too random to be useful.
Huge thanks to all the Soylent volunteers without whom this community (and this post) would not be possible.
(Score: 2) by takyon on Saturday September 19 2015, @07:27PM
I am on board with the idea of a "VR keyboard". It seems more feasible than laser-projected keyboards, and could enable a phone+headset desktop replacement combo.
[SIG] 10/28/2017: Soylent Upgrade v14 [soylentnews.org]
(Score: 3, Interesting) by gman003 on Saturday September 19 2015, @07:10PM
It sounds like they've gotten pretty decent accuracy out of the system, and that's with only one hand being actively monitored.
I wonder if this technique could serve as a way to have a "full-size" keyboard, for phone/tablet use, using just a pair of watches? Just "type" on an imaginary keyboard in front of you, let the accelerometers figure out what you're typing. This would only work for users with muscle-memory typing skills, but hey, they're the ones who would want a full-size keyboard the most, anyways.
There might be issues with user's mental pictures of the keyboard drifting over time, or uneven spacing, but that should be solvable, or at least mitigatable using corrective models. It probably will never be as good as an actual keyboard but it should be better than an on-screen touch keyboard at least, since it won't occupy half your screen and force you to type perpendicular to the natural way.
(Score: 2) by takyon on Saturday September 19 2015, @07:30PM
From above: I am on board with the idea of a "VR keyboard". It seems more feasible than laser-projected keyboards, and could enable a phone+headset desktop replacement combo.
I am sitting at my laptop right now, and I'm far away enough and angled so that the keyboard is within my FOV. So if I had a VR headset on with a front-facing camera, it should be able to detect where my hands are, their movements, and allow me to interact with a virtual keyboard, tapping on a desk surface. Dual smartwrists not required.
[SIG] 10/28/2017: Soylent Upgrade v14 [soylentnews.org]
(Score: 1) by VanderDecken on Sunday September 20 2015, @02:30AM
... on News at Six, hacked smart watches used as keystroke loggers for non-connected systems.
The two most common elements in the universe are hydrogen and stupidity.
(Score: 2) by maxwell demon on Sunday September 20 2015, @08:31AM
Wear your smart watch on your left arm and type with your right index finger. The watch will not even find out that you are typing at all!
Actually I'm using something in between; I use both hands and several fingers, but don't touch-type. It's surely not as efficient as touch typing, but much faster than one-finger typing, and effective enough for me (and as a bonus, I don't have too much trouble to type key combinations that are problematic for touch typists — which occur e.g. when typing C++ code on a German keyboard layout —; also I'm not forced to keep my hands in an unnatural position). I wonder if the watch could figure out what I write this way. Given that in my typing style keys near the middle of the keyboard may be typed using any hand, and also by different fingers of the same hand, depending on the letters surrounding them, I guess it's not as easy for the watch; on the other hand, my hands move around much more than for a touch typist, which in turn should help the watch.
The Tao of math: The numbers you can count are not the real numbers.