The Register runs an interesting story about the Fitbit's accuracy:
Scientists have tested a pair of wearable fitness gadgets from Fitbit and found they get heart rates wrong by as much as 25 beats per minute. The study (PDF) was commissioned by law firm Lieff Cabraser, which is running a class action against Fitbit over inaccurate heart rate readings.
Not only is this a dangerous flaw, but imagine what happens to your insurance when wrong numbers are propagated in the system.
The results are not only scary, but dangerous as well for those who rely on non-medical equipment's data:
This study will scare the many athletes - serious competitors and weekend warriors alike – who aim to train at certain heart rates. If the devices over-report heart rates, users will have trained at lower heart rates than they wanted to achieve. If the devices under-report, users may be straining to reach heart rates beyond their optimal peak levels. Which can end badly.
Previously: On Fitbits, Their Accuracy, and Required Usage at Oral Roberts University
Related Stories
Oral Roberts University Requires Fitbit Ownership & Data Syncing
Oklahoma's Oral Roberts University opened in 1965 with a fitness course requirement for its newest incoming freshman and transfer students. Students first had to manually log their fitness regimens in required course notebooks. The university has now taken the concept a step further: students in the fitness course are required to purchase and use a Fitbit with heart rate-tracking capabilities (although they are not required to use a model that tracks or syncs GPS data).
ORU President William M. Wilson said: "ORU offers one of the most unique educational approaches in the world by focusing on the Whole Person – mind, body and spirit. The marriage of new technology with our physical fitness requirements is something that sets ORU apart. In fact, when we began this innovative program in the fall of 2015, we were the first university in the world to offer this unique approach to a fitness program."
Sources:
1] http://www.oru.edu/news/oru_news/20160104_fitbit_tracking.php
2] http://arstechnica.com/gadgets/2016/02/evangelical-university-requires-fitbit-ownership-data-syncing-for-freshmen/
[Read on for coverage of lawsuits regarding Fitbit accuracy]
(Score: 4, Interesting) by opinionated_science on Tuesday May 24 2016, @02:17PM
the problem with these optical measurements of HR, is there are too many parameters to foul it up - although in principle it should work!
I would recommend getting a heart strap to compare. Once you get them to agree, this problem would go away...
The thing about the strap, is it directly measures the electrical impulses of the heart (which is electrical), and therefore is a direct vs indirect (blood flow) measurement.
One of these days there will be a "smart" running top, that will give a full ECG....kickstarter anyone?
(Score: 2) by Phoenix666 on Tuesday May 24 2016, @03:01PM
I used to run with a polar watch with chest strap for that HR measurement. The chest strap did help me monitor my heart rate accurately, but the elastic of the strap felt really constricting, especially when I was chugging up hills and fighting for every scrap of oxygen and energy. Eventually the salt in my abundant sweat, initially a boon to the strap's conductance, eroded the hell out of the whole system; the back of the watch seated against the skin was eaten away.
So I like your idea of a smart running top and hope it becomes reality.
Washington DC delenda est.
(Score: 3, Interesting) by opinionated_science on Tuesday May 24 2016, @03:26PM
I wash my strap -every run! Then use hand wash (woolite) every now and again. 3 years still working fine, even the 85F running days!
Trick is wash under the warm tap *before* run, and as soon as you finish running.
So tight running tops, just need a network of electrodes, and then clip on the bluetooth ECG!!
As with most medicine, really good continuous diagnostics might catch a lot of conditions early enough to treat....
(Score: 5, Informative) by mcgrew on Tuesday May 24 2016, @05:14PM
There's a far better story about this with a LOT more information and a LOT less sensationalism at C|NET [cnet.com]. The device you used was probably as inaccurate as a fitbit unless it was a medical grade device.
What the Register omitted (and that's why I dislike that rag) was that the study was commissioned by lawyers trying to extort money from fitbit and was junk science with no rigor using all consumer-grade equipment. A real scientific study would have pitted fitbit against medical grade gear; it's very possible that the fitbit was extremely accurate while the equipment it was pitted against wasn't.
The Register implied that it was a lawsuit brought on by deaths, but C|NET reports nothing of the sort. And if some idiot had run himself to death, don't you think there would be names?
El Reg is untrustworthy.
mcgrewbooks.com mcgrew.info nooze.org
(Score: 2) by Runaway1956 on Tuesday May 24 2016, @04:47PM
"measures the electrical impulses of the heart (which is electrical)"
Plain old electrical impulses? When are we going to upgrade to modern electronics? Is that even an issue in this election year? Is MIT working on it? We really need to modernize that millenia old design we've been working with . . . .
(Score: 1, Insightful) by Anonymous Coward on Tuesday May 24 2016, @02:54PM
If 25 bpm is make-or-break, including the scary possiblity of death implied in the story, and you are relying on these devices or your phone/watch sensors, then you are doing it wrong. You should spend the top dollar and get the expensive and accurate monitors. These devices are uncalibrated toys, which can be very useful toys, but they are still toys.
(Score: 3, Insightful) by q.kontinuum on Tuesday May 24 2016, @03:14PM
They are not advertised as mere toys. In many cases wrong information is far worse than no information.
Registered IRC nick on chat.soylentnews.org: qkontinuum
(Score: 3, Insightful) by ikanreed on Tuesday May 24 2016, @04:37PM
In many cases wrong information is far worse than no information.
See: US Politics.
(Score: 1, Insightful) by Anonymous Coward on Tuesday May 24 2016, @07:51PM
But they're not advertised as accurate monitors either. You can go to your local big box hardware store and purchase the POS table saw they offer, but don't complain when you've burned through a bunch of expensive hardwood trying to do fine cabinetry and getting crappy results. Cheap and popular doesn't equate to accurate, and unless they're making any kind of claims along those lines beyond that it monitors one's heart rate, then you get what you pay for.
(Score: 2) by https on Wednesday May 25 2016, @02:02AM
The range of the cardio zone, between the mere fat-burning exercise zone and the call-an-undertaker zone, is not exactly fixed - but it does narrow as you get older, and even at the age of 20 it's only about 25 BPM across.
It is absolutely make-or-break. Sadly, few people wear watches any more, so self-monitoring on the go is cumbersome and error prone.
Offended and laughing about it.
(Score: 0) by Anonymous Coward on Tuesday May 24 2016, @03:05PM
Not only is this a dangerous flaw, but imagine what happens to your insurance when wrong numbers are propagated in the system.
We'll just hack the damn stupid things?
Truly: The Geek shall inherit the earth.
(Score: 4, Informative) by WizardFusion on Tuesday May 24 2016, @03:12PM
I actually read the story as I have a Fitbit Charge HR. This makes the whole story rubbish..
The study didn't use a colossal sample – just 43 people were tested – and only tested subjects once.
(Score: 1) by AlphaMan on Tuesday May 24 2016, @05:40PM
Even without reading the article, it occurred to me that these are lawyers, not statisticians. The important thing to figure out is if the 25 bpm error was transient or persistent, and, if transient, were the errors unbiased?
(Score: 2) by Immerman on Wednesday May 25 2016, @03:21AM
Does it really matter? To prove the things are dangerous, or at least generally unfit for purpose, all you have to do is show that there's a noteworthy segment of the population for which they generate substantially incorrect results. Unless they pre-selected their sample from people they believed would generate bad data, any frequent errors are pretty damning, if not necessarily scientificaly rigorous.
It's when trying to prove safety that large sample sizes become more important, specifically because it's important to catch the "corner cases". "Proving" safety experimentally is like proving that all crows are black - strictly speaking it can't be done, but a sufficiently large sample size lets you narrow the margin of error to acceptable levels. Disproving it is easy though - all you have to do is find one white crow. Or in this case, a bunch of people who generate bad results.
And in this case it sounds like proving safety wasn't even seriously attempted - as a non-medically certified monitoring device it didn't need to pass even mildly rigorous accuracy tests. Which is fine so long as it's marketed as a novelty - nobody expects mood rings to actually tell your mood. But it becomes a real problem as soon as it's marketed in contexts where inaccurate data may cause health or insurance problems. Which covers, at a guess, the vast majority of people using it. I rather many people bought a "health monitor" because they were idly curious about their heart rate through the course of the day.
(Score: 2, Insightful) by Anonymous Coward on Tuesday May 24 2016, @03:35PM
Before the Fitbit was invented, humanity was unable exercise, and nobody could train for distance running or sports.
(Score: 0) by Anonymous Coward on Tuesday May 24 2016, @06:03PM
It's like nobody could play games on their PC until there were $600 GPUs.
People for whom that is one of their big things, are willing to pay more and do more work than seems sensible.
(Score: 0) by Anonymous Coward on Tuesday May 24 2016, @03:43PM
Fitbit adding a disclaimer to their packages and radio ads.
This is not another Therano because nobody in their right mind should take Fitbit's HR numbers (or Precor's or LifeFitnesses', from the treadmill at the gym) that seriously. These numbers are intended as a general guide, with little or no clinical significance.
(Score: 4, Insightful) by korger on Tuesday May 24 2016, @04:20PM
Listen to your body, not Fitbit or other wearable boondoggles. Your body knows what's good for it, and it will tell you, just listen to it. Do so much exercise that still feels good, don't push yourself until it becomes painful. I don't get this obsession with achieving a given heart rate--what is that supposed to accomplish? This is just another instance of people doing it because technology allows it, and they don't think twice if that technology makes sense.
Using a wearable device for working out is good for monitoring progress, nothing else. You shouldn't assume that it is your personal trainer.
(Score: 0) by Anonymous Coward on Tuesday May 24 2016, @04:30PM
There's something that's been going around about an optimal heartrate for burning fat. It's yet another way that people try to hack (minmax?) fitness so it'll fit into its designated 30 minute spot on the schedule.
(Score: 3, Funny) by bob_super on Tuesday May 24 2016, @04:42PM
Too often, people minmax by dropping INT and WIS for a little more CON.
Turns out God is one of those nasty DMs who will make you regret that lower saving throw...
(Score: 3, Touché) by mcgrew on Tuesday May 24 2016, @05:16PM
Excuse me, but would you mind translating those meaningless acronyms into English for those of us not up on ghetto slang?
mcgrewbooks.com mcgrew.info nooze.org
(Score: 4, Informative) by bob_super on Tuesday May 24 2016, @05:42PM
AD&D (dungeons and dragons role playing game) has characters defined by 6 attributes: STRengh, CONstitution, INTelligence, WISdom, CHARisma and DEXterity.
Some versions of the game decide those randomly, others allow spreading points between the attributes to get a good score in the most important attribute for your chosen type (INT for wizards, STR for fighters...)
Similar systems have been reused by many many games, tabletop or computer-based.
MinMaxing is optimizing your character attributes, skills, equipment to get the max or min rules-allowed values (most damage, most hit points, biggest F'ing Fireball...). Because of game balance mechanics, you have to drop other characteristics pretty low: Your Barbarian will slice an elephant in two blows, but he's dumber and crazier than an oversized red baseball cap.
Game Masters (or Dungeons Master, DM) have to preserve the balance of the game, and will usually make sure that you do encounter things you can't slice in half before they unleash a mind effect that people with low Mental attributes will fail to resist (a Savings Throw to escape a mind effect is hard when you have MinMaxed for raw damage).
In short, the initial comment was about how people trying to max their STR and CON are not the wisest...
(Score: 2) by Capt. Obvious on Wednesday May 25 2016, @09:39AM
WIS didn't impact saving throws until D&D 3+ (around 2000 or so). AD&D and AD&D 2 (last millennium) had saving throws based just on class. A high WIS just granted immunity to a bunch of spells, and a high INT granted an escalating immunity to illusions
(Score: 2, Insightful) by Anonymous Coward on Tuesday May 24 2016, @05:53PM
Naah, just pump it all into CHA along with choosing lawful evil, seems that strategy works the best in this world :)
(Score: 3, Insightful) by Runaway1956 on Tuesday May 24 2016, @05:01PM
"o so much exercise that still feels good, don't push yourself until it becomes painful."
Hmmm. Not really. You've heard "no pain, no gain" I'm sure. If you're seriously building your body up, whether it just be for health, or competition, you gotta hurt some. Without any pain, you're just maintaining the status quo. But, an intelligent workout limits the pain to a reasonable threshold. "Reasonable" being a lot more pain than the average lard-arse is willing to endure, but considerably less than a bunch of crazy, drunken rednecks are willing to endure.
You are right though - if you've pushed yourself to the limit, and every breath burns, every muscle burns, your heart is beating it's way out of your chest, and your legs weigh about 1/4 ton each, but the silly gadget says you haven't reached your goal, then it's time to throw the gadget away.
(Score: 2) by gidds on Wednesday May 25 2016, @12:21PM
That's easy, then: none.
Seriously, I have never felt good during or after exercise. Mild exercise just makes me a little tired. Heavier exercise makes me hot, breathless, and very tired. Extreme exercise makes me (metaphorically) want to die.
At school, I remember a PE teacher talking about this 'warm glow' you were supposed to get after exercise. I've no idea what that's supposed to be, but I've yet to experience it.
Is there something wrong with me? Or is all this talk of exercise 'feeling good' simply a lie that people tell themselves?
[sig redacted]
(Score: 0) by Anonymous Coward on Wednesday May 25 2016, @01:07PM
The "runner's high" was presumably what was being referred to as being that warm glow. It has something to do with endorphins being released due to the strain of exercise and the resulting mild euphoria felt. I've never experienced it myself.
(Score: 1) by korger on Wednesday May 25 2016, @08:59PM
I can also confirm what your PE teacher has said. It's not what I was talking about though; I was referring to how you feel during the exercise, whereas he was talking about how you feel hours after the exercise. It's true, there is a certain warm glow some people feel for hours after a good workout, I sometimes feel it the whole next day. I'm not sure what's causing it; it may be endorphins [wikipedia.org], but in any case it's very rewarding. However, I only started to feel this after I had been exercising regularly for a whole year; this change in the body doesn't come quickly.
(Score: 2) by MichaelDavidCrawford on Tuesday May 24 2016, @04:22PM
If I stop running to take my pulse the old-fashioned way, my pulse does not drop that quickly during the brief rest.
While I do need to lose my love handles these days, there have been plenty of times that I've been in excellent shape. I've never used any manner of electronic gadget to enable that.
A while back I heard a talk by a guy who was boasting about wearing a half-dozen Wearable Gadgets 24/7. So The Series Of Tubes knows about it whenever he takes a damn leak.
I once interviewed for a contract at Nike where I would write the Mac OS X app that would talk to accellerometers built into their shoes. I expect the reason I didn't get the gig was that I made plain that I regarded that as a really, really stupid idea. You'd never see a pro athlete or an Olympian wearing shoes like that.
On the other hand, Nike's Beaverton campus is really quite upscale...
Yes I Have No Bananas. [gofundme.com]
(Score: 0) by Anonymous Coward on Tuesday May 24 2016, @04:35PM
I regarded that as a really, really stupid idea
I have come to the conclusion most ideas are rather bad or stupid. It is my job to make it work. What do I care if people want to waste money?
(Score: 2) by MichaelDavidCrawford on Tuesday May 24 2016, @05:20PM
Filter error: Comment too short
Yes I Have No Bananas. [gofundme.com]
(Score: 0) by Anonymous Coward on Tuesday May 24 2016, @06:31PM
Thats was sort of my point. Almost *all* of them are stupid ideas. Then I found out there are stupid people out there who come off large sums of cash for stupid things. Now given they are kinda stupid I can pretty much do whatever I want. Just as long as I make their stupid idea work.
(Score: 4, Insightful) by isostatic on Tuesday May 24 2016, @04:43PM
I once interviewed for a contract at Nike where I would write the Mac OS X app that would talk to accellerometers built into their shoes. I expect the reason I didn't get the gig was that I made plain that I regarded that as a really, really stupid idea.
Glad I'm not the only one.
Life's too short to do work you hate. A typical working stiff spends 40 hours a week at work - that's over 1/3rd of your waking life. If you can help it, there really is no point in not doing something you like, even if it means you can afford to buy a 50" TV instead of a 30" or whatever people spend money on.
(Score: 2) by Runaway1956 on Tuesday May 24 2016, @05:05PM
"that's over 1/3rd of your waking life"
I read "Nike" and "shoes", then I scrolled down and read your sentence as "that's over 1/3rd of your walking life".
And, naturally, I says, "WTF?" Waking life - got it.
(Score: 2) by SanityCheck on Tuesday May 24 2016, @05:19PM
The program is complete
(Score: 1) by OrugTor on Tuesday May 24 2016, @04:45PM
I laughed out loud. The idea that you went to an interview for a contract that would have paid handsomely and told them their idea was really stupid. Implying that the people and the company were stupid too. Anyway, couldn't they just get some third-world children to program the app for next to nothing?
(Score: -1, Flamebait) by Anonymous Coward on Tuesday May 24 2016, @09:21PM
Ah, our resident idiot Michael posting about himself again.
(Score: 0) by Anonymous Coward on Tuesday May 24 2016, @10:53PM
You've got that backwards. Those two groups are, by far, the easiest to get to try all sorts of nonsense, provided that it doesn't actually hinder their performance. First off, they get them for free, so, why not? Secondly, these people are all bunched together with regards to their abilities where a very modest performance gain can make a big difference to them. For a fat-ass like me, if I improve my running speed or endurance time by 5%, that doesn't mean shit because I'm still finishing near the back of any race I'm in. However, a top-notch athlete would run over their mother for a 5% improvement in something. That's why you'll find them wearing copper bracelets, copper-infused clothing, "shark-like" head-to-toe swimsuits, etc.
(Score: 2) by VLM on Tuesday May 24 2016, @04:40PM
Which can end badly.
Can it?
Not in a cardiology patient or a soon to be cardiology patient but just healthy J Random Hacker?
I googled around and can't get a straight answer. What little I can find is that aerobically beyond a certain minimal level of fitness (good by amerifat standards, not so amazing in an abstract sense) you're likely to go thermal limiting. I can assure you as a dude living in the frozen north you can go thermal limit even coatless snow shoveling. In fact thats a highly effective way to die of hypothermia once you slow down and are all sweaty and wet. Even better you can get heat exhaustion AND frostbit ears and fingers and the same time. Fun times.
One wikipedia article implies due to various nervous system issues unless you're "broken" its not possible to pump over two hundred and something because of a lot of neuron blah blah basically in EE terms your totem pole output transistors are current limited by design and feed a whomping big capacitor so the surface of a nerve could be modeled as a low pass filter with a rather steep rolloff. All the components are active not passive so this is not a perfect linear circuits analogy. Of course if things are broken and pulses are wandering around asynchronously the doctors like to pick a random ish high number that faulty instruments report, but the EE types would be like you just have a herd of random unsynchronized oscillators so you don't really have "a" beat rate. Whats the frequency of a white noise source or a heart in fibrillation? Well...
Some reddit (reddit ranges from great to trash) along the lines of docs only see sick people with high heart rates so they get out of whack but healthy people can't be hurt by high rates. Of course the easiest way to find out if you're not healthy is to push the limit and see if you end up in the ER or morgue. Logic does dictate that its very unusual for high heart rate to kill people, given that healthy people have spent a hell of a lot of time over the millennia doing warfare and track -n- field and aside from obvious heat casualties and dehydration casualties its incredibly unusual for a healthy person to be scared to death or fall over dead when they sprint or whatever.
Some reddit also got into the engineering where peak flow rate happens at 160 or so BPM for a variety of fluid dynamic reasons and both your muscles and heart depending on flow rate, running above 160 or so is long term unsustainable. Much like flying an airplane you can go into fast-flight-mode at one angle of attack and slow-flight mode at a higher angle of attack at the same power setting so 190 BPM isn't any more productive than 100 or whatever (its not simple and symmetric like most fluid dynamics). Long term unsustainable means your muscles stop working due to cramping or just outright stopping, which tends to help with the flow rate in the medium term, and if the heart nutrient intake drops enough it slows down (not to mention pain) so its all kinda self limiting. Anyway that's why people doing the 50 meter sprint don't all flip the heart rate to 1000 bpm and therefore die. Superficially with a linear model ignoring fluid dynamics and assuming constant pump volume that's about what you'd need to keep up a 50m pace for a marathon distance, and your innards don't know any better so they go to a max out until everything catches back up.
Now something like a core body temperature meter that falsely read low would in fact result in bodies stacked like cordwood as thermal casualties are a big problem. Or a dehydration meter that always read high water levels, that could really kill a lot of people. But this is just heart rate... The docs on reddit, some of whom might even be docs, saw the heart rate in healthy people as being only a little more important than astrological sign, at least compared to all this EKG stuff they have a major interest in. Apparently WRT health the shape of the waveform is immensely more important than the frequency of the waveform, or at least that was my interpretation of people on reddit who might be doctors.
(Score: 2) by Gravis on Tuesday May 24 2016, @06:00PM
The results are not only scary, but dangerous as well for those who rely on non-medical equipment's data:
here's the problem right here, it's NOT MEDICAL EQUIPMENT! as such, the resulting information has no requirement of being terribly accurate but only representative in some measure. did they say it can measure your heartbeat? yes and it can! can it do so very accurately, IT'S NOT MEDICAL EQUIPMENT SO STOP TREATING IT LIKE IT IS!
(Score: 0) by Anonymous Coward on Wednesday May 25 2016, @01:13PM
Agreed. However, what are the odds that this hue and cry is made as an attempt to drag a class of interesting gadgets into the realm of "medical equipment", a realm I understand to be highly regulated at the point of US government guns? Considering the study was of questionable quality and conducted by lawyers, I'd figure the odds to be good.
(Score: 2) by PizzaRollPlinkett on Tuesday May 24 2016, @06:58PM
Yeah, "wrong numbers are propagated in the system" has always been my concern. Vast databases are being compiled that affect people's lives, but there's no real recourse for individuals when background checks, credit scores, FBI databases, and so on have incorrect information. How do you clear your name, your credit, and you employment status when impersonal companies that deal only with other businesses compile information you don't know about in many cases? A lot of public databases have laughably wrong information about you when you look yourself up. Are these private databases any different?
(E-mail me if you want a pizza roll!)
(Score: 2) by bob_super on Wednesday May 25 2016, @12:05AM
Given the chaotic nature of the whole system, I believe that 25bpm error in the input is definitely above the Butterfly Effect threshold.
Watch out for the LawyerNado!
(Score: 2, Informative) by oldmac31310 on Tuesday May 24 2016, @09:48PM
It is 'exercise' damn it.
(Score: 0) by Anonymous Coward on Wednesday May 25 2016, @11:07AM
One big piece of evidence against them was their own fitbit.
So they're trying to poke holes in it.