SoylentNews is people
SoylentNews
According to a study conducted through heartbeat measurement app Cardiogram and the University of California, San Francisco, the Apple Watch is 97 percent accurate in detecting the most common abnormal heart rhythm when paired with an AI-based algorithm.
The study involved 6,158 participants recruited through the Cardiogram app on Apple Watch. Most of the participants in the UCSF Health eHeart study had normal EKG readings. However, 200 of them had been diagnosed with paroxysmal atrial fibrillation (an abnormal heartbeat). Engineers then trained a deep neural network to identify these abnormal heart rhythms from Apple Watch heart rate data.
Cardiogram began the study with UCSF in 2016 to discover whether the Apple Watch could detect an oncoming stroke. About a quarter of strokes are caused by an abnormal heart rhythm, according to Cardiogram co-founder and data scientist for UCSF's eHeart study Brandon Ballinger.
Yes, but can the Apple Watch then pace you or shock you?
(Score: 2) by kaszz on Sunday May 14 2017, @11:09AM (6 children)
The shocking part comes with version 2.0 which will print the Apple wallet bill right at your wrist on a paper scroll. Looking at it will revive any dead worker slave for pay time ;-)
On a more serious note. How hard is this to implement as open source? the ECG diagram five states PQRST should be possible to analyze using phase and frequency analysis. Then comes the next question as how much computer performance that is needed to do these tasks. But supposedly Apple S1 single core 520 MHz 32-bit ARMv7, RAM 512 MB, storage 8 GB is enough. Apple S2 is two cores at 780 MHz.
Another issue is if it will continuously analyze heart rhythms or if it will sample 10s and then sleep for say 190s to save battery.
What alternatives is there to the Apple Watch that won't glue you to another walled garden either?
(Score: 2) by theluggage on Sunday May 14 2017, @12:50PM (5 children)
Then comes the next question as how much computer performance that is needed to do these tasks.
Sample it, upload it to iTunes Music Match and see if what comes back is Rock'n'Roll (normal) or Experimental Jazz (atrial fibrillation).
Seriously (disclosure: I have permanent AF) it is not a subtle effect that can only be spotted by experts poring over ECGs - I'm not saying it would be trivial to generate reliable diagnoses that meet clinical standards, but it ain't gonna to be a job for Deep Blue. A watch that could detect it would be good for those with intermittent AF that won't perform on demand for the doctor. Might have got me an earlier diagnosis before it set in permanently.
(Score: 2) by Runaway1956 on Sunday May 14 2017, @01:25PM (4 children)
OK - question. You have the specific condition mentioned in the article. My lifelong condition, has been a heart murmur. I suppose that given time, Apple will detect heart murmurs and other problems. But, the question is, do you really want to know every time your heart does a little out-of-synch dance step?
Speaking personally, I knew about my condition when I was a kid. But, I had things to do, places to go, and people to do. Among those things, was joining the Navy, and seeing the world. When asked at my induction physical if I had any heart conditions, I just kept my mouth shut. Hey, if they figured it out, I'd just say I didn't know, right? Well, I slipped by that physical, and I slipped past more physicals at boot camp. A corpsman out in the fleet finally caught it, and he asked if I knew that I have a murmur. "Huh, what? What is that?" He sent me for another physical, with a real doctor, who couldn't find it.
Now, if I had spend my youth wearing a heart monitor, that told me every time my heart stuttered, I'm certain that my life would have been very different.
So, I'm wondering how many people really WANT to monitor the workings of their bodies? It's sure to make you worry - and probably needlessly.
And, hey, what about false positives? An especially bad case of gas might convince the monitor that you're dying! (or, it might interpret a bad heart attack as a case of gas, lol)
(Score: 3, Informative) by AthanasiusKircher on Sunday May 14 2017, @03:04PM
It's sure to make you worry - and probably needlessly.
It's worse than that in this case. As I noted in a post below, the algorithm in this study actually had a 75% false positive rate (that is, if it flagged you for "abnormal heart rhythm," there's a ~75% chance you don't have one at all).
Given other characteristics of the test (i.e., low false negative rate), this could still work okay as a preliminary diagnostic tool in a medical context. But most people (frankly, including most doctors) don't have a good intuitive sense about how stats work in medical tests. And a lot of people who ultimately end up with "false positives" will go around worrying about why they were flagged -- even if it's just a bad test. The anxiety produced by such tests can actually lead to even more negative health outcomes.
That's one of the reasons why doctors sometimes recommend LESS screening -- they come under fire by people who say, "But, but, but... we need to screen young women for breast cancer all the time, because even if we only find one case, we save a life!" But -- aside from unnecessary invasive confirmation tests like biopsies -- studies on false positives show that they can become an obsession for some people; studies show that women with false positives are as psychologically damaged as women who actually have breast cancer for about 6 months after a false screening, and measurable negative psychological effect persist for an average of 3 years. Anxiety can lead to depression, depression can lead to bad habits (smoking, obesity, etc.).
Excessive screenings where there are high false positive rates can thus actually CAUSE more medical problems than they solve. I have no doubt that this Apple Watch algorithm at its current accuracy level would cause more harm than good, given that there are other easy and non-invasive tests for abnormal heart rhythm with a much lower false positive rate.
(Score: 2) by kaszz on Sunday May 14 2017, @03:06PM
The point is to monitor so that a health care facility can fix the problem before it becomes a real problem. Not to make people unable to do what they want. This way people won't even have to tell what the real deal is. They can always say it's for "workout performance monitoring" etc.
(Score: 3, Interesting) by theluggage on Sunday May 14 2017, @03:08PM (1 child)
OK - question. You have the specific condition mentioned in the article. My lifelong condition, has been a heart murmur. I suppose that given time, Apple will detect heart murmurs and other problems. But, the question is, do you really want to know every time your heart does a little out-of-synch dance step?
Correction: I (now) have permanent AF (and if your doctor doesn't spot that in 30 seconds you should probably check their pulse) - TFA is talking about the intermittent form. But no, I'm not going to buy an Apple Watch to get reminders of a condition I already have and which doesn't need day-to-day management.
However, in my case intermittent AF was the precursor of a bigger tachycardia problem that eventually led to heart failure (from which I recovered, but it wasn't funny) and permanent AF. However, when I initially went to my doctor having experienced bouts of irregular heartbeat, nothing showed up on the day and he literally said "there's nothing wrong with your heart". If I'd been able to gather evidence then I'd have pursued it more aggressively.
Current solution (you've probably had it) to data gathering is that they loan you a portable heart monitor the size of a cigarette packet, with about 5 self-adhesive electrodes which fall off after an hour because the hospital has bought them from the lowest bidder.
What would be more useful in my case is a tachycardia alarm that might help regulate my beta-blocker dose. A Pre-condition of that is that it should be able to distinguish AF (not a problem if you know about it) from tachycardia (definitely a problem)...
Slightly O/T: ever looked at the manual for one of those home blood pressure monitors? "May give erroneous results in the presence of irregular heartbeat". (Translation: if you have AF take 3 readings and write down the one that is consistent with you still being conscious). I'm absolutely, positively sure that the doctors and hospitals that use automatic BP monitors have special industrial-strength versions that don't have that problem (however, I did notice that my cardiologist took BP the old fashioned way).
Now, if I had spend my youth wearing a heart monitor, that told me every time my heart stuttered, I'm certain that my life would have been very different.
But then there are always cases of apparently healthy people who discover their heart condition when they drop dead during military training or on the sports field... I guess, people should be able to make their own choice about how much they want to know.
However, yeah - there's an ethical elephant in the room that people don't seem to mention - even when discussing perception of risk:
Say you are an individual and you see evidence that taking magic pill X reduces your chance of acute conflobulitus from 2:100 to 1:100. Cue headlines on "New pill halves the risk of conflobulitus!" and subsequent responses by sensible people of how misleading it is to describe increases in small risks like that and how it would be quite rational to decline the kind offer of pill X or, at least, make serious enquiries about the possible side effects.
However, say you are now a healthcare provider serving millions of people and dealing with 20,000 cases of conflobulitus. Reducing that to 10,000 really is going to have an impact on your conflobulitus budget, so from your point of view that is a big deal (even if you don't get a kickback from the makers of Pill X). The numbers don't really matter: the point is that doctors, hospitals, insurers mostly only deal with ill people which gives them a much smaller denominator for their risk assessments.
(Score: 2) by Runaway1956 on Sunday May 14 2017, @04:14PM
OMG! Conflobulitus! I was bitten by mosquitos in Africa!
Seriously, thanks for the answer. You've put things into perspective, pretty nicely.
I'll add that "think of the children" plays in there somewhere. Risks that I was quite happy to take, for myself, look entirely different if it's the kids who are at risk. Or, today, risks that are acceptable even for my kids, are not acceptable for the grandchildren. Funny how perspective changes, with time.
(Score: 2) by zeigerpuppy on Sunday May 14 2017, @11:57AM
AF or Atrial Fibrillation is a very common abnormal rhythm of the heart. Some people have AF that starts and stops (paroxysmal) and other have it all the time. The risk of stokes is elevated in the paroxysmal form especially as clots can form and move when the rate reverts to normal. The rhythm that results from AF is irregularly irregular. Essentially the usual timing cue coming from the left atrium is disrupted and the heart reverts to spontaneous muscular rhythm. It's a stupendously easy rhythm to pick up and I'm even surprised they'd use AI. I imagine you could miss quite a lot of beats with the sensor and still have a really good detection rate for AF, so this sounds like self-congratulatory BS. Passive monitoring is cool and may have some great benefits, but there's nothing particularly special about Apple's hardware or implementation as far as I can see.
(Score: 2) by Gaaark on Sunday May 14 2017, @12:26PM (1 child)
"Yes, but can the Apple Watch then pace you or shock you?"
No, but it knows EXACTLY where you are and if any
NSA AgentsApple fan doctors are around you.--- Please remind me if I haven't been civil to you: I'm channeling MDC. ---Gaaark 2.0 ---
(Score: 2) by kaszz on Sunday May 14 2017, @01:23PM
The NSA agents will cure anyone from opposition. As soon as a cardiac event happens. Suddenly emergency services will have "computr probmes".. Something with Widows 11.
(Score: 2) by coolgopher on Sunday May 14 2017, @12:35PM
But I hear the user interface can be pretty shocking...
(Score: 0) by Anonymous Coward on Sunday May 14 2017, @01:45PM (9 children)
Screening is all nice and well, but let's get the numbers together:
- 6158 people
- 200 of which had been diagnosed with paroxysmal atrial fibrillation ('unhealthy')
-> 5958 of which had been diagnosed with no paroxysmal atrial fibrillation ('healthy'). (Note: As I don't know any verified numbers, I take these numbers as representative for the general populace.)
- Accuracy of 97 % (as they don't specify, I take that as each sensitivity and specificity)
So follows:
- Of 200 affected ('unhealthy') people, 97 % (194 people) are getting the correct positive diagnosis
- Of 5958 healthy people, 3 % (179 people) are getting a false positive diagnosis
-> If you get a positive diagnosis, there's only about a 50:50 chance (179:194) you actually suffer from paroxysmal atrial fibrillation
The diagnosis doesn't help much, especially if you consider that paroxysmal atrial fibrillation only increases the risk of strokes, it doesn't somehow make them. So, if you are really worried about strokes, don't use a smart watch, go to your cardiologist and let them take an EKG proper.
(Score: 0) by Anonymous Coward on Sunday May 14 2017, @02:12PM (7 children)
The techcrunch article is garbage. If you read here you discover they mean AUC instead of accuracy:
https://blog.cardiogr.am/applying-artificial-intelligence-in-medicine-our-early-results-78bfe7605d32 [cardiogr.am]
You can get 90% accuracy with this data by always guessing "not arrhythmia".
(Score: 0) by Anonymous Coward on Sunday May 14 2017, @02:33PM
Oops, that would be 97% rather than 90%.
(Score: 3, Insightful) by AthanasiusKircher on Sunday May 14 2017, @02:46PM (1 child)
Yep. Actual numbers, based on the actual research:
- 6158 people
- 98.04% sensitivity, which means out of 200 people with heart conditions, they found 196 of them. 4 people who had heart conditions apparently weren't identified correctly (false negatives). Well, that's based on the number of unhealthy participants reported by Techcrunch; I don't know where the ".04%" comes from if those numbers are accurate.
- 90.2% specificity, which means out of the 5958 "healthy people," the Apple Watch correctly identified 5374 as actually healthy, as well as 584 as having a heart problem. Total people flagged as having heart problems therefore seems to be 780 of the 6158.
Thus, if the Apple Watch identifies you as having an "abnormal heart rhythm," you have a 196/780 = ~25% of actually having one. If the Apple Watch identifies you as NOT having an "abnormal heart rhythm," you have a ~0.07% of having one (and not having it detected).
To those unfamiliar with medical stats, this may sound overly good or overly bad. But this is typical of a preliminary diagnostic test -- high sensitivity, but mediocre specificity. What you want there is a very low false negative rate, and a subsequent confirmation test can weed out the false positives. The problem with such tests is that they end up worrying a large number of people who don't have any problem at all.
----
Stats note: Techcrunch misleadingly uses AUC of 0.97 as "accuracy," but that's not what the stat usually is interpreted as. Generally speaking, it usually is equal to the probability [wikipedia.org] that the classification scheme will rank a randomly chosen positive higher than a randomly chosen negative. Whether AUC is actually a good proxy for "accuracy" or strength of a model is debatable and depends on the situation. Based on the definition, AUC is obviously high for tests with very low false negative rates, but it won't necessarily give you good info about false positives.
(Score: 0) by Anonymous Coward on Sunday May 14 2017, @06:07PM
Original AC here. Thanks for digging into the actual numbers from TFA, they don't make it any better. ;)
So beware of the tests! As parent said: 'The problem with such tests is that they end up worrying a large number of people who don't have any problem at all.'
This automatically happens whenever a rare condition is tested (e. g., when the condition is significantly rarer than 1 in 2 people, say, 10 % and lower) and with a specificity only in the nineties—99.9 % and each more significant digit raises the bar, but which test/cheap screening gets that good?
(Score: 1) by Rich26189 on Sunday May 14 2017, @03:36PM (3 children)
This inquiring mind wants to know, what does ACU stand for in this context? I’ve no experience with stats or medical research. I followed the link, saw the acronym but no explanation, did I miss it? A DDG search for the acronym returned nothing of value neither did searching on “allacronyms.com. You’re warning us it should not be confused with accuracy, what is it?
(Score: 0) by Anonymous Coward on Sunday May 14 2017, @04:04PM (1 child)
It isn't a medical term, it means area under the ROC curve: https://www.kaggle.com/wiki/ROC [kaggle.com]
(Score: 1) by Rich26189 on Monday May 15 2017, @05:44PM
Thanks
(Score: 0) by Anonymous Coward on Sunday May 14 2017, @04:36PM
The kaggle wiki also contains this helpful page: https://www.kaggle.com/wiki/AreaUnderCurve [kaggle.com]
(Score: 3, Informative) by Nerdfest on Sunday May 14 2017, @02:19PM
Basically more free Apple advertising from a tech blog.