SoylentNews is people
SoylentNews
According to a study conducted through heartbeat measurement app Cardiogram and the University of California, San Francisco, the Apple Watch is 97 percent accurate in detecting the most common abnormal heart rhythm when paired with an AI-based algorithm.
The study involved 6,158 participants recruited through the Cardiogram app on Apple Watch. Most of the participants in the UCSF Health eHeart study had normal EKG readings. However, 200 of them had been diagnosed with paroxysmal atrial fibrillation (an abnormal heartbeat). Engineers then trained a deep neural network to identify these abnormal heart rhythms from Apple Watch heart rate data.
Cardiogram began the study with UCSF in 2016 to discover whether the Apple Watch could detect an oncoming stroke. About a quarter of strokes are caused by an abnormal heart rhythm, according to Cardiogram co-founder and data scientist for UCSF's eHeart study Brandon Ballinger.
Yes, but can the Apple Watch then pace you or shock you?
(Score: 3, Insightful) by AthanasiusKircher on Sunday May 14 2017, @02:46PM (1 child)
Yep. Actual numbers, based on the actual research:
- 6158 people
- 98.04% sensitivity, which means out of 200 people with heart conditions, they found 196 of them. 4 people who had heart conditions apparently weren't identified correctly (false negatives). Well, that's based on the number of unhealthy participants reported by Techcrunch; I don't know where the ".04%" comes from if those numbers are accurate.
- 90.2% specificity, which means out of the 5958 "healthy people," the Apple Watch correctly identified 5374 as actually healthy, as well as 584 as having a heart problem. Total people flagged as having heart problems therefore seems to be 780 of the 6158.
Thus, if the Apple Watch identifies you as having an "abnormal heart rhythm," you have a 196/780 = ~25% of actually having one. If the Apple Watch identifies you as NOT having an "abnormal heart rhythm," you have a ~0.07% of having one (and not having it detected).
To those unfamiliar with medical stats, this may sound overly good or overly bad. But this is typical of a preliminary diagnostic test -- high sensitivity, but mediocre specificity. What you want there is a very low false negative rate, and a subsequent confirmation test can weed out the false positives. The problem with such tests is that they end up worrying a large number of people who don't have any problem at all.
----
Stats note: Techcrunch misleadingly uses AUC of 0.97 as "accuracy," but that's not what the stat usually is interpreted as. Generally speaking, it usually is equal to the probability [wikipedia.org] that the classification scheme will rank a randomly chosen positive higher than a randomly chosen negative. Whether AUC is actually a good proxy for "accuracy" or strength of a model is debatable and depends on the situation. Based on the definition, AUC is obviously high for tests with very low false negative rates, but it won't necessarily give you good info about false positives.
(Score: 0) by Anonymous Coward on Sunday May 14 2017, @06:07PM
Original AC here. Thanks for digging into the actual numbers from TFA, they don't make it any better. ;)
So beware of the tests! As parent said: 'The problem with such tests is that they end up worrying a large number of people who don't have any problem at all.'
This automatically happens whenever a rare condition is tested (e. g., when the condition is significantly rarer than 1 in 2 people, say, 10 % and lower) and with a specificity only in the nineties—99.9 % and each more significant digit raises the bar, but which test/cheap screening gets that good?