SoylentNews is people
SoylentNews
Beginning in the early 1990s a quality-improvement program began in New York State and has since spread to many other states where report cards were issued to improve cardiac surgery by tracking surgical outcomes, sharing the results with hospitals and the public, and when necessary, placing surgeons or surgical programs on probation. But Sandeep Jauhar writes in the NYT that the report cards have backfired. "They often penalized surgeons, like the senior surgeon at my hospital, who were aggressive about treating very sick patients and thus incurred higher mortality rates," says Jauhar. "When the statistics were publicized, some talented surgeons with higher-than-expected mortality statistics lost their operating privileges, while others, whose risk aversion had earned them lower-than-predicted rates, used the report cards to promote their services in advertisements."
Surveys of cardiac surgeons in The New England Journal of Medicine have confirmed that reports like the Consumer Guide to Coronary Artery Bypass Graft Surgery have limited credibility among cardiovascular specialists, little influence on referral recommendations and may introduce a barrier to care for severely ill patients. According to Jauhar, there is little evidence that the public — as opposed to state agencies and hospitals — pays much attention to surgical report cards anyway. A recent survey found that only 6 percent of patients used such information in making medical decisions. "Surgical report cards are a classic example of how a well-meaning program in medicine can have unintended consequences," concludes Jauhar. "It would appear that doctors, not patients, are the ones focused on doctors' grades — and their focus is distorted and blurry at best."
(Score: 3, Insightful) by Anonymous Coward on Thursday July 23 2015, @08:43PM
Not all metrics are equal, folks.
If I were trying to improve kids' discipline at school and I measured it by the time they got home on the theory that this would reflect their detention rates, I'd also catch a lot of keen applicants for the Ivy League who had a huge list of extracurriculars.
Same here - does what you want to know match the measure you try to collect?
Government in particular is rife with this kind of nonsense, which is why activists scream about all sorts of hideous abuses (sometimes they're even right, although they're infamous for cherry-picking data to fit their predetermined narratives) and wave statistics to prove their case at the same time as corporate compliance officers carefully point out precisely how they complied with the exact regulations intended to prevent the supposed abuses.
Any programmer could have told them: GIGO.
(Score: 2, Insightful) by FrogBlast on Thursday July 23 2015, @08:58PM
Yeah, it's a surgeon complaining about report cards for surgeons. He should probably just be advocating for a better scoring system. He's already identified the other value that needs to be incorporated into the metric - difficulty of cases taken - but the essay still reads as a dismissal of any type of evaluation.
(Score: 0) by Anonymous Coward on Thursday July 23 2015, @09:13PM
Surgeons would prefer IQ:
http://anepigone.blogspot.com/2011/01/average-iq-by-occupation-estimated-from.html [blogspot.com]
(Score: 2) by Wootery on Friday July 24 2015, @10:20AM
It seems unlikely that the average IQ of a surgeon is the world-record [mostextreme.org] IQ. [quora.com]
(Score: 0) by Anonymous Coward on Friday July 24 2015, @04:16PM
After reading multiple comments critiquing the post for the same thing, it's clear that I did a poor job of explaining what the table purports to show. It's not supposed to be an exact measure of IQ by profession by any means, as it is based entirely on average annual income figures. In other words, it's an income table with the values converted to IQ scores (and thus, as silly girl points out, it's a bit of a mislabeling on my part, although that is the point--to get an instinctive feel for how related IQ and income are at the career level).
And this is why IQ should be treated as nothing but a joke. We have yet to even truly define intelligence, and yet simplistic thinking leads people to arbitrarily conclude that money and grades indicate how intelligent you are. I see no hard evidence for any of this; people often simply assume that those things indicate your intelligence, with no proof to back it up.
"People who do well at our arbitrary and simplistic tests tend to also do well at these other things." would be a bit more honest, but most people (especially normal people) don't seem to get that.
(Score: 3, Interesting) by Francis on Thursday July 23 2015, @09:00PM
That's not at all surprising. I was teaching at a place where we were given a feedback score from each student. It was .5 stars to 5 and we'd receive the scores weeks late. To make matters worse, anything below 4.5 was not acceptable to the school which meant that anything less than a 5 would potentially get us in trouble.
That wouldn't have been bad enough, but they had secret metrics and secret feedback that were never shared with the teacher making the whole thing a game of how much crap could they give the teacher before they'd quit. Not surprisingly, the quality of instruction suffered as we weren't ever being measured for that, it was just one huge popularity contest where you hoped that the students were in a good mood before they started the class.
(Score: 1, Interesting) by Anonymous Coward on Thursday July 23 2015, @10:40PM
> To make matters worse, anything below 4.5 was not acceptable to the school which meant that anything less than a 5 would potentially get us in trouble.
Anytime you work with a sales person or a customer support rep and you are asked to rate them afterwards, it is the same thing. Anything less than a perfect score is a failing score. So, unless the person was an outright asshole to you, give them a perfect score. Everybody screws up sometimes, but the PHBs who push the metrics on the plebes can't seem to comprehend that, which is ironic given that their failure of comprehension is a pretty big screw-up itself.
(Score: 2) by Francis on Thursday July 23 2015, @10:50PM
Indeed, it's a stupid system, but the people requiring it are too stupid to realize it so 9/10 is the new 5/10 and God help you if you're anything less than 90% of perfect.
Admittedly participating in it has it's own problems, but I sleep better giving people perfect scores that nobody deserves than getting them fired and replaced by somebody that's not any better and will be abused equally.
(Score: 0) by Anonymous Coward on Friday July 24 2015, @03:57PM
which is why activists scream about all sorts of hideous abuses (sometimes they're even right, although they're infamous for cherry-picking data to fit their predetermined narratives)
Which activists? Government abuse is simply common, so it may be that they're right more often than not.