SoylentNews is people
SoylentNews
from the is-a-love-of-statistics-a-normal-distribution? dept.
Over on the npr blog 13.7 Cosmos and Culture , contributor Adam Frank has written a commentary on how he learned to love statistics.
What I loved about physics were its laws. They were timeless. They were eternal. Most of all, I believed they fully and exactly determined everything about the behavior of the cosmos.
Statistics, on the other hand, was about the imperfect world of imperfect equipment taking imperfect data. For me, that realm was just a crappy version of the pure domain of perfect laws I was interested in. Measurements, by their nature, would always be messy. A truck goes by and jiggles your equipment. The kid you paid to do the observations isn't really paying attention. The very need to account for those variations made me sad.
Now, however, I see things very differently. My change of heart can be expressed in just two words — Big Data. Over the last 10 years, I've been watching in awe as the information we have been inadvertently amassing has changed society for better and worse. There is so much power, promise and peril for everyone in this brave new world that I knew I had to get involved. That's where my new life in statistics began.
(Score: 0) by Anonymous Coward on Tuesday December 06 2016, @01:07PM
was correctly predicted by Michael Moore instead of Nate Silver and all the other experts. Similar with Brexit, although I don't know if 538 had a dog in that race.
So what, that's just an election? That's supposed to be an easy problem for forecasters. They have tons of data and many motivated people who have spent careers looking at it.
(Score: 0) by Anonymous Coward on Tuesday December 06 2016, @01:53PM
No. The error was sampling methods.
When you sample 3,000 people nationwide... You are getting Popular Vote, and that was not wrong. But that is not how US Elections work.
When you sample at least 100 in each Electoral College District, so 53,800, then you are modeling the right information.
Remember:
1) There are Liars, Damn Lairs and then Statisticians.
2) 83.5% of statistics are made up on the spot.
3) A statistician can support ANY claim with ANY data set.
(Score: 0) by Anonymous Coward on Tuesday December 06 2016, @01:55PM
Except that many of the forecasters were taking the electoral college into account when they were calling the race.
(Score: 1, Troll) by BK on Tuesday December 06 2016, @02:25PM
The problem was the sampling method and weighting strategy. They don't ask 1000 people something and then publish the raw percentages. The massage they numbers... if they have too many or too few black transsexuals or hispanic women or skinheads or coal miners or unemployed asian computer programmers, they 'fix it'.
They try to adjust for turnout but there is no way to _know_ the correct weights for samples before the election shows you what actual turnout is going to be. In the end, the adjustments probably tell you more about the pollsters than anything else.
...but you HAVE heard of me.
(Score: 0) by Anonymous Coward on Tuesday December 06 2016, @02:07PM
Of course statistics are 54.68% more credible for each digit after the decimal point.
(Score: 1, Insightful) by Anonymous Coward on Tuesday December 06 2016, @05:07PM
> was correctly predicted by Michael Moore instead of Nate Silver and all the other experts.
You weren't paying attention.
Nate Silver consistently said there was a ~30% chance of Trump winning.
I read Silver's site multiple times per day and he got a lot of shit by actual partisans for saying that.
http://www.mediaite.com/online/nate-silver-warns-media-against-dangerous-assumption-trump-isnt-really-closing-in-on-hillary/ [mediaite.com]
http://www.businessinsider.com/nate-silver-hillary-clinton-donald-trump-election-prediction-2016-11 [businessinsider.com]
https://fivethirtyeight.com/features/trump-is-just-a-normal-polling-error-behind-clinton/ [fivethirtyeight.com]