SoylentNews is people
SoylentNews
Cliff Zukin writes in the NY Times that those paying close attention to the 2016 election should exercise caution as they read the polls because election polling is in near crisis as statisticians say polls are becoming less reliable. According to Zukin, two trends are driving the increasing unreliability of election and other polling in the United States: the growth of cellphones and the decline in people willing to answer surveys. Coupled, they have made high-quality research much more expensive to do, so there is less of it. This has opened the door for less scientifically based, less well-tested techniques. To top it off, a perennial election polling problem, how to identify “likely voters,” has become even thornier. Today, a majority of people are difficult or impossible to reach on landline phones. One problem is that the 1991 Telephone Consumer Protection Act has been interpreted by the Federal Communications Commission to prohibit the calling of cellphones through automatic dialers, in which calls are passed to live interviewers only after a person picks up the phone. To complete a 1,000-person survey, it’s not unusual to have to dial more than 20,000 random numbers, most of which do not go to actual working telephone numbers.
The second unsettling trend are rapidly declining response rates, reaching levels once considered unimaginable. In the late 1970s, pollsters considered an 80 percent response rate acceptable but by 2014 the response rate has fallen to 8 percent. "Our old paradigm has broken down, and we haven’t figured out how to replace it," concludes Zukin. "In short, polls and pollsters are going to be less reliable. We may not even know when we’re off base. What this means for 2016 is anybody’s guess."
Original Submission
(Score: 1, Disagree) by Anonymous Coward on Monday June 22 2015, @08:12PM
Asking a few thousand people a few questions and then assuming the answers were honest and accurate isn't exactly scientific, after all.
(Score: 0, Disagree) by Anonymous Coward on Monday June 22 2015, @08:36PM
Disagree?
Want to post a response to that?
I thought it was a very valid point although it is better to make that statement and back it up with some data.
(Score: 3, Insightful) by Tork on Monday June 22 2015, @09:18PM
I thought it was a very valid point...
There's your problem.
🏳️🌈 Proud Ally 🏳️🌈
(Score: 0) by Anonymous Coward on Tuesday June 23 2015, @02:23AM
Yeah it is a problem to use things like reasoning and logic when discussing science. It is like three quarters of the people here take it as a religion and it is blasphemy to point out when something actually is not scientific at all.
(Score: 3, Touché) by Tork on Tuesday June 23 2015, @05:02AM
Yeah it is a problem to use things like reasoning and logic when discussing science.
Try it.
🏳️🌈 Proud Ally 🏳️🌈
(Score: 0) by Anonymous Coward on Tuesday June 23 2015, @02:06PM
You try, since all you've been doing is making very short comments that explain absolutely nothing. How insightful.
(Score: 2) by Tork on Tuesday June 23 2015, @03:05PM
🏳️🌈 Proud Ally 🏳️🌈
(Score: 3, Insightful) by tibman on Monday June 22 2015, @09:38PM
Not sure why people are hating on you. Land-line based phones would be a terrible way to get a cross-section of voter's intentions. It was probably useful ten years ago but not now.
SN won't survive on lurkers alone. Write comments.
(Score: 2) by aristarchus on Tuesday June 23 2015, @08:50AM
Not sure why people are hating on you.
Not hating, just pointing out an idiot when one raises its ugly head. If the so egregoiusly down-modded poster had the slighted understanding of sampling, statistics, inductive reasoning, and just plain SCIENCE, they would not have said such a stupid thing. And now you, you have stepped right in with your lack of understanding of what is being discussed. The poster being so properly chastised said nothing about landlines, that is actually something that is in the FA! And now you defend them by making it seem like the post may have been on topic? Fey, fey, my dear sir (or madam), these are dangerous times, when we must choose our terms carefully, lest they become fodder for such idiots as above.
Now about sampling (and I am quite sure that professional pollsters are well aware of their difficulties), everyone should be aware of the US presidential election of 1948, where the Chicago https://en.wikipedia.org/wiki/Dewey_Defeats_Truman [wikipedia.org] Daily Tribune announce that Dewey won. Remember President Dewey? So what happened? First ever telephone poll. In 1948, who owned telephones, a rather new fangled technology? That's right, wealthier people. And who do stupid rich people vote for? Right! Republicans! Even in 1948 (some things never change). Classic example of a prediction based on a biased sample. (This is also why, nowadays, when rightwing newspapers publish poll results, they are not trying to report on the election, they are trying to influence it, just like Karl Rove on the Fox Armaggeddon Channel, when he had the "real maths".)
So the only point being made in the Fine Article is 1) landlines are no longer a reliable way to sample public opinion, and 2) polling may no longer be a reliable way to sample public opinion. Now knowing both of these facts, I am sure that professional pollsters will have no problem producing product for us in the forseeable future. It will just be a question of whether it is pull polling, or push polling, or whether advertising is no longer a viable business model for the internet. It's like the guy in the dingy in the hold of the Exxon Valdez at the end of the not-so-great movie "Waterworld", when the flare is dropping down, and he says, "Oh, Thank god."
(Score: 0) by Anonymous Coward on Tuesday June 23 2015, @03:38PM
s/stupid/ignorant/
I'm not sure whether you're genuinely stupid or just ignorant on the difference between those words, though.
(Score: 0) by Anonymous Coward on Tuesday June 23 2015, @03:53PM
Not hating, just pointing out an idiot when one raises its ugly head. If the so egregoiusly down-modded poster had the slighted understanding of sampling, statistics, inductive reasoning, and just plain SCIENCE, they would not have said such a stupid thing.
So it's scientific to simply assume you're right that people's answers are correct. This has nothing to do with denying statistics, sampling, inductive reasoning, or science; it's simply denying bad science built on too may assumptions.
(Score: 2) by aristarchus on Tuesday June 23 2015, @05:15PM
So it's scientific to simply assume you're right that people's answers are correct.
My Gawd, the stupid just keeps on coming! Who said any such thing? I never did, honestly, would I lie to you?
This has nothing to do with denying statistics, sampling, inductive reasoning, or science; it's simply denying bad science built on too may assumptions.
OK, I see there are some cognitive liabilities involved here. The idiot frist poster said any sampling was wrong,
Asking a few thousand people a few questions
Or were we to take the veracity the be the main issue? If we asked more people more questions, would it counteract the mendacity of the source? Or do we have to reduce the dishonesty and then the numbers won't matter? It's all terribly confusing, involving sampling and statistics and correct inductive reasoning, and margins of error. And guess what! Do you not think that pollsters take honesty into account? It is not that hard to confirm your assumptions in election polling, you can just compare your prediction to the official results! If you are wrong, there is something wrong with your methods and/or sources. This is what the Fine Article is saying, they know this already.
To just say, "Grumble grumble Small Sample size! Grumble grumble, People could lie! So Grumble grumble, Polls suck!" sounds just like, exactly like, and nearly indistinguishable from, a climage-change denier. Begin Handwaving, . . . NOW!
(Score: 0) by Anonymous Coward on Tuesday June 23 2015, @05:42PM
The idiot frist poster said any sampling was wrong,
Nope. Notice the "and" in the sentence and you will understand your mistake.
Do you not think that pollsters take honesty into account?
The polls could be manipulating the election results, and the pollsters themselves could be dishonest.
Not only that, but the polls are a useless waste of time.
(Score: 2) by aristarchus on Tuesday June 23 2015, @07:05PM
The idiot frist poster said any sampling was wrong,
Nope. Notice the "and" in the sentence and you will understand your mistake.
Getting rather tiring, but nope on the nope. "And" is a logical conjunction, which means that both claims are asserted to be true. I was only asking for clarification, such that the contrary could be true if one of the conjuncts was not, as in "if the sample size were large enough, would polls be accurate despite respondent mendacity?". Evidently I will not be getting an answer, since
Not only that, but the polls are a useless waste of time.
Why are polls a waste of time? What are they for? Why do some people pay so much money to have them done, and who are these people? Evidently polls are reliable enough for them! So are we going with Karl Rove's "real math", or Nixon's "Silent Majority"? Everyone knows that every agrees with me, so we can ignore polls. Yeah, that's the ticket!
(Score: 0) by Anonymous Coward on Tuesday June 23 2015, @08:17PM
Getting rather tiring, but nope on the nope. "And" is a logical conjunction, which means that both claims are asserted to be true.
It's describing a specific scenario. Learn the way humans actually use English rather than acting like a robot.
Why are polls a waste of time? What are they for? Why do some people pay so much money to have them done, and who are these people? Evidently polls are reliable enough for them! So are we going with Karl Rove's "real math", or Nixon's "Silent Majority"? Everyone knows that every agrees with me, so we can ignore polls. Yeah, that's the ticket!
You're ranting about random nonsense.
Why exactly are polls not a waste of time? Is there some pressing need to know what the election results will likely be before the election that doesn't involve just manipulating voters into voting a certain way? Probably not, or at least not for me. If you want to pay attention to stupid popularity polls, have at it.
(Score: 2) by looorg on Monday June 22 2015, @10:41PM
Why would you assume they all lie or are dishonest? Most people don't go around lying just cause they can. Do you deliberately go around lying to people when asked mundane questions? If they feel they don't want to answer then they don't -- they usually hang up or say no thanks (or you get told to fuck off). Most people don't drag such things out for 10 minutes. But I could be wrong. It's just one of those things or questions that are rarely investigated because it would be so hard to validate or determine if they where truthful or not.
I would assume the disagreement is in regards to you being wrong. Without even getting into the whole statistics probability vs "truth" argument. Asking as few as a thousand people is usually more then enough, decades of public election polling backs this up, I have not checked US numbers lately but for most countries they tend to be in the ballpark -- there are always exceptions but as with most statistics you can usually ignore the extreme outliers. They are very rarely off by big amounts, if they are that tends to mean something is wrong with the method or formula weights or that something beyond the scope of the survey took place. Increasing the amount of people polled usually don't provide much better results, none that can justify the amount of extra time and money spent. Would total surveys (where you ask every one) be better? Sure. But time and money tends to eliminate that as an option.
The biggest problem isn't people having land lines anymore. The biggest problem is the giant stream of polls that are conducted. People just get bored and zone out. To many polls and it's sometime hard to judge if this is a serious poll, some pr-marketing stunt type thing or if it's some organization with an agenda. If you go back to the 70s or before the once that did large surveys was government agencies and universities etc and it was a somewhat clear distinction from the commercial polls (such as if coke is better then pepsi or whatever). These days the overlap is to big and it's hard to tell. The market has been flooded with polling agencies, some that are clearly less serious then others and it's dragging everyone and everything down.
As mentioned in the article telephone polls are loosing efficiency due to cellphones. But also because people find it annoying. They can't seem to tell the difference between serious polls and telemarketers and (scam) pollsters. The growth in the market has lead to an erosion of trust. So instead of having to decide whom to answer you make the easy choice and tell everyone to fuck off. They are also interrupting your daily life by demanding answers over the phone. So some of the older alternatives are coming back again with mailings (it's expensive and the return or answer frequency isn't much better but you can usually ask a large amount of questions -- which is also a drawback; people don't want to answer 20 pages filled with questions about everything under the sun) and personal interviews where you visit the people and ask them in person (problematic if you are asking about personal preferences or sensitive matters -- and it's obvious really expensive).
The biggest problem today, if you ask me, is probably all the small "self-recruited" polls; where people sign up to be polled. They are filled with people on various agendas and the results tend to reflect that. Easy to do and they also tend to give the answers you want. But in the internet and news-cycle they get just as much room as serious polls. People just can't tell the difference, anymore, between good and bad polls.
(Score: 1, Interesting) by Anonymous Coward on Monday June 22 2015, @10:55PM
Why would you assume they all lie or are dishonest?
Not assuming that people gave honest and accurate answers doesn't mean you assume that people are liars, so that's a straw man. Science should be as objective and rigorous as possible. "That would be hard to do." isn't a valid excuse for bad science.
Asking as few as a thousand people is usually more then enough, decades of public election polling backs this up
Or perhaps it's just a means of manipulation, even if unintentional.
I would assume the disagreement is in regards to you being wrong.
There's your issue.
I don't really pay attention to *any* of these silly polls or surveys, much like I don't pay attention to the social 'sciences'; they're meaningless garbage.
(Score: 4, Insightful) by looorg on Monday June 22 2015, @11:25PM
Science should be as objective and rigorous as possible. "That would be hard to do." isn't a valid excuse for bad science.
In most cases science tends to be as objective and rigorous as is possible with the limitations given. There is very little free roaming science around that can do whatever it wants beyond the boundaries of time and money. You want or long for some imaginary world of absolutes that just isn't there and probably hasn't been around for aeons. Most scientific fields today relies heavily upon statistics and modelling and not just the social sciences that you seem to have massive trust issues with for some reason. You have clearly singled it out for your distrust while at the same time ignoring the abundance of statistics used in most other fields. Even if you remove the social sciences from the equation most of the hard or real sciences (whatever they are, we clearly don't share definitions) don't follow or adhere to your strict ideals for objectivity, rigor or absolute answers either.
If you have any suggestion about how to do cheap, instant and accurate polling I'm fairly certain the world or statistics would be eager to hear about them. As with most things time and money does matter.
I don't really pay attention to *any* of these silly polls or surveys, much like I don't pay attention to the social 'sciences'; they're meaningless garbage.
So you don't believe in statistics, social sciences and polling. But yet you had to start a conversation about it. You should probably have put that in your first post and it would have been tagged for trolling instead of disagreement. Shame on me for falling into your trap.
(Score: 0) by Anonymous Coward on Monday June 22 2015, @11:43PM
In most cases science tends to be as objective and rigorous as is possible with the limitations given.
When the accuracy is questionable, the study should say as much. People simply shouldn't assume things to be true just because it would be difficult to investigate much further. The problem is a combination of bad media reporting and, at times, even the ones who conducted the poll are dishonest.
You want or long for some imaginary world of absolutes that just isn't there and probably hasn't been around for aeons.
Straw man.
Most scientific fields today relies heavily upon statistics and modelling and not just the social sciences that you seem to have massive trust issues with for some reason.
Straw man; statistics isn't the main issue.
Even if you remove the social sciences from the equation most of the hard or real sciences (whatever they are, we clearly don't share definitions) don't follow or adhere to your strict ideals for objectivity, rigor or absolute answers either.
They adhere to the ideal far better, however. And no one is talking about absolutes except you.
The main issues with the social 'sciences' is that they usually deal with extremely subjective issues that are almost impossible to verify or investigate. But that doesn't stop the media or many social scientists from claiming that the results of the studies are objective and highly accurate. That doesn't stop researchers from reaching arbitrary conclusions based on data about subjective matters (i.e. porn makes people callous towards women). If the researchers are honest about the limitations of their studies, then that is a good first step. But I won't call current standards good simply because we don't know of a better way to investigate these issues; that's nonsensical.
If you have any suggestion about how to do cheap, instant and accurate polling
That it would be hard to do it better is not a reason to call current methods good, if that is what you're saying. Do you understand that logic?
So you don't believe in statistics, social sciences and polling. But yet you had to start a conversation about it. You should probably have put that in your first post and it would have been tagged for trolling instead of disagreement.
Ah, I see. Not accepting the social 'sciences' because of their obvious problems and limitations [arachnoid.com] is "trolling" somehow. Does anyone even know what a troll is anymore?
Fine, if we're going to play that game, then I say you're trolling because no one can seriously believe that the social 'sciences' are all that credible. Your move.
(Score: 0) by Anonymous Coward on Tuesday June 23 2015, @11:09AM
You are a perfect example of the current culture of replication bullying. Your defamation of science is unacceptable. We will unmask your IP address from behind whatever hackcode you are hiding it with and sue you. Be glad we don't throw you in jail or the asylum for destabilizing society.
https://politicalsciencereplication.wordpress.com/2014/05/25/replication-bullying-who-replicates-the-replicators/ [wordpress.com]
(Score: 0) by Anonymous Coward on Tuesday June 23 2015, @04:46AM
I don't see why anyone wastes their time on it. If they want my opinion, they can pay me in advance, otherwise, just leave me alone. I turned down a some kind of TV survey once and got a "but it's for the Hollywood people" (likely fake) baffled sounding response from the pollster. Who the f*** cares who it's for?
(Score: 2) by TheLink on Tuesday June 23 2015, @06:31AM
Why would you assume they all lie or are dishonest? Most people don't go around lying just cause they can. Do you deliberately go around lying to people when asked mundane questions?
It's not so much due to people being dishonest, but people being easily manipulated and of pollsters being dishonest and not interested in accuracy:
https://www.youtube.com/watch?v=G0ZZJXw4MTA [youtube.com]
https://www.youtube.com/watch?v=If9EWDB_zK4 [youtube.com]