Stories
Slash Boxes
Comments

SoylentNews is people

posted by janrinok on Monday June 22 2015, @08:00PM   Printer-friendly
from the little-more-than-guesswork dept.

Cliff Zukin writes in the NY Times that those paying close attention to the 2016 election should exercise caution as they read the polls because election polling is in near crisis as statisticians say polls are becoming less reliable. According to Zukin, two trends are driving the increasing unreliability of election and other polling in the United States: the growth of cellphones and the decline in people willing to answer surveys. Coupled, they have made high-quality research much more expensive to do, so there is less of it. This has opened the door for less scientifically based, less well-tested techniques. To top it off, a perennial election polling problem, how to identify “likely voters,” has become even thornier. Today, a majority of people are difficult or impossible to reach on landline phones. One problem is that the 1991 Telephone Consumer Protection Act has been interpreted by the Federal Communications Commission to prohibit the calling of cellphones through automatic dialers, in which calls are passed to live interviewers only after a person picks up the phone. To complete a 1,000-person survey, it’s not unusual to have to dial more than 20,000 random numbers, most of which do not go to actual working telephone numbers.

The second unsettling trend are rapidly declining response rates, reaching levels once considered unimaginable. In the late 1970s, pollsters considered an 80 percent response rate acceptable but by 2014 the response rate has fallen to 8 percent. "Our old paradigm has broken down, and we haven’t figured out how to replace it," concludes Zukin. "In short, polls and pollsters are going to be less reliable. We may not even know when we’re off base. What this means for 2016 is anybody’s guess."


Original Submission

 
This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 2) by looorg on Monday June 22 2015, @10:41PM

    by looorg (578) on Monday June 22 2015, @10:41PM (#199625)

    Why would you assume they all lie or are dishonest? Most people don't go around lying just cause they can. Do you deliberately go around lying to people when asked mundane questions? If they feel they don't want to answer then they don't -- they usually hang up or say no thanks (or you get told to fuck off). Most people don't drag such things out for 10 minutes. But I could be wrong. It's just one of those things or questions that are rarely investigated because it would be so hard to validate or determine if they where truthful or not.

    I would assume the disagreement is in regards to you being wrong. Without even getting into the whole statistics probability vs "truth" argument. Asking as few as a thousand people is usually more then enough, decades of public election polling backs this up, I have not checked US numbers lately but for most countries they tend to be in the ballpark -- there are always exceptions but as with most statistics you can usually ignore the extreme outliers. They are very rarely off by big amounts, if they are that tends to mean something is wrong with the method or formula weights or that something beyond the scope of the survey took place. Increasing the amount of people polled usually don't provide much better results, none that can justify the amount of extra time and money spent. Would total surveys (where you ask every one) be better? Sure. But time and money tends to eliminate that as an option.

    The biggest problem isn't people having land lines anymore. The biggest problem is the giant stream of polls that are conducted. People just get bored and zone out. To many polls and it's sometime hard to judge if this is a serious poll, some pr-marketing stunt type thing or if it's some organization with an agenda. If you go back to the 70s or before the once that did large surveys was government agencies and universities etc and it was a somewhat clear distinction from the commercial polls (such as if coke is better then pepsi or whatever). These days the overlap is to big and it's hard to tell. The market has been flooded with polling agencies, some that are clearly less serious then others and it's dragging everyone and everything down.

    As mentioned in the article telephone polls are loosing efficiency due to cellphones. But also because people find it annoying. They can't seem to tell the difference between serious polls and telemarketers and (scam) pollsters. The growth in the market has lead to an erosion of trust. So instead of having to decide whom to answer you make the easy choice and tell everyone to fuck off. They are also interrupting your daily life by demanding answers over the phone. So some of the older alternatives are coming back again with mailings (it's expensive and the return or answer frequency isn't much better but you can usually ask a large amount of questions -- which is also a drawback; people don't want to answer 20 pages filled with questions about everything under the sun) and personal interviews where you visit the people and ask them in person (problematic if you are asking about personal preferences or sensitive matters -- and it's obvious really expensive).

    The biggest problem today, if you ask me, is probably all the small "self-recruited" polls; where people sign up to be polled. They are filled with people on various agendas and the results tend to reflect that. Easy to do and they also tend to give the answers you want. But in the internet and news-cycle they get just as much room as serious polls. People just can't tell the difference, anymore, between good and bad polls.

    Starting Score:    1  point
    Karma-Bonus Modifier   +1  

    Total Score:   2  
  • (Score: 1, Interesting) by Anonymous Coward on Monday June 22 2015, @10:55PM

    by Anonymous Coward on Monday June 22 2015, @10:55PM (#199631)

    Why would you assume they all lie or are dishonest?

    Not assuming that people gave honest and accurate answers doesn't mean you assume that people are liars, so that's a straw man. Science should be as objective and rigorous as possible. "That would be hard to do." isn't a valid excuse for bad science.

    Asking as few as a thousand people is usually more then enough, decades of public election polling backs this up

    Or perhaps it's just a means of manipulation, even if unintentional.

    I would assume the disagreement is in regards to you being wrong.

    There's your issue.

    I don't really pay attention to *any* of these silly polls or surveys, much like I don't pay attention to the social 'sciences'; they're meaningless garbage.

    • (Score: 4, Insightful) by looorg on Monday June 22 2015, @11:25PM

      by looorg (578) on Monday June 22 2015, @11:25PM (#199638)

      Science should be as objective and rigorous as possible. "That would be hard to do." isn't a valid excuse for bad science.

      In most cases science tends to be as objective and rigorous as is possible with the limitations given. There is very little free roaming science around that can do whatever it wants beyond the boundaries of time and money. You want or long for some imaginary world of absolutes that just isn't there and probably hasn't been around for aeons. Most scientific fields today relies heavily upon statistics and modelling and not just the social sciences that you seem to have massive trust issues with for some reason. You have clearly singled it out for your distrust while at the same time ignoring the abundance of statistics used in most other fields. Even if you remove the social sciences from the equation most of the hard or real sciences (whatever they are, we clearly don't share definitions) don't follow or adhere to your strict ideals for objectivity, rigor or absolute answers either.

      If you have any suggestion about how to do cheap, instant and accurate polling I'm fairly certain the world or statistics would be eager to hear about them. As with most things time and money does matter.

      I don't really pay attention to *any* of these silly polls or surveys, much like I don't pay attention to the social 'sciences'; they're meaningless garbage.

      So you don't believe in statistics, social sciences and polling. But yet you had to start a conversation about it. You should probably have put that in your first post and it would have been tagged for trolling instead of disagreement. Shame on me for falling into your trap.

      • (Score: 0) by Anonymous Coward on Monday June 22 2015, @11:43PM

        by Anonymous Coward on Monday June 22 2015, @11:43PM (#199651)

        In most cases science tends to be as objective and rigorous as is possible with the limitations given.

        When the accuracy is questionable, the study should say as much. People simply shouldn't assume things to be true just because it would be difficult to investigate much further. The problem is a combination of bad media reporting and, at times, even the ones who conducted the poll are dishonest.

        You want or long for some imaginary world of absolutes that just isn't there and probably hasn't been around for aeons.

        Straw man.

        Most scientific fields today relies heavily upon statistics and modelling and not just the social sciences that you seem to have massive trust issues with for some reason.

        Straw man; statistics isn't the main issue.

        Even if you remove the social sciences from the equation most of the hard or real sciences (whatever they are, we clearly don't share definitions) don't follow or adhere to your strict ideals for objectivity, rigor or absolute answers either.

        They adhere to the ideal far better, however. And no one is talking about absolutes except you.

        The main issues with the social 'sciences' is that they usually deal with extremely subjective issues that are almost impossible to verify or investigate. But that doesn't stop the media or many social scientists from claiming that the results of the studies are objective and highly accurate. That doesn't stop researchers from reaching arbitrary conclusions based on data about subjective matters (i.e. porn makes people callous towards women). If the researchers are honest about the limitations of their studies, then that is a good first step. But I won't call current standards good simply because we don't know of a better way to investigate these issues; that's nonsensical.

        If you have any suggestion about how to do cheap, instant and accurate polling

        That it would be hard to do it better is not a reason to call current methods good, if that is what you're saying. Do you understand that logic?

        So you don't believe in statistics, social sciences and polling. But yet you had to start a conversation about it. You should probably have put that in your first post and it would have been tagged for trolling instead of disagreement.

        Ah, I see. Not accepting the social 'sciences' because of their obvious problems and limitations [arachnoid.com] is "trolling" somehow. Does anyone even know what a troll is anymore?

        Fine, if we're going to play that game, then I say you're trolling because no one can seriously believe that the social 'sciences' are all that credible. Your move.

  • (Score: 0) by Anonymous Coward on Tuesday June 23 2015, @04:46AM

    by Anonymous Coward on Tuesday June 23 2015, @04:46AM (#199738)

    I don't see why anyone wastes their time on it. If they want my opinion, they can pay me in advance, otherwise, just leave me alone. I turned down a some kind of TV survey once and got a "but it's for the Hollywood people" (likely fake) baffled sounding response from the pollster. Who the f*** cares who it's for?

  • (Score: 2) by TheLink on Tuesday June 23 2015, @06:31AM

    by TheLink (332) on Tuesday June 23 2015, @06:31AM (#199757) Journal

    Why would you assume they all lie or are dishonest? Most people don't go around lying just cause they can. Do you deliberately go around lying to people when asked mundane questions?

    It's not so much due to people being dishonest, but people being easily manipulated and of pollsters being dishonest and not interested in accuracy:
    https://www.youtube.com/watch?v=G0ZZJXw4MTA [youtube.com]
    https://www.youtube.com/watch?v=If9EWDB_zK4 [youtube.com]