Stories
Slash Boxes
Comments

SoylentNews is people

posted by janrinok on Thursday July 01 2021, @11:18AM   Printer-friendly
from the good-science-is-boring dept.

Social science papers that failed to replicate racked up 153 more citations, on average, than papers that replicated successfully.

This latest result is "pretty damning," says University of Maryland, College Park, cognitive scientist Michael Dougherty, who was not involved with the research. "Citation counts have long been treated as a proxy for research quality," he says, so the finding that less reliable research is cited more points to a "fundamental problem" with how such work is evaluated.

[...] University of California, San Diego, economists Marta Serra-Garcia and Uri Gneezy were interested in whether catchy research ideas would get more attention than mundane ones, even if they were less likely to be true. So they gathered data on 80 papers from three different projects that had tried to replicate important social science findings, with varying levels of success.

Citation counts on Google Scholar were significantly higher for the papers that failed to replicate, they report today in Science Advances, with an average boost of 16 extra citations per year. That's a big number, Serra-Garcia and Gneezy say—papers in high-impact journals in the same time period amassed a total of about 40 citations per year on average.

And when the researchers examined citations in papers published after the landmark replication projects, they found that the papers rarely acknowledged the failure to replicate, mentioning it only 12% of the time.

Well, nobody likes a Debbie Downer, do they?

Journal Reference:
Marta Serra-Garcia, Uri Gneezy. Nonreplicable publications are cited more than replicable ones [open], Science Advances (DOI: 10.1126/sciadv.abd1705)


Original Submission

 
This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
(1)
  • (Score: 4, Insightful) by Opportunist on Thursday July 01 2021, @11:32AM (15 children)

    by Opportunist (5545) on Thursday July 01 2021, @11:32AM (#1151756)

    The reason for this is easy to explain. What do you think gets more spotlight, a paper that basically confirms what was already established or something that claims to fundamentally shake established knowledge and turn the world upside down, because large parts of what we used to think was true have to be rewritten?

    Now add that due to decades of science doing rigorous testing results to ensure that what we know is actually more than a bunch of hunches and it should be very obvious why that shouldn't come as any surprise.

    • (Score: 3, Interesting) by Anonymous Coward on Thursday July 01 2021, @01:06PM (5 children)

      by Anonymous Coward on Thursday July 01 2021, @01:06PM (#1151771)

      This is a good reminder to researchers (of any sort) that citing a paper means you read it. Well...at least you scanned it (one can hope)...and didn't just crib the your list of cites from some other paper.

      A cite doesn't mean that you verified the results of that paper. It usually means that you incorporated ideas (or even words, properly referenced of course) from that paper into your own work and paper. It doesn't always mean that you agree with the results of the cite.

      My take is that citation counts are about like mod points here--a popularity contest? I may look at mods, but they don't mean all that much.

      • (Score: 3, Insightful) by looorg on Thursday July 01 2021, @02:10PM (1 child)

        by looorg (578) on Thursday July 01 2021, @02:10PM (#1151790)

        That is the best case scenario. A sad and quite likely other outcome is that my research assistant did a search for some papers that could back up what we are doing and your paper was inline with that or related to it somehow on a keyword level or something similar so we are including you to get out citation number up since if we cite you we are then more likely to be cited ourselves when the next person looks to cite someone and finds you and then also us so it's a gigantic citation-circlejerk.

        • (Score: 4, Interesting) by JoeMerchant on Thursday July 01 2021, @04:12PM

          by JoeMerchant (3937) on Thursday July 01 2021, @04:12PM (#1151848)

          A sad and quite likely other outcome is that my research assistant did a search for some papers that could back up what we are doing and your paper was inline with that or related to it somehow on a keyword level or something similar so we are including you to get out citation number up

          I remember working with "lab partners" in social sciences in Junior College - and this is the best possible behavior I could imagine coming out of any of them. Whatever the minimum possible effort to meet the requirements, and often less, is what I saw the majority of them doing - even the ones who were pursuing it as their major and potential career.

          --
          Україна досі не є частиною Росії Слава Україні🌻 https://news.stanford.edu/2023/02/17/will-russia-ukraine-war-end
      • (Score: 2) by DeathMonkey on Thursday July 01 2021, @03:14PM

        by DeathMonkey (1380) on Thursday July 01 2021, @03:14PM (#1151815) Journal

        And of course we only know it's not reproduceable because somebody tried to reproduce it. And that failed reproduction would also increase the citation count!

      • (Score: 4, Insightful) by Thexalon on Thursday July 01 2021, @05:28PM

        by Thexalon (636) Subscriber Badge on Thursday July 01 2021, @05:28PM (#1151883)

        Among other things: "Here's a Detailed Explanation of Why Popular Study X is Wrong" will invariably cite the study that they're presenting an argument attempting to debunk it. So that means that well-hyped-and-wrong research beats obscure-and-right research on citation counts every time. If you're looking for a lot citations, forget trying to get that acceptance at a prominent conference or publication in Nature, what you really want is your study being sensational enough or financially backed enough that it shows up on CNN.

        And if you think that leads to bunk research, you're absolutely right.

        --
        The only thing that stops a bad guy with a compiler is a good guy with a compiler.
      • (Score: 2) by Opportunist on Friday July 02 2021, @06:33AM

        by Opportunist (5545) on Friday July 02 2021, @06:33AM (#1152175)

        So far the theory.

        In fact, a cite usually means that whoever cited found the paper in a keyword search, did a cursory read to see whether it supports or contradicts him and in the first case, it gets used.

    • (Score: 4, Interesting) by driverless on Thursday July 01 2021, @02:19PM (7 children)

      by driverless (4770) on Thursday July 01 2021, @02:19PM (#1151793)

      It's also a bit of a special case, social-science is barely science and more in the realm of woo-woo. Friend of mine started studying it at Uni and got into repeated arguments with the lecturer about her total lack of understanding of even basic statistical methods, and eventually quit and switched to another field. That episode did not inspire confidence in the amount of actual science present in "social science".

      • (Score: 5, Insightful) by DeathMonkey on Thursday July 01 2021, @03:16PM (5 children)

        by DeathMonkey (1380) on Thursday July 01 2021, @03:16PM (#1151816) Journal

        It's just a lot harder to prove things about such a complicated system. That doesn't make it fake.

        • (Score: 5, Insightful) by Socrastotle on Thursday July 01 2021, @03:57PM (2 children)

          by Socrastotle (13446) on Thursday July 01 2021, @03:57PM (#1151838) Journal

          And what would? This is the most insidious problem with pseudo sciences. They generally not only cannot be tested, but they also cannot be falsified. And so their belief or doubt rests largely on cultural, rather than scientific, norms.

          Astrology is an obvious example. For the vast majority of its life astrology was a scholarly science, not especially different than astronomy. It was believed that the positioning, and behavior of the stars and bodies in the universe would have an influence on the individuals born under them. Why? Well, let me turn that around - prove it's fake. Simply put, you cannot. You might show that astrological predictions do not hold true, yet the same is true of those within social science. The observations that do come true, if not for noise than because of various confounding factors, will be held up as evidence of its soundness - identical to the social sciences.

          In fact it was ultimately only ended by another unfalsifiable entity that was more influential. The Roman Catholic Church around the 17th century felt that the implications of Astrology were incompatible with the notions of church regarding free will and so on. And so it became relegated from science to superstition. Incidentally, that also ties directly back into this issue. "Scientific" astrology briefly made a come back in the late 20th century. Carl Jung, the founder of analytical psychology, was a major advocate for astrology and also pursued it as a scholarly component of psychology. Let us thank our lucky stars that at least this component of psychology was left in the past. Now for the rest of it...

          • (Score: 1, Insightful) by Anonymous Coward on Thursday July 01 2021, @06:48PM (1 child)

            by Anonymous Coward on Thursday July 01 2021, @06:48PM (#1151943)

            You are holding an impossible standard.

            Let's say I gave you a coin, and you flipped it 10000 times, coming up heads 7000 times and tails 3000 times. Can you "prove" that the coin will come up heads next time you flipped it? No... but does your inability to falsify that make the information useless?

            I fully agree that social sciences can be more abused than others. I'm thinking particular of "gender studies," among other things. However, those acting in good faith can and do still do quality work which provides value (see: marketing).

            That it is difficult-to-impossible to do double-blind studies doesn't make it useless information... any more than the fact that "we can't predict what the precise temperature will be 30 days from now" means that "all climate science is a hoax."

            (Note the caveat of "acting in good faith. Those acting in bad faith can do exceptional damage in science in general, and even more in the soft-sciences.)

            • (Score: 2) by Socrastotle on Friday July 02 2021, @06:31AM

              by Socrastotle (13446) on Friday July 02 2021, @06:31AM (#1152174) Journal

              I'm not just poking at the probabilistic nature of things. Quantum mechanics, for instance, is inherently probabilistic - yet few would call it littered with fake assertions. The issue is that if you tell me that you flipped a coin 10,000 times and it came up heads 7,000 times then if I repeat your experiment, and you were honest, then I'm also going to get extremely close to 7,000 heads when I do your experiment - with some slight range for variation. The problem we're running into in the social sciences is that, instead, people are getting 2,300 heads far more often than not. And that means that the original experiment was invalid. The reason for that invalidity can be many things, but one of the biggest concerns is p-hacking. [nih.gov]

              p-hacking being one of many practices, but one of the most obvious is after-the-fact data dredging. Imagine you measure a large number of variables about something. And then you run 10,000 trials. As your number of variables increase, you're going to find more and more patterns in the data that mean absolutely nothing. For instance US spending on science is strongly correlated with suicides by hanging. Of course that's obviously a spurious correlation [tylervigen.com] but it's only obvious because those two variables "obviously" (another danger, but that's another topic) have nothing to do with one another. In fact that correlation is far stronger than most published correlations with the ever-implied-but-not-explicitly-stated suggestion of causation - it's a 99.79% correlation rate.

              But any good social scientist who wants to engage in proper p-hacking will only be measuring variables that sound viably related to their study. And so when they find these completely unrelated variables that *sound* possibly related: Boom - patch up that hypothesis a bit, and publish. Of course the next scientist who tries to carry out your experiment will find no such pattern, but you published, got some grants, and padded out your CV - so who cares? So poking at astrology again, imagine an astrologer observes a correlation that when people born when Venus was closest to Earth and also blocking out Mars had children who had a fertility rate 37% higher than average. It's pretty easy to see how you can now spin this into an causal astrological effect, where you are using past data (which was clearly just correlational) to make future predictions.

              And with enough hand-waving you'll be able to show it again in the future. Perhaps if it doesn't work out one way or another, just add "Ahh! Of course, we also need consider the relationship of Jupiter in the picture." Or perhaps it now has to do with with these two events and their relationship to the Equinox. Just make your model more and more complex so that it always shows what you want it to show, even when it is all based on a completely spurious correlation. This in general is also why the introduction of computer models and computer aided data digging have undoubtedly had a hugely negative impact on science, even though they, on the surface, sound like things that would instead be unimaginably positive for scientific pursuits. Now with computers and computer generated models, find a new spurious correlation can be done at the press of a button. And that's before we even get into "AI"...

        • (Score: 2) by Freeman on Thursday July 01 2021, @07:37PM

          by Freeman (732) Subscriber Badge on Thursday July 01 2021, @07:37PM (#1151965) Journal

          I know, I know, but why are we talking about Facebook's moderation system?

          --
          Joshua 1:9 "Be strong and of a good courage; be not afraid, neither be thou dismayed: for the Lord thy God is with thee"
        • (Score: 2) by driverless on Friday July 02 2021, @06:51AM

          by driverless (4770) on Friday July 02 2021, @06:51AM (#1152185)

          It wasn't that they had trouble proving anything, it was that they had no idea how statistics worked. The data was probably all there, but their ability to analyse it and draw conclusions was missing. As a result you couldn't draw any conclusions from any results they published without going through the analysis yourself to see whether they'd got it right. Going from my friend's experience - this was a first-year undergrad student having to stop and correct the errors in analysis being made by a tenured professor - I wouldn't put much faith in the reports being published.

      • (Score: 0) by Anonymous Coward on Thursday July 01 2021, @09:09PM

        by Anonymous Coward on Thursday July 01 2021, @09:09PM (#1152008)
    • (Score: 0) by Anonymous Coward on Thursday July 01 2021, @02:56PM

      by Anonymous Coward on Thursday July 01 2021, @02:56PM (#1151804)
      Its worse than that. We have people publishing papers citing other papers that have nothing to do with the subject. Nobody he ks the cotations. And 90% of ALL medical studies have replication failires. If you think the social sciences are bad, look at big pharma psychiatry studies. The DSM is a joke. It's safe to say now that psychiatry isn't a science.

      The approval of an alzheimers drug that has zero proof of working is just the latest. "We have to give people hope." You don't do that by fraud. But of course when hour paycht/bribe depends on it, you will justify anything. Just ask the Nazis - we were nust following orders.

      And its only with the discovery of mass graves of children that Canada is finally having to admit that it was straight-up genocide, not "only cultural genoct." Lying smug we're better than you fucks. (- an angry canadian)

  • (Score: 4, Funny) by c0lo on Thursday July 01 2021, @11:35AM (11 children)

    by c0lo (156) on Thursday July 01 2021, @11:35AM (#1151757) Journal

    University of California, San Diego, economists Marta Serra-Garcia and Uri Gneezy were interested in whether catchy research ideas would get more attention than mundane ones, even if they were less likely to be true.

    Ah, so they are "economy scientists", right? And, surprise, they came up with an idea of a catchy research.

    And then "they gathered data on 80 papers from three different projects". Oh, wow, that many?

    --
    https://www.youtube.com/watch?v=aoFiw2jMy-0
    • (Score: 3, Funny) by DannyB on Thursday July 01 2021, @03:04PM (2 children)

      by DannyB (5839) Subscriber Badge on Thursday July 01 2021, @03:04PM (#1151809) Journal

      Isn't citation-count a rather outmoded way of determining the quality of a paper in the 21st century?

      Shouldn't quality be determined by number of clicks on the title?

      --
      If you think a fertilized egg is a child but an immigrant child is not, please don't pretend your concerns are religious
      • (Score: 2) by Opportunist on Friday July 02 2021, @06:36AM (1 child)

        by Opportunist (5545) on Friday July 02 2021, @06:36AM (#1152177)

        Here's a novel idea, how about basing the quality of a paper on how many tried and succeeded to repeat the results?

        • (Score: 2) by DannyB on Friday July 02 2021, @04:02PM

          by DannyB (5839) Subscriber Badge on Friday July 02 2021, @04:02PM (#1152272) Journal

          I think you may be on to something!

          That fixes (at least) two problems.

          Published results that are fake and thus not reproducible.

          The incentive for some researchers to actually spend time and money to reproduce results. (You can't exactly get a grant for that right now.)

          --
          If you think a fertilized egg is a child but an immigrant child is not, please don't pretend your concerns are religious
    • (Score: 2) by DannyB on Thursday July 01 2021, @03:09PM (6 children)

      by DannyB (5839) Subscriber Badge on Thursday July 01 2021, @03:09PM (#1151813) Journal

      A long time ago, in a galaxy far, far away, the word "science" actually meant something.

      Now it is a suffix word you tack on to confer trust, credibility, study, rigor, care.

      How about some sort of pet food science? Oh, wait, we have science diet.

      What about a shampoo science?

      Fashion Design science.

      Gender Studies science.

      Masturbation science.

      --
      If you think a fertilized egg is a child but an immigrant child is not, please don't pretend your concerns are religious
      • (Score: 2, Interesting) by Anonymous Coward on Thursday July 01 2021, @03:28PM (3 children)

        by Anonymous Coward on Thursday July 01 2021, @03:28PM (#1151820)

        I think, more than anything, we are simply a "secular" (which is much more anti-religious than secular) society, which is rapidly going through the exact steps that historically would have led to the creation of religions. We've already laid out the dogma, the taboo, the general ethical and moral frameworks, and so on. But the problem is now that we need a God. Because without appeal to the ultimate Authority you get tangled down in petty bickering and disputes about nuances, logic, evidence, rationale, and all these other awful things. And so Science, with a capital S, is now becoming the new God.

        The fact that things such as dogma and taboo are entirely antithetical to everything that science stands for matters not to Science, because Science is not science. It is religion and it is God. And who is anybody to question Science? Even the input of a scientist is meaningless before that of a Scientist.

        • (Score: 5, Insightful) by DannyB on Thursday July 01 2021, @03:47PM (2 children)

          by DannyB (5839) Subscriber Badge on Thursday July 01 2021, @03:47PM (#1151833) Journal

          Science, with a capital S, is now becoming the new God.

          Science with a capitol S gives us reproducible results. Experiments with variables and controls. And statistical analysis. Consideration of what possible errors the results could contain.

          Science with a capitol S gives us real theories. Not what squishy S science calls "theories". A big S theory means it explains all available evidence to date, and makes predictions that can be verified now or in the future. (See Relativity for example. It was published with some predictions about the orbit of Mercury, which at the time could be verified. And predicted gravitational lensing which was soon observed. And other predictions that wouldn't be verified until much later.)

          If you don't like a theory of big S science, then all you need is one single reproducible result that falsifies the theory. Your result now becomes part of the body of observed evidence that the next theory must also explain.

          Climate change, for example, has observable data and predictions. The controversy is manufactured for political purposes.

          Darwin didn't know anything about DNA, but proposed in his theory that there must be some actual mechanism by which traits are passed on and some selected for by various survival pressures. Later DNA was discovered and is the actual mechanism.

          The only people saying science is the new God are people who are anti science for purely political reasons. Because they don't like the results.

          The earth is flat.
          Vaccines cause autism.
          The climate is not changing, and if it is, it's not our fault, and even if it were, it would be unprofitable to do something about it.
          5G causes covid, cancer and a list of other problems.
          Trump won the election.

          --
          If you think a fertilized egg is a child but an immigrant child is not, please don't pretend your concerns are religious
          • (Score: 2) by cmdrklarg on Thursday July 01 2021, @09:26PM

            by cmdrklarg (5048) Subscriber Badge on Thursday July 01 2021, @09:26PM (#1152018)

            +6 Insightful

            --
            Answer now is don't give in; aim for a new tomorrow.
          • (Score: 2) by deimtee on Friday July 02 2021, @03:33PM

            by deimtee (3272) on Friday July 02 2021, @03:33PM (#1152261) Journal

            Science with a capitol S gives us reproducible results. Experiments with variables and controls. And statistical analysis. Consideration of what possible errors the results could contain.

            I think you and the AC have simply swapped which s/S-cience you give the capital letter to. And in my opinion, the AC is correct. The capital S goes to the god-like entity which can not be challenged, not the practice of investigating the universe.

            --
            No problem is insoluble, but at Ksp = 2.943×10−25 Mercury Sulphide comes close.
      • (Score: 0) by Anonymous Coward on Thursday July 01 2021, @03:32PM

        by Anonymous Coward on Thursday July 01 2021, @03:32PM (#1151823)

        > How about some sort of pet food science? Oh, wait, we have science diet.

        I've talked to some people that work at Hills in Topeka. It sounds like they really do a lot of work there in regard to pet food nutrition. Including studies and trials with various cats and dogs that get treated very well. You might be surprised how much science they actually do, just saying.

      • (Score: 4, Funny) by c0lo on Thursday July 01 2021, @05:26PM

        by c0lo (156) on Thursday July 01 2021, @05:26PM (#1151882) Journal

        Masturbation science.

        Don't belittle it, even if lacking the prediction capabilities of science, that's a craft. Refineable to the art level. I hear many interpretative artists get good money for it on onlyfans.

        --
        https://www.youtube.com/watch?v=aoFiw2jMy-0
    • (Score: 2) by bzipitidoo on Thursday July 01 2021, @11:00PM

      by bzipitidoo (4388) Subscriber Badge on Thursday July 01 2021, @11:00PM (#1152064) Journal

      Are you saying their work is stinky bad, or at least minimal? They didn't science the shit out of it?

  • (Score: 1, Insightful) by crafoo on Thursday July 01 2021, @12:35PM (10 children)

    by crafoo (6639) on Thursday July 01 2021, @12:35PM (#1151767)

    Social Sciences are Fake News. Well, fake science at least. Cult ideology and propaganda cloaked in the trappings of the scientific process.

    • (Score: 0) by Anonymous Coward on Thursday July 01 2021, @01:54PM

      by Anonymous Coward on Thursday July 01 2021, @01:54PM (#1151784)

      Damn straight! The only real sciences are the Antisocial Sciences that make your brain hurt when you study them.

    • (Score: 3, Interesting) by HiThere on Thursday July 01 2021, @02:05PM (8 children)

      by HiThere (866) on Thursday July 01 2021, @02:05PM (#1151789) Journal

      If you said "...often fake news..." I'd agree with you. And very often weak science. Some of it's good, though. And a lot of chemistry isn't all *that* good.

      This one looks like an example of "weak science". 80 isn't a large number of papers. (OTOH, I wasn't interested enough to more than read the summary.)

      To be good science they'd need a much larger number of papers, and they'd need to stratify "cites" into categories of at least "agreed with" and "disagreed with". But their point that "cite counts" is a very poor measure of paper quality it probably correct.

      --
      Javascript is what you use to allow unknown third parties to run software you have no idea about on your computer.
      • (Score: 3, Informative) by Socrastotle on Thursday July 01 2021, @02:35PM (7 children)

        by Socrastotle (13446) on Thursday July 01 2021, @02:35PM (#1151797) Journal

        Statistics can often be somewhat paradoxical. You can, assuming you have a random and representative sample, derive conclusions with extremely small samples.

        80, for the purposes here, is huge. One simple way to start to play with numbers a bit is to use a binomial calculator [stattrek.com]. So let's say we believe that 70% of studies are, on average, replicable. That's already abysmal, but I want to lowball things here to emphasize the point. And so we do a replication study of 80 different studies that we believe are representative of the sample we're considering, and are randomly selected. I'd normally expect to get around 56 "successes" (80 * 0.7) but instead I only got 40. That's "only" 16 away, and the sample size is relatively small. So if I assume that the real rate is indeed 70%, how often should only 40 or fewer replicate?

        The answer is

        0.014%

        . And emphasizing, that is a percent, so in other words this would be expected to happen, by random chance, less than

        1 in 7000

        times. So you can say with an exceptionally high degree of certainty that the real rate is not 70%. If you don't follow how to use the calculator:

        Probability of success = 0.7
        Number of trials = 80
        Number of successes = 40

        And we're looking at P(X = x).

        • (Score: 0) by Anonymous Coward on Thursday July 01 2021, @02:59PM (1 child)

          by Anonymous Coward on Thursday July 01 2021, @02:59PM (#1151806)
          >> Let's say 70% of studues are replicable (that's probably lowball)

          People who have done replication stidies found 90% were not replicable.

          • (Score: 2) by Socrastotle on Thursday July 01 2021, @03:19PM

            by Socrastotle (13446) on Thursday July 01 2021, @03:19PM (#1151817) Journal

            I'm not familiar with any that low, but I do know that social psychology in particular has a replication rate of around 25%.

            I was speaking in terms of ideals. In general what percent of our science would we like to be "valid"? Of course 100%, but that's impractical. So what is practical? Perhaps something like 97%. But with such a high figure it becomes intuitively obvious that hitting e.g. 50% over 80 samples is not just noise. But what about hitting 50% when you're only aiming for 70%? It doesn't, intuitively, seem so implausible, especially over "only" 80 samples.

            So the point is that even if you want to set our "real" figure far lower than anybody would ever actually want or think (again - 70% is an abysmal replication rate for idealized "science"), 80 samples is far more than enough to draw some extremely high probability conclusions.

        • (Score: 1) by shrewdsheep on Thursday July 01 2021, @05:05PM (3 children)

          by shrewdsheep (5215) on Thursday July 01 2021, @05:05PM (#1151871)

          I guess your intention is to calculate a P-value for the observation of 40 successes for 80 replications under the null hypothesis of a success rate of .7. To get a meaningful P-value, you would have to calculate the probability P(X ≤ 40). The probability of a simple outcome (i.e. P(X = x)) is almost always meaningless, as a matter of fact it is always zero for continuous distributions. Also for the binomial it tends to zero for any outcome and success probability as N tends to infinity. The P-value is the probability of the event of observing our outcome plus all more extreme outcomes.

          • (Score: 2) by Socrastotle on Thursday July 01 2021, @05:14PM (2 children)

            by Socrastotle (13446) on Thursday July 01 2021, @05:14PM (#1151878) Journal

            It is the P(X <= x) of course. I got HTML'd with the less than sign.

            • (Score: 2) by Anti-aristarchus on Thursday July 01 2021, @09:02PM (1 child)

              by Anti-aristarchus (14390) on Thursday July 01 2021, @09:02PM (#1152006) Journal

              But shouldn't it be:

              P(A|B) = [P(A) P(B|A) /P(B)]

                  One must take prior probabilities into account, whether frequentist or subjectivist.

              • (Score: 2, Informative) by shrewdsheep on Monday July 05 2021, @07:19AM

                by shrewdsheep (5215) on Monday July 05 2021, @07:19AM (#1152967)

                As a frequentest your prior would be uniform (and might therefore be improper), if you are an empirical Bayesian you would again have a uniform (improper) prior, but on the hyperparameters, and as a full Bayesian, well, the full fudging would start.

        • (Score: 2) by HiThere on Thursday July 01 2021, @05:26PM

          by HiThere (866) on Thursday July 01 2021, @05:26PM (#1151880) Journal

          From the summary, it seems clear that the sample was not "random and representative sample" as what they did was "So they gathered data on 80 papers from three different projects".

          --
          Javascript is what you use to allow unknown third parties to run software you have no idea about on your computer.
  • (Score: 1) by shrewdsheep on Thursday July 01 2021, @01:00PM (2 children)

    by shrewdsheep (5215) on Thursday July 01 2021, @01:00PM (#1151770)

    To start with, citing does not imply approving. Controversial papers can accrue a lot of citations. Then comes the number of co-authors. If you have hundreds of co-authors each only self-citing occasionally, such papers rake in thousands of citations. Then comes the field, with short-lived research such as epidemiologcal/social science studies, analyzing the influence of a certain factor on some outcome being easily digestible and therefore citable. More fundamental STEM papers often take years to influence the community. By many metrics those are seen as unimportant which is why reputation of journals still matters a lot in these fields.
     

    • (Score: 2) by looorg on Thursday July 01 2021, @02:15PM

      by looorg (578) on Thursday July 01 2021, @02:15PM (#1151792)

      Kind of funny, an economist tries to claim they are hard science and is outraged over that other soft sciences are quoted more often then something else. It has nothing to do with quality.

      The reason they are quoted more often is that they make the news more often cause they are things that are relatable to normal (or non-academic) people. People can understand on some level social science research since it actually relates to them and their lives more often. While they don't understand math, physics, biology research as much cause it requires a lot more specialized education and are on a level that is just not very interesting to average Joe and Jane.

      The failure to replicate is probably not even an interesting factor here, unless they are cited cause they are so bad so that is why they get cited. But that is not very likely.

    • (Score: 2) by JoeMerchant on Thursday July 01 2021, @04:14PM

      by JoeMerchant (3937) on Thursday July 01 2021, @04:14PM (#1151849)

      The 12% acknowledgement of demonstrated non-repeatability is... telling.

      --
      Україна досі не є частиною Росії Слава Україні🌻 https://news.stanford.edu/2023/02/17/will-russia-ukraine-war-end
  • (Score: 4, Interesting) by Socrastotle on Thursday July 01 2021, @01:42PM (17 children)

    by Socrastotle (13446) on Thursday July 01 2021, @01:42PM (#1151776) Journal

    In the past when the media said something, you could generally take it with some reasonable degree of assurance that it was very likely to be true. In the worst case it was generally just misleading.

    Now a days? There's a pretty good chance that it's not only misleading but completely fake. One of the most overt examples of this in recent times was the Officer during the January 6th riots/protests who was 'killed after a rioter smashed his head in with a fire extinguisher.' It turns out he was unharmed during the protests/riots, and died of completely natural causes (stroke from a preexisting blood clot) long after all the riots/protests were over. And so people increasingly only believe news that confirms their own biases, because they *want* to believe it - with relatively little regard to what they genuinely and objectively think the chances of it being true are.

    Why did the media start acting this way? Because hyperbole and sensationalism sells, and lies sell even better.

      - "Scientists discover star with unusual pattern of dimming." = 3 clicks
      - "Scientists discover proof aliens are building giant Dyson Sphere, and they're close." = 3,000,000 clicks.

    We've always had crap like the latter in things like the National Enquirer, but people just shrugged because the paper had long since destroyed any reputation it had. Now a days previously reputable papers have swapped to the exact same nonsense, but the frequently political / bias confirming angle of it all (alongside the fact that these papers used to be "real") is making people slow to accept what's happened. Somebody repeating crop circles each week is a bit different than somebody repeating everything you want to be true (but probably, deep down enough, know isn't) every week.

    And I think the same is increasingly becoming true of science. Social Psychology in particular has a 25% [wikipedia.org] replication rate. If you take anything stated in a paper related to social psychology (which is the domain of racism, systemic bias, social interactions, social interactions with regards to sexuality, etc) and simply assume the exact opposite of what it says - then you'd be vastly more informed than somebody who takes the "research" at face value. But because people *want* to believe what's said, they're slow to accept what's happened.

    And this is going to send confidence and trust in science plummeting to levels proportional to those of the media. In other news, America now has the least trusted [ox.ac.uk] media in the developed world. And our science isn't far behind if this doesn't change.

    • (Score: 0) by Anonymous Coward on Thursday July 01 2021, @02:23PM (9 children)

      by Anonymous Coward on Thursday July 01 2021, @02:23PM (#1151794)

      You are assuming people won't believe what they are being fed. When in fact, people are believing it. The News Paper or News Channel said it, so they believe it. Survey's say the media trust is low in the US. That isn't stopping people from believing what they are being told. Just look at the state of US Society and how divided it is on almost everything. People are not thinking for themselves, and they aren't accepting people who think for themselves. Believe the Social Science news reports, and if you don't you are wrong. Weird situation we are in.

      • (Score: 5, Interesting) by Socrastotle on Thursday July 01 2021, @03:06PM (1 child)

        by Socrastotle (13446) on Thursday July 01 2021, @03:06PM (#1151811) Journal

        Ah! But again I think you're not considering the order of causality here. Somebody does not become right wing because they stumbled onto and started reading Breitbart. Instead people with right wing tendencies seek out Breitbart and then believe what they read because they *want* it to be true. And similarly, somebody does not become left wing because they picked up a copy of the NYTimes and started reading it. Instead people with left wing tendencies seek out the NYTimes and then believe what they are told because they *want* it to be true. If you swapped the ideological narrative of both sites, yet kept the same respective quality (or lack thereof) of truthfulness, their readers would most certainly suddenly stop believing what they were told. Because it would not be what they *want* to be true.

        And so people are thinking for themselves, on at least a basic level. They choose what they want to be true, and then seek out information to confirm that view. If their news sources of choice stop confirming that view? No problem they just move to the next one that does. In the age of the internet, they are limitless.

        • (Score: 0) by Anonymous Coward on Thursday July 01 2021, @06:42PM

          by Anonymous Coward on Thursday July 01 2021, @06:42PM (#1151942)

          If you swapped the ideological narrative of both sites, yet kept the same respective quality (or lack thereof) of truthfulness, their readers would most certainly suddenly stop believing what they were told.

          I used to believe that, but I'm no longer sure. The Trump presidency stands as empirical evidence of that. There are literally too many instances to count, but the big one which got me was how quickly "Lock Her Up" disappeared when Trump said a word about not wanting to pursue that anymore. And remember how much of the Republican Party was "Never Trump"ers, but how quickly that changed?

          As another example, consider the Democrat's (and the "left") opinion of Russia. It want from "why can't we all get along, this isn't the cold war anymore" to "you don't understand, they are a major threat to our democracy." Likewise, the Republican's (and the "right") quickly went to "you can never trust a Russian" to "ehh, Putin isn't such a bad guy."

          My current theory is that people establish their affiliation with something (be it an idea, a group, or what have you), and then that defines them. If that group changes, then most people go along with that change. If The Party announced "the cacao ration has been raised from 10 grams to 8 grams," sure, some would question it... but I think a lot more would cheer the party for doing such a great job.

          (Which is why it is all the more important for Benjamins of the world to not just be cynical asses and to actually do something to help the situation... to mix fictional metaphores.)

      • (Score: 3, Insightful) by Anonymous Coward on Thursday July 01 2021, @03:07PM (6 children)

        by Anonymous Coward on Thursday July 01 2021, @03:07PM (#1151812)

        My favorite is when you point out to someone a particular claim is completely false with documentation, sometimes even a retraction. Yet they repeat the lie even if they don't completely believe it because somehow it makes them feel good. Example lie: Donald Trump suggested people inject themselves with bleach for coronavirus. What moron could've ever believed Trump would say this in the first place?

        • (Score: 2, Flamebait) by DeathMonkey on Thursday July 01 2021, @03:20PM (5 children)

          by DeathMonkey (1380) on Thursday July 01 2021, @03:20PM (#1151818) Journal

          What moron could've ever believed Trump would say this in the first place?

          The morons who believe their own eyes and ears over the words of a liar?

          Here, try for yourself. [youtube.com]

          "And then I see the disinfectant where it knocks it out in a minute. One minute. And is there a way we can do something like that, by injection inside or almost a cleaning?

          • (Score: 5, Insightful) by PiMuNu on Thursday July 01 2021, @03:37PM (4 children)

            by PiMuNu (3823) on Thursday July 01 2021, @03:37PM (#1151827)

            To state the obvious, he isn't suggesting people inject themselves except through a very perverse reading of the words. You are exactly confirming GP's point.

            • (Score: 2) by DeathMonkey on Thursday July 01 2021, @04:33PM (3 children)

              by DeathMonkey (1380) on Thursday July 01 2021, @04:33PM (#1151855) Journal

              He said doctors should try injecting people with bleach. Is that supposed to make it less stupid?

              • (Score: 3, Insightful) by Socrastotle on Thursday July 01 2021, @05:33PM

                by Socrastotle (13446) on Thursday July 01 2021, @05:33PM (#1151887) Journal

                No, he said that disinfectant has the ability to completely destroy the virus in the realm of 1 minute and it'd be interesting to see if doctors could find a way to inject it or somehow use it for a "cleaning."

                Consider, for instance, antibiotics. Part of the reason the uptake was so slow (it took more than a decade) on penicillin, the first antibiotic, is because it's extracted directly from the penicillium mold. You know that bluish green fungus that forms of rotting food? That is (or at least may contain) penicillium molds, which can be harmful to you in a number of ways if directly exposed. The idea of isolating that and then injecting it just because it seemed to destroy some bacteria in a petri dish is, to say the least, counter-intuitive, and so the discoverer had substantial difficulty simply getting people to accept what he claimed - which would ultimately revolutionize healthcare and the entire world.

                It's like saying in that case that Fleming was suggesting people toss some mold into a blender and inject. That's just being intentionally obtuse, at best - and, again, rather emphasizing the above posters points.

              • (Score: 1, Insightful) by Anonymous Coward on Thursday July 01 2021, @05:35PM

                by Anonymous Coward on Thursday July 01 2021, @05:35PM (#1151891)

                You do see the question mark at the end, dont you?

              • (Score: 4, Touché) by PiMuNu on Friday July 02 2021, @07:33AM

                by PiMuNu (3823) on Friday July 02 2021, @07:33AM (#1152190)

                No, he is asking for research into using bleach or equivalent stuff to kill coronavirus. It clearly is a stupid idea, but that is irrelevant.

                You have *twice* deliberately misinterpreted what Trump said to make a political point, which confirms exactly what the GGP said - i.e. people don't look at evidence, they would far rather confirm their own political bias.

    • (Score: 2) by DannyB on Thursday July 01 2021, @03:53PM (4 children)

      by DannyB (5839) Subscriber Badge on Thursday July 01 2021, @03:53PM (#1151836) Journal

      In the past when the media said something, you could generally take it with some reasonable degree of assurance that it was very likely to be true. In the worst case it was generally just misleading.

      In the past, the media was motivated to provide real news. Some strange thing called Journalism. Who, what, when, where, why, how.

      And some editorial opinion, which could be misleading.

      Ownership of a news outlet could bias its coverage to be a bit misleading.

      Yet the facts were true. They even had offices of factual verification.

      Now, the media has become entertainment. Or infotainment. And the news outlet's owners have no shame about their news infotainment being heavily biased. To such an extreme that in 2013 when the Snowden story broke, CNN didn't even make a pretense of objectivity, there simply couldn't possibly be any other POV than the government's view.

      --
      If you think a fertilized egg is a child but an immigrant child is not, please don't pretend your concerns are religious
      • (Score: 3, Interesting) by JoeMerchant on Thursday July 01 2021, @04:21PM (3 children)

        by JoeMerchant (3937) on Thursday July 01 2021, @04:21PM (#1151850)

        What past are you all living in? The past where the Yellow press pushed the U.S. into the Spanish American War? The past where Northern and Southern ideals were so strongly reinforced that a civil war broke out? Or the past where Walter Cronkite read the news to you and because he hadn't made a total self contradictory ass of himself for his employer, you trusted him?

        The past I remember most clearly is the one where our local news programs would read editorials in line with their political views and present them very professionally, very convincingly, and then a week or six later let some local kook on for 1/3 the time to present an "opposing viewpoint" by someone who clearly doesn't do this for a living, probably wrote their own copy, probably dropped out of high school after failing English too many times, stumbles over their words, and has makeup that makes them look like a carnival clown who slept in late before going on camera.

        --
        Україна досі не є частиною Росії Слава Україні🌻 https://news.stanford.edu/2023/02/17/will-russia-ukraine-war-end
        • (Score: 2) by Socrastotle on Friday July 02 2021, @03:59PM (2 children)

          by Socrastotle (13446) on Friday July 02 2021, @03:59PM (#1152267) Journal

          One can only speak for themselves, but I am speaking of times I described in a post below. [soylentnews.org] I rarely recall the media in 'the golden age' going so nutters over political stuff. There is always some degree of implicit bias based on what is covered and what isn't covered, but now a days when you look at the front page even of formerly reputable papers like the NYTimes, it's scarcely different from what used to be relegated to things like post-debate 'spin rooms'. The Fox News model of bias + coverage has not only become the rule, but has become intensely magnified. There seems to be little to nothing left in the way of remotely impartial coverage.

          Of course I have to also consider the possibility that it's a product of [relative] youth. I only began to be somewhat politically aware around the Iraq War and, lo and behold, from my perspective the complete deterioration of the media also began at just about the same time. Of course there are many things that make me suspect this is not the case. This [brookings.edu] article from Bob Kaiser (who had a 50 year career during the 'golden era' of the Washington Post working his way up to managing editor), suggests as much from an individual with a rather radically different perspective and age. And the case he make as an insider directly match the observations I have seen from the outside.

          In any case, I've no delusions of the past of our media being ideal. But I do believe it was orders of magnitude better than its state today which seems designed entirely to generate clicks. And the problem with this is that impartial reporting doesn't generate clicks. Hyperbole, sensationalism, exploiting emotions, appealing to biases, even indeed even lying to generate even more sordid tales - those sort of things generate clicks. At least until they don't. When people begin appreciating that most of all modern media is little different than the "National Enquirers" that we grew up with, the show will be over.

          • (Score: 2) by JoeMerchant on Wednesday July 21 2021, @02:06PM (1 child)

            by JoeMerchant (3937) on Wednesday July 21 2021, @02:06PM (#1158738)

            The Fox News model of bias + coverage has not only become the rule, but has become intensely magnified.

            As compared to the Cronkite days, I agree. But deeper in the past I think bias + coverage + more bias was quite common, particularly in the print media when not everyone could read and they got bias from the people who did the reading for them plus the people who paid for the printing and distribution.

            --
            Україна досі не є частиною Росії Слава Україні🌻 https://news.stanford.edu/2023/02/17/will-russia-ukraine-war-end
            • (Score: 2) by Socrastotle on Wednesday July 21 2021, @03:00PM

              by Socrastotle (13446) on Wednesday July 21 2021, @03:00PM (#1158745) Journal

              I also get the exact same impression:

              It is a melancholy truth, that a suppression of the press could not more compleatly deprive the nation of it's benefits, than is done by it's abandoned prostitution to falsehood. Nothing can now be believed which is seen in a newspaper. Truth itself becomes suspicious by being put into that polluted vehicle. The real extent of this state of misinformation is known only to those who are in situations to confront facts within their knolege with the lies of the day.

              I really look with commiseration over the great body of my fellow citizens, who, reading newspapers, live & die in the belief, that they have known something of what has been passing in the world in their time; whereas the accounts they have read in newspapers are just as true a history of any other period of the world as of the present, except that the real names of the day are affixed to their fables.

              General facts may indeed be collected from them, such as that Europe is now at war, that Bonaparte has been a successful warrior, that he has subjected a great portion of Europe to his will, &c., &c.; but no details can be relied on. I will add, that the man who never looks into a newspaper is better informed than he who reads them; inasmuch as he who knows nothing is nearer to truth than he whose mind is filled with falsehoods & errors. He who reads nothing will still learn the great facts, and the details are all false.

              That quote [uchicago.edu] was from Thomas Jefferson, 1807. And there were countless other similar sentiments from various founding fathers and other great thinkers of the times. I used to believe these sentiments were part rhetorical, part metaphor. But now we're entering into an era where what he said, even taken completely at face value, is not even especially hyperbolic.

              So in many ways I think our current era, as crazy as seems things is not really new. Rather we "all" (being the majority of humans born in the last 50-60 years in the developed world), basically just grew in a sort of mini-utopia that was little more than a bubble in time. And so as that bubble pops it kind of feels like we're heading towards some unprecedented insanity. But no, it's quite precedented. What isn't precedented is how nice we had things for some time.

              This also even applies if you go much further back. Reading the writings of Aristotle, Plato, Socrates, Aurelius, and so on - it feels very much as if they are describing the world we now live in. "The Republic" in particular is practically a transcription of the happenings of the day.

    • (Score: 1, Touché) by Anonymous Coward on Thursday July 01 2021, @06:33PM (1 child)

      by Anonymous Coward on Thursday July 01 2021, @06:33PM (#1151936)

      In the past when the media said something, you could generally take it with some reasonable degree of assurance that it was very likely to be true. In the worst case it was generally just misleading.

      Really, now?

      To your request of my opinion of the manner in which a newspaper should be conducted, so as to be most useful, I should answer, `by restraining it to true facts & sound principles only.' Yet I fear such a paper would find few subscribers. It is a melancholy truth, that a suppression of the press could not more compleatly deprive the nation of it's benefits, than is done by it's abandoned prostitution to falsehood. Nothing can now be believed which is seen in a newspaper. Truth itself becomes suspicious by being put into that polluted vehicle. The real extent of this state of misinformation is known only to those who are in situations to confront facts within their knowlege with the lies of the day. I really look with commiseration over the great body of my fellow citizens, who, reading newspapers, live & die in the belief, that they have known something of what has been passing in the world in their time; whereas the accounts they have read in newspapers are just as true a history of any other period of the world as of the present, except that the real names of the day are affixed to their fables. General facts may indeed be collected from them, such as that Europe is now at war, that Bonaparte has been a successful warrior, that he has subjected a great portion of Europe to his will, &c., &c.; but no details can be relied on. I will add, that the man who never looks into a newspaper is better informed than he who reads them; inasmuch as he who knows nothing is nearer to truth than he whose mind is filled with falsehoods & errors. He who reads nothing will still learn the great facts, and the details are all false.

      Thomas Jefferson to John Norvell, June 11, 1807

      Just how far into the past shall we find this utopia of honest journalism?

      • (Score: 2) by Socrastotle on Friday July 02 2021, @05:40AM

        by Socrastotle (13446) on Friday July 02 2021, @05:40AM (#1152165) Journal

        In general I'm referring to times much more recent. The media started to rapidly deteriorate with the advent of the internet. There's a phenomenal article about this by Robert Kaiser, The Bad News About the News [brookings.edu]. Kaiser worked at the Washington Post for 50 years, seeing it from its peak as a reputable paper to its more recent collapse. I could not recommend that article more highly, and I don't think I can do it justice with cliff notes.

        And so I suspect ultimately that the hey day of the media, perhaps some time between 1960 - 1990, give or take some years either way, was likely an anomaly in time. And so our modern media has become much more similar to the media Jefferson was familiar with where in an effort to move a rag (or in this case, generate), they will say anything and everything - regardless of its truthfulness. Yet because times prior to the that golden age of media feel like they may as well have been a millennia ago, we perceive the media in terms of its current abysmal state, and only have what we know came before to contrast it against.

        It's quite remarkable that the NYTimes went from, in 1971, publishing the Pentagon Papers [wikipedia.org] to, just 30 years later, publishing outright intelligence agency propaganda such as "Irrefutable and Undeniable [nytimes.com] where they unabashedly tried to sell a war, built on fabrications and lies, to the American people. And it's only been downhill from there. The contrast makes the decline all the more overt.

  • (Score: 0, Troll) by Anonymous Coward on Thursday July 01 2021, @06:42PM (1 child)

    by Anonymous Coward on Thursday July 01 2021, @06:42PM (#1151941)

    Because females exist.

    • (Score: 2, Insightful) by Anonymous Coward on Thursday July 01 2021, @09:16PM

      by Anonymous Coward on Thursday July 01 2021, @09:16PM (#1152012)

      True, in the sense that if they did not, we would also not and thus there would be no humans to study, or to study them.

(1)