Stories
Slash Boxes
Comments

SoylentNews is people

SoylentNews is powered by your submissions, so send in your scoop. Only 16 submissions in the queue.

We Must Demand Evidence of Peer Review

posted by martyb on Wednesday May 23 2018, @08:20AM   Printer-friendly
from the peerless-recommendation dept.

canopic jug writes:

The Scientist has an opinion piece that insufficient evidence of peer review is happening in scholarly publishing. In it, the author writes a call for publishing of anonymized peer reviews.

Scientific rigor demands that claims be substantiated by evidence. If I claim that gene A regulates gene B and provide no evidence, my claim will be dismissed. It must be dismissed. Yet, if a journal claims to conduct peer review and provides no evidence of it, the claim is rarely dismissed.

However, given the specialized nature of some disciplines and the small number of researchers, it is likely that the anonymity would not last for long. How do Soylentils weigh in on the opinion piece?

[Ed's Comment: The link is unreliable, but patience tends to get through eventually]

Original Submission

 
This discussion has been archived. No new comments can be posted.
We Must Demand Evidence of Peer Review | Log In/Create an Account | Top | 51 comments | Search Discussion
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.

  • (Score: 2) by c0lo on Thursday May 24 2018, @06:39AM (3 children)

    by c0lo (156) Subscriber Badge on Thursday May 24 2018, @06:39AM (#683447) Journal

    You can make up the data any way you want.

    Generate yourself a over 1000 points dataset picking at random levelA,B in [-1, 1] and take
    levelC = exp(-(levelA+levelB))^2) -0.5 + 0.01*unit_random_noise.
    Forget how the data was generated (you never knew anyway), then study on the data set what correlations you want using only pairs of dimensions, see where this gets you.

    --
    https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
    Starting Score:    1  point
    Karma-Bonus Modifier   +1  
    Total Score:   2  

  • (Score: 0) by Anonymous Coward on Thursday May 24 2018, @01:24PM (2 children)

    by Anonymous Coward on Thursday May 24 2018, @01:24PM (#683534)

    Ok, here it is in R (There was an extra parentheses in your level_C, so I'm not sure if I did it right):

    set.seed(8008135)
    n   = 5
    dat = data.frame(A = sample(c(-1, 1), n, replace = T),
                     B = sample(c(-1, 1), n, replace = T))

    dat$C = exp(-(dat$A + dat$B)^2) - 0.5 + 0.01*rnorm(n)

    It looks like this:

       A  B          C
    1 -1 -1 -0.4850803
    2  1  1 -0.4889374
    3 -1  1  0.5022517
    4 -1  1  0.5167859
    5  1 -1  0.5177900

    In this case you have defined A and B to be independent. I am saying this never happens in nature, the correlations may get negligibly small and difficult to detect but they are always there when enough effort is put forth.

    This was already tested and well known by the 1960s:

    These armchair considerations are borne out by the finding that in psychological and sociological investigations involving very large numbers of subjects, it is regu- larly found that almost all correlations or differences between means are statisti- cally significant. See, for example, the papers by Bakan [1] and Nunnally [8]. Data currently being analyzed by Dr. David Lykken and myself, derived from a huge sample of over 55,000 Minnesota high school seniors, reveal statistically signifi-cant relationships in 91% of pairwise associations among a congeries of 45 miscel-laneous variables such as sex, birth order, religious preference, number of siblings, vocational choice, club membership, college choice, mother’s education, dancing, interest in woodworking, liking for school, and the like. The 9% of non-significant associations are heavily concentrated among a small minority of variables having dubious reliability, or involving arbitrary groupings of non-homogeneous or non-monotonic sub-categories. The majority of variables exhibited significant relation- ships with all but three of the others, often at a very high confidence level (p < 10^–6).

    https://meehl.dl.umn.edu/sites/g/files/pua1696/f/074theorytestingparadox.pdf [umn.edu]

    • (Score: 2) by c0lo on Thursday May 24 2018, @01:52PM (1 child)

      by c0lo (156) Subscriber Badge on Thursday May 24 2018, @01:52PM (#683539) Journal

      When I wrote [-1, 1] I intended the continuous interval between -1 and 1 including the limits. Should I have used the [-1..1] notation?

      In this case you have defined A and B to be independent.

      Not quite true. Taking values for any two of them, the third is determined.
      If you want, you can express it as an implicit function under the form f(A, B, c)=0 (something like "homoeostasis happens only when the level of activation of these 3 genes follows this relation..." - without knowing those 3 genes are dependent somehow, using simple linear correlations between will tell you nothing).

      --
      https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford

      • (Score: 0) by Anonymous Coward on Thursday May 24 2018, @02:44PM

        by Anonymous Coward on Thursday May 24 2018, @02:44PM (#683564)

        Should I have used the [-1..1] notation?

        Since you were referring to "levels", saying [-1,1] looks like "the set of -1 and 1" to me. I don't think it really mattesr, either way the situation is set up so there will be no correlation.

        Not quite true. Taking values for any two of them, the third is determined.

        That isn't what independent means. There is no feedback from C onto A or B in your model. I am saying that in any real situation there will be feedback. Then the values of A and B will be dependent upon C and each other. Then there will be a correlation.

        without knowing those 3 genes are dependent somehow, using simple linear correlations between will tell you nothing

        Sure it will, it will tell you that the expression of those genes are correlated with each other. I am saying this is worthless info since we already know it to be true.

        "homoeostasis happens only when the level of activation of these 3 genes follows this relation..."

        This is exactly what biologists like in TFA should be doing. That is not what they are doing. As I noted my the original comment, they are looking for the existence of simple correlations.