The Scientist has an opinion piece that insufficient evidence of peer review is happening in scholarly publishing. In it, the author writes a call for publishing of anonymized peer reviews.
Scientific rigor demands that claims be substantiated by evidence. If I claim that gene A regulates gene B and provide no evidence, my claim will be dismissed. It must be dismissed. Yet, if a journal claims to conduct peer review and provides no evidence of it, the claim is rarely dismissed.
However, given the specialized nature of some disciplines and the small number of researchers, it is likely that the anonymity would not last for long. How do Soylentils weigh in on the opinion piece?
[Ed's Comment: The link is unreliable, but patience tends to get through eventually]
(Score: 0) by Anonymous Coward on Wednesday May 23 2018, @08:53AM (26 children)
Gene A does "regulate" (affect the expression of) gene B, thats for any A and B. Its just a matter of how much and in what way. Do not want the author of that to be peer reviewing anything.
(Score: 1, Funny) by Anonymous Coward on Wednesday May 23 2018, @09:04AM (9 children)
Yeah, but those sciency egg-heads are hiding something! Science should be like the Protestant bible, where anyone can read it and clearly discern its true meaning!
(Score: 2, Informative) by khallow on Wednesday May 23 2018, @09:10AM (7 children)
Bad analogy, but otherwise accurate. One of the key purposes of scientific endeavors is to disseminate that knowledge to a variety of parties, including the outside world. After all, what is the point of a few dozen people knowing something, only to die within a few decades? Doesn't do the rest of us any good.
(Score: 3, Insightful) by aristarchus on Wednesday May 23 2018, @11:15PM (6 children)
No! Very precise analogy, or as precise as one could be without involving automobiles. Opening up peer-review is not like disseminating knowledge, it is more like allowing those with very little knowledge, say, no knowledge of Latin, read a translation of a book they do not understand, and draw their own conclusions about what it means based on their own invincible ignorance. That is Protestantism. Not as bad, perhaps, as the Traditional Catholic position, that such things as catechism are best left to the experts, and couched in a language the common folk cannot comprehend (Catholics and Harry Potter fans agree on this), but it is almost certainly destined to emerge in Westboro Baptists and Anthropogenic Global Warming Deniers. There is a reason it is PEER review, and not "average internet expert" review. Trust me on this, khallow!
(Score: 1) by khallow on Thursday May 24 2018, @02:13AM
That's the positive aspect of post-modernism in a nutshell. Not feeling the concern way over here.
(Score: 0) by Anonymous Coward on Thursday May 24 2018, @03:55AM (4 children)
We need to use taxes to fund independent replication. Of course, using tax money for public good instead of private pork barrel good would instantly turn us into Nazi Germany and Stalinist Russia simultaneously.
(Score: 3, Funny) by aristarchus on Thursday May 24 2018, @06:39AM
Agreed. We need to leave this to the incredible (literally) engine of private enterprise! Who are you gonna believe, Big Pharma, or your own inevitable demise?
(Score: 1) by khallow on Thursday May 24 2018, @08:50AM (2 children)
Because using taxes for the initial experiments worked so well in the first place that now independent replication is necessary. But how independent will said replication be when they have the same funding source?
(Score: 0) by Anonymous Coward on Friday May 25 2018, @06:07PM (1 child)
You don't understand science, do you?
(Score: 1) by khallow on Saturday May 26 2018, @03:26AM
(Score: 1, Funny) by Anonymous Coward on Wednesday May 23 2018, @03:01PM
It already is like that. Tell me any claim about human health and Ill find evidence for it in the literature.
(Score: 2) by c0lo on Wednesday May 23 2018, @09:49AM (15 children)
Possible translation: "Any gene has influence on all others, only the magnitude of the influence vary. Don't trust anyone who tells you otherwise for a peer review".
Is this what you wanted to say? 'Cause it's extremely confusing.
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: 0) by Anonymous Coward on Wednesday May 23 2018, @02:59PM (14 children)
Yep, you got it. Its been obvious for awhile that everything regarding human biology and behaviour is correlated with everything else regarding human biology and behaviour. So all these people discovering that correlations exist are just wasting time, and you do not want timewasters involved in peer review. What is confusing?
(Score: 2) by c0lo on Wednesday May 23 2018, @10:10PM (12 children)
You forgot the more-than-by-pairs interactions. You know? like a system of equations.
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: 0) by Anonymous Coward on Thursday May 24 2018, @12:32AM (11 children)
Not sure what you mean. Can you give an example where there is no correlation measured between any pair? It doesn't need to be real, made up data will do.
(Score: 2) by c0lo on Thursday May 24 2018, @12:47AM (9 children)
I mean influencing interrelations between N-tuples of genes with N>2
Note: your original statement is quite a hard one to disprove within the limits of the current knowledge/technology. But it's equally hard to prove.
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: 0) by Anonymous Coward on Thursday May 24 2018, @01:02AM (8 children)
I never talked about "proving" anything, it is a matter of evidence not mathematics. There is no a priori reason the universe needs to be that way, but it is.
However, can you give one example of what you are talking about with fake data and calculations?
(Score: 2) by c0lo on Thursday May 24 2018, @02:33AM (7 children)
Say the effect one gene has on another is described by a unitary/orthogonal matrix [wikipedia.org] with N>2.
Then, depending where your start, if you iteratively apply the transform matrix over a "gene group's activation-level vector" you are going to fall into one or the other eigenvector of that matrix.
Or any system in which you need to consider the influences of more than 2 parts to determine the evolution of a system.
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: 0) by Anonymous Coward on Thursday May 24 2018, @04:09AM (6 children)
Please just give one actual example. Do the correlation calculations with large sample size to show what you are talking about does not lead to guaranteed pairwise correlations...
(Score: 2) by c0lo on Thursday May 24 2018, @04:43AM (5 children)
I'm talking about more than pairwise influence.
Pure imagination, this may mean that, depending on the value of activation of a gene C, the influence of a gene A over a gene B may be show by a positive/negative or even no correlation at all.
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: 0) by Anonymous Coward on Thursday May 24 2018, @04:56AM (4 children)
I really don't see what the problem is. Please go beyond using imagination and give a proof of concept dataset and calculation. You can make up the data any way you want.
(Score: 2) by c0lo on Thursday May 24 2018, @06:39AM (3 children)
Generate yourself a over 1000 points dataset picking at random levelA,B in [-1, 1] and take
levelC = exp(-(levelA+levelB))^2) -0.5 + 0.01*unit_random_noise.
Forget how the data was generated (you never knew anyway), then study on the data set what correlations you want using only pairs of dimensions, see where this gets you.
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: 0) by Anonymous Coward on Thursday May 24 2018, @01:24PM (2 children)
Ok, here it is in R (There was an extra parentheses in your level_C, so I'm not sure if I did it right):
It looks like this:
In this case you have defined A and B to be independent. I am saying this never happens in nature, the correlations may get negligibly small and difficult to detect but they are always there when enough effort is put forth.
This was already tested and well known by the 1960s:
https://meehl.dl.umn.edu/sites/g/files/pua1696/f/074theorytestingparadox.pdf [umn.edu]
(Score: 2) by c0lo on Thursday May 24 2018, @01:52PM (1 child)
When I wrote [-1, 1] I intended the continuous interval between -1 and 1 including the limits. Should I have used the [-1..1] notation?
Not quite true. Taking values for any two of them, the third is determined.
If you want, you can express it as an implicit function under the form f(A, B, c)=0 (something like "homoeostasis happens only when the level of activation of these 3 genes follows this relation..." - without knowing those 3 genes are dependent somehow, using simple linear correlations between will tell you nothing).
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: 0) by Anonymous Coward on Thursday May 24 2018, @02:44PM
Since you were referring to "levels", saying [-1,1] looks like "the set of -1 and 1" to me. I don't think it really mattesr, either way the situation is set up so there will be no correlation.
That isn't what independent means. There is no feedback from C onto A or B in your model. I am saying that in any real situation there will be feedback. Then the values of A and B will be dependent upon C and each other. Then there will be a correlation.
Sure it will, it will tell you that the expression of those genes are correlated with each other. I am saying this is worthless info since we already know it to be true.
This is exactly what biologists like in TFA should be doing. That is not what they are doing. As I noted my the original comment, they are looking for the existence of simple correlations.
(Score: 1) by khallow on Saturday May 26 2018, @03:56AM
Sure, when they aren't correlated. Let us remember that the space of vectors with almost no correlation would be much larger than the space of vectors which almost perfectly correlate or anti-correlate, because the former would be a locus of a (N-1)-sphere on a N-sphere while the latter would the locus of two points (the 0-sphere for your edification).
(Score: 1) by khallow on Saturday May 26 2018, @04:29AM
Or anti-correlated. There is a big difference in behavior between reinforcing and inhibiting the expression of a gene. As I see it, you're ignoring the point of correlation. It's not a flag you set, but a matter of degree. Even in the linear, two gene case, there is a significant amount of information in correlations which are far from zero.
(Score: 2, Funny) by chromas on Wednesday May 23 2018, @09:24AM (2 children)
Science is no place for evidence!
(Score: 2, Informative) by Anonymous Coward on Wednesday May 23 2018, @11:38AM (1 child)
Science is a tool of Western Colonialism. [youtube.com]
-- Sent from my iPhone
(Score: 3, Informative) by c0lo on Wednesday May 23 2018, @03:01PM
Brief explanation [reddit.com]
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: 0) by Anonymous Coward on Wednesday May 23 2018, @10:19AM (6 children)
I prep with a strict diet of fried chicken and 20g of MiraLAX for 2 consecutive days and on the third day the magic happens. After another fried chicken breakfast, I grab a burrito for lunch and wash it down with copious Yuengling. When the moment arrives, I take the text to be reviewed into a bathroom stall and spread the papers over the floor. Loosening of the belt must be done gently so the review is not unleashed prematurely. Squatting and relaxing is the sure way my peer review will hit the mark and any text not suitably reviewed by this comprehensive method is used to wipe.
(Score: 0) by Anonymous Coward on Wednesday May 23 2018, @11:39AM (2 children)
And the result of that "magic" is your post? Maybe you should go back to the drawing board.
(Score: 1, Touché) by Anonymous Coward on Wednesday May 23 2018, @11:46AM
I have your comment selected for a partial review later today.
(Score: 0) by Anonymous Coward on Thursday May 24 2018, @04:01AM
I think GP's from the green site.
(Score: 2) by MichaelDavidCrawford on Wednesday May 23 2018, @11:50AM (2 children)
Just one word:
"Magnesium citrate solution"
Enough said
Yes I Have No Bananas. [gofundme.com]
(Score: 1, Funny) by Anonymous Coward on Wednesday May 23 2018, @12:09PM (1 child)
The word of a man well versed in the brown arts of peer review.
(Score: 2) by c0lo on Wednesday May 23 2018, @03:08PM
That's not a single man, he's at least 3.
Proof: on the promise of a single word, each of his personalities (that were the fastest to react) came with a different one
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: 1, Flamebait) by The Mighty Buzzard on Wednesday May 23 2018, @10:51AM (7 children)
Honestly, I wouldn't think it's worth the bother. Unless your shit's being reviewed by someone who's trying to prove you wrong, their review isn't worth a fuck anyway. There are a lot of disciplines nowadays that are so universally infected by groupthink that the peer review process is pretty much masturbatory for them anymore.
My rights don't end where your fear begins.
(Score: 0) by Anonymous Coward on Wednesday May 23 2018, @03:31PM (4 children)
Not necessarily. Scientists, in general, are skeptical people and are trained to find holes and weak points in scientific claims. Even if they "believe" the conclusion, scientists will usually suggest additional experiments to strengthen or clarify the claim.
Have you ever had any experience with peer review? Possibly this is a field-specific issue.
(Score: 4, Funny) by KritonK on Wednesday May 23 2018, @05:17PM
According to TFA, that's the big question. I don't know!
(Score: 2) by The Mighty Buzzard on Wednesday May 23 2018, @06:19PM (2 children)
FTFY. The original statement was not true of many fields anymore.
My rights don't end where your fear begins.
(Score: 1, Informative) by Anonymous Coward on Thursday May 24 2018, @03:14AM (1 child)
Put up or shut up. Whataboutism is a deadly boring argument fail.
(Score: 2) by The Mighty Buzzard on Thursday May 24 2018, @10:38AM
I think you need to look that term up. It's not even sort of applicable here.
My rights don't end where your fear begins.
(Score: 2) by insanumingenium on Wednesday May 23 2018, @03:44PM (1 child)
Wouldn't publishing that review process be the most obvious way for you to support calling them on that bullshit then? "Sunlight is said to be the best of disinfectants..."
(Score: 1) by nitehawk214 on Wednesday May 23 2018, @05:11PM
Well, it is a matter of intensity. Things placed in a low enough orbit of the sun will be disinfected.
"Don't you ever miss the days when you used to be nostalgic?" -Loiosh
(Score: 4, Interesting) by VLM on Wednesday May 23 2018, @12:35PM (1 child)
No one wants to talk about the actual issue, eh?
Been agitation since forever for paid peer review or peer review reform in general. Four options usually proposed:
1) The rich journal publishers could cough up $1K for each peer reviewer, raising the prices charged to institutions, essentially making them a middleman in paying the academics salaries. There are cost of living issues internationally, a variety of entertaining tax issues, etc. However having an employment contract with a prof would seem to motive the prof to put some work into the review and if caught lying (good luck) have to cough up the dough. Probably a bad idea.
2) Publish who reviewed and in "publish or perish" world, a review might be worth 1/10th a paper at tenure time or whatever. The problem is people will censor themselves to kiss up. Sure Stephan Hawkings paper is great who would want to be seen in public shitting on a popular science icon in a wheelchair? One solution is to increase the systemic workload such that 100 people review a paper and in a population of 100, at least one might tell the truth to authority sometimes. This would probably enforce extreme groupthink in the liberal arts. Probably a bad idea.
3) Publish anonymized reviews with no names attached. Hello social status signalling vs the 4chan anons battle, plus or minus intense groupthink, what a disaster this would be. What the world needs is more divide and conqueror and more internecine infighting and more holier than thou public and social media bragging. Or maybe not. Probably a bad idea.
4) Do nothing. This gets you pencil whipping reviews of spelling and grammar and often little verification. Especially in a field thats not specialized enough, your reviewers likely have no idea WTF you're talking about anyway. Probably a bad idea.
You'll note all four options are probably a bad idea. That kinda how it rolls.
There are far out ideas, like get rid of it and instead of publish or perish start rating academics on getting footnotes and references, if they produce crap no one will care. Or get rid of publish or perish and you won't get people shoveling out shitty papers and everyone's too busy to review closely because they need a replacement car or house downpayment or given academic salaries for all except the top, because they need the dollar menu at mcd or whatever. Most of the problem seems to boil down to problems with the far downstream effects of publish or perish, not actually a problem with peer review itself.
(Score: 2) by opinionated_science on Wednesday May 23 2018, @02:39PM
the central problem, is that publication is part of the feedback mechanism for "academic credit".
The publishing companies are getting the intellectual work of the community for free.
I suspect, that if peer-review was setup to spot "crafty incremental work", perhaps we'd get a better signal/noise ratio ;-)
As you can tell - I have a bias.....!
(Score: 2, Insightful) by Anonymous Coward on Wednesday May 23 2018, @01:42PM (2 children)
Here's a thorough explanation of how the peer review system we have today generally works in practice:
https://thelogicofscience.com/2016/08/29/who-reviews-scientific-papers-and-how-do-reviews-work/ [thelogicofscience.com]
Unless you’re someone who has actually published several peer-reviewed scientific papers themselves and has first-hand experience of the process the way the author of that blog post has, please read though it first before chiming in with opinions or suggestions on how the peer review process can be improved or complaining that it’s bunk.
(Score: 4, Interesting) by melikamp on Wednesday May 23 2018, @02:30PM (1 child)
(Score: 2) by Azuma Hazuki on Wednesday May 23 2018, @05:34PM
You could say he's giving it a real whipping...
I am "that girl" your mother warned you about...
(Score: 2) by Thexalon on Wednesday May 23 2018, @02:09PM
The lack of attention to peer review and replication is what allows junk science to stand when it really really shouldn't. You know, the sort where p-hacking is used to reach the conclusion whoever provided the grant wants you to reach, and at the very least gets the academic in question through their current publish-or-perish phase.
The only thing that stops a bad guy with a compiler is a good guy with a compiler.