Stories
Slash Boxes
Comments

SoylentNews is people

posted by hubie on Wednesday July 08, @12:53PM   Printer-friendly

Potentially fraudulent papers often cite each other and could be inflating the impact factor of journals in which they are published:

Cancer research articles with telltale signs of being produced by paper mills garner double the number of citations than do genuine papers in the field, finds an analysis of tens of thousands of articles.

In a study posted on the preprint server bioRxiv, the authors report that papers that were probably produced by paper mills frequently cite, or are cited by, other potentially fraudulent articles. Paper mills are businesses that produce and sell low-quality manuscripts — often containing fabricated data and results — designed to resemble genuine research.

Adrian Barnett, a statistician at the Queensland University of Technology in Brisbane, Australia, and his colleagues say that their analysis indicates that coordinated citation manipulation is inflating the impact metrics of journals in molecular oncology.

These metrics measure of how often a journal's papers are cited in other research, among other things. In many nations, having papers published in journals with high impact factors is taken into account when researchers apply for jobs and funding.

Research-integrity sleuths have long suspected that paper mills are inflating citations, says René Aquarius, a neurosurgery researcher at Radboud University Medical Center in Nijmegen, the Netherlands. "But it's nice to see this confirmed in such an elegant way," he adds.

Barnett says that there is an assumption that studies produced by paper mills are only being published in journals with low impact metrics, but his team's research suggest that this is not the case.

Barnett and his colleagues analysed 33,159 papers published between 2012 and 2023 in 20 high-impact molecular-oncology journals. The team used an artificial-intelligence tool called BERT that they had developed previously to identify suspicious papers. It looks for telltale characteristics that often appear in retracted paper-mill articles. Paper mills often mass-produce manuscripts that use fabricated data sets, manipulated images and unusual or 'tortured' phrases that are designed to evade plagiarism detectors. For each paper it reviewed, BERT assigned a probability score estimating how likely the study was to have been produced by a paper mill.

The tool flagged 4,085 papers — 12.3% of all of the papers examined — as having characteristics associated with paper-mill articles. The researchers identified potentially fraudulent papers in 19 out of the 20 journals that they examined. Nature Cancer was the only journal in the analysis that had not published any suspicious papers. ( Nature 's news team is editorially independent of its publisher, Springer Nature, which also publishes Nature Cancer. )

[...] Barnett says that the 4,085 flagged papers probably include some genuine studies. But given the large sample size, these will not change the general patterns seen in their findings. He notes that the tool is meant to point towards probable suspects and general patterns of paper-mill activity — rather than definitively determine whether a paper is genuine or not.

The team also examined the citation patterns of suspicious papers and the journals in which they were published. They found that the suspect articles had been cited by other papers up to twice as many times as papers that had not been flagged. The difference between the citation rate of suspicious and non-suspicious papers was highest in the first few years after publication, but dropped off over time.

The researchers also identified a pattern of suspicious papers citing other suspicious papers. The authors suggest that paper mills are citing their previous papers. "When a dishonest researcher buys a paper mill product, they not only get a published paper, but also get citations to that paper as part of the bargain," says Barnett.

In some journals, suspicious papers accounted for more than half of the citations that they received. For example, for Molecular Cancer and the Journal of Experimental & Clinical Cancer Research , 57% of citations came from flagged papers. (Both journals are published by the BMC portfolio, which is owned by Springer Nature.)


Original Submission

This discussion was created by hubie (1068) for logged-in users only. Log in and try again!
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
(1)
  • (Score: 5, Insightful) by PiMuNu on Wednesday July 08, @12:59PM (19 children)

    by PiMuNu (3823) on Wednesday July 08, @12:59PM (#1447632)

    This is an end point of a stupid HR policy, where we do statistical analysis of papers and citations to understand how good a scientist is.

    The idea is a noble one - trying to make sure that universities employ the best scientists based on objective measures. But any such analysis can be gamed and actually rewards nonsense - the opposite of the intention.

    In the old days, subjective measures received greater weight - e.g. reputation.

    • (Score: 2) by JoeMerchant on Wednesday July 08, @01:27PM (2 children)

      by JoeMerchant (3937) on Wednesday July 08, @01:27PM (#1447635) Journal

      Yet another use for LLMs - automated "paper quality analysis" - the whole publish or perish culture is toxic and counter-good-science, but at least we can develop independent measures of reliability within this sea of toxic "science" dumping.

      --
      🌻🌻🌻🌻✌️ [google.com]
      • (Score: 3, Touché) by vux984 on Wednesday July 08, @04:26PM (1 child)

        by vux984 (5045) on Wednesday July 08, @04:26PM (#1447662)

        "Yet another use for LLMs - automated "paper quality analysis"

        Creating a golden age of 8 seconds, before everyone turns to ...

        Ooooh - Yet another use for LLMs - automated "paper quality analysis" result stuffing (looks-maxxing!) for the scoring LLMs. Actual quality doesn't matter, of course, just make it look good to the LLM evaluators.

    • (Score: 5, Insightful) by Thexalon on Wednesday July 08, @02:31PM (6 children)

      by Thexalon (636) on Wednesday July 08, @02:31PM (#1447643)

      The problem is:
      1. Managers who do not know how to do the job of their subordinates try to come up with metrics to "objectively" measure their performance.
      2. Smart subordinates (e.g. scientists) learn what metrics are being used, and learn how to game them to make themselves look good.
      3. The managers figure out that the metrics are being gamed and try to find a new metric. Goto step 1.

      A few cycles of this, and all you have is management coming up with BS metrics while the worker bees try to satisfy those metrics knowing full well it has no bearing on what they're supposed to be doing.

      The real solution to the problem, of course, is to fire the managers in question and replace them with people who know how to do the job they're managing as well as gain the trust of those people and manage them effectively. But since the managers are making the decision about who to fire, they will never pick themselves no matter how incompetent they are at their job.

      The whole situation is made even worse by the fact that the cult of the MBA has taken over pretty much all organizations, including academic ones, and a key principle of the cult of the MBA is the idea that you can effectively manage people who do work you do not even come close to understanding. More effective organizations try to fight this, e.g. in their heyday you could not become a manager at Ford without demonstrating that you, with your own two hands, could build a car.

      --
      "Think of how stupid the average person is. Then realize half of 'em are stupider than that." - George Carlin
      • (Score: 2) by PiMuNu on Wednesday July 08, @03:02PM

        by PiMuNu (3823) on Wednesday July 08, @03:02PM (#1447648)

        I think the metrics arise out of an attempt to be "fair". It's not just HR, it's also funding agencies that play these games (arguably the university hiring is just following the steer of the funding agencies).

      • (Score: 0) by Anonymous Coward on Wednesday July 08, @06:18PM

        by Anonymous Coward on Wednesday July 08, @06:18PM (#1447669)

        There's a 4.

        Those who are interested in the subject matter and judge their progress relative to history. They do not game the metrics because that doesn't yield progress. They get fired.

      • (Score: 2) by JoeMerchant on Thursday July 09, @04:11AM

        by JoeMerchant (3937) on Thursday July 09, @04:11AM (#1447711) Journal

        > it has no bearing on what they're supposed to be doing.

        Publish or perish says otherwise. They are exactly supposed to be enhancing the reputation of their employing institution specifically by beefing up those metrics.

        Nobody knows if you're kind or a good teacher or an ethical scientist anymore, what they look at when it comes to handing out the choice assignments for next semester is: who made me look good by putting up the big numbers?

        And grant income... my Uni Department was all about keeping the big grant $$$ rolling in.

        --
        🌻🌻🌻🌻✌️ [google.com]
      • (Score: 2) by jb on Thursday July 09, @08:25AM (2 children)

        by jb (338) on Thursday July 09, @08:25AM (#1447725)

        The problem is:
        1. Managers who do not know how to do the job of their subordinates try to come up with metrics to "objectively" measure their performance.
        ...
        The real solution to the problem, of course, is to fire the managers in question and replace them with people who know how to do the job they're managing as well as gain the trust of those people and manage them effectively.

        Couldn't agree more. This is by far the biggest failing of most universities today ... and it applies to the teaching side just as much as to the research side.

        Two things are required to fix it:

        First and foremost, get rid of the "super-faculties" (where a huge number of disciplines are lumped in together under one management structure) and go back to the traditional model of having a separate faculty for each reasonably narrow discipline (and semi-autonomous departments within the faculties for each somewhat-narrower discipline within those). Without this first step, you will always end up with Deans who don't understand the work in most of the disciplines within their faculties.

        Then and only then, do pretty much as you suggested: get rid of all the "professional managers"; and return to the model of having the tenured professors within each department elect one of their own as head of department (and the heads of department within each faculty then elect one of their own as the dean of their faculty), which worked well for centuries. Under that model if need be a Vice-Chancellor (or perhaps the university Council as a whole) can still fire a Dean, e.g. for misconduct, but when he does so the V-C has absolutely no say in who the replacement will be (that's left entirely up to the remaining heads of department within the ex-Dean's faculty). Likewise a Dean can still fire a head of department, but then the Dean has no say in who the professors choose as the replacement.

        • (Score: 2) by Thexalon on Thursday July 09, @11:44AM (1 child)

          by Thexalon (636) on Thursday July 09, @11:44AM (#1447733)

          Luckily, my alma mater does mostly run the way it's supposed to - the faculty run the show, there's still a lot of tenured folks around, and while the president is chosen by the trustees the departments still pretty much run themselves (and at least in the departments I worked with, most of the faculty tried to avoid being chair, because they didn't want to do paperwork, they wanted to teach and do research).

          But about 30 years ago, some businesspeople decided that what academia really needed was to be run like a business, and used their money and political influence to start making that happen. We now have lots of universities being run by people who have never taught a class, and whose goals are (1) to make money for the institution and more importantly themselves, and (2) punish faculty who say or write anything the new management doesn't like regardless of how true it is. As far as they're concerned, maximize tuition and maximize the customer base, while minimizing spending on employees or providing a product. And that's how you have classes that cost $5000 per student to take that pay their adjunct instructor $4000 to teach it, and the instructor is sometimes not even allowed to develop their own lesson plans.

          --
          "Think of how stupid the average person is. Then realize half of 'em are stupider than that." - George Carlin
          • (Score: 2) by jb on Saturday July 11, @08:45AM

            by jb (338) on Saturday July 11, @08:45AM (#1447898)

            Again, I'd agree with almost everything you said.

            Only exception is that I don't have any problem with the "maximise tuition fees" imperative, so long as: (a) it's done on the basis that we're competing on genuine quality of education (therefore we can afford to charge a premium); and (b) we still offer at least some academic-merit-based scholarships (so we're not just a "school for the rich"). Sadly, very few institutions are prepared to satisfy criterion [a] these days, for exactly the reason (corporatisation) you pointed out.

            On the flip side, yes underpayment (and underemployment) of teaching staff is a chronic problem at most universities ... which might also help to explain why there are somewhat fewer top notch lecturers at most universities than there were a generation or two ago (we expect to get paid somewhat less in academia than in industry and that's fine ... but not that much less ... and we expect far better job security as part of the deal).

            And yes, academic freedom is crucial too: it's okay to start out inheriting a predecessor's material, but if you're not allowed to take ownership of the course properly within your first semester teaching it, then it's probably not worth taking the job.

    • (Score: 0) by Anonymous Coward on Wednesday July 08, @02:36PM (5 children)

      by Anonymous Coward on Wednesday July 08, @02:36PM (#1447644)

      Where does peer review fit into this? Is peer review being skipped by some journals, or ignored completely with the on-line "pre-print" options now available?

      I peer review for an engineering journal and I see a lot of junk and even some well-written papers that don't contribute anything new. I make sure to pass that along to the editor(s) in my reviews. With a lot of experience in my niche, I like to think that I'm fairly good at rejecting useless papers...and every now and then all the reviewers agree.

      Maybe things work differently in science?

      • (Score: 5, Informative) by PiMuNu on Wednesday July 08, @03:07PM (2 children)

        by PiMuNu (3823) on Wednesday July 08, @03:07PM (#1447650)

        A "paper mill" is a journal that intentionally publishes junk papers to reward scientists who game the system. The process is something like:

        1. Get my buddy to set up a journal
        2. Groups all submit papers to said journal citing each others work. The content doesn't really matter, just the citations and number of publications.
        3. Metrics assess number of citations. Attempts to fight the gaming like "no self-citations" are easily worked around.

        How can anyone tell the difference between a handful of scientists in an exciting active area who are all developing something really cool, and a handful scientists who are making junk papers to improve their score for funding and HR stuff? Well, you have to know the subject matter and then make a *subjective* assessment based on your competence. Which irks funders and HR folks for various reasons.

        • (Score: 0) by Anonymous Coward on Wednesday July 08, @03:29PM (1 child)

          by Anonymous Coward on Wednesday July 08, @03:29PM (#1447655)

          Might have learned something today(grin).

          My understanding of scientific paper mills was that they sold pre-written research papers (an adult version of buying an essay to turn in for grade school homework). Did not realize that it was a whole ecosystem that also includes publishing the fraudulent papers.

          • (Score: 2) by PiMuNu on Friday July 10, @07:10AM

            by PiMuNu (3823) on Friday July 10, @07:10AM (#1447827)

            I get emails every day (filtered to junk) from some junk journal hosted by mdpi.com.

            At the end of the day a lot of journals are just a website (many reputable journals don't do much of a paper copy).

      • (Score: 2) by Thexalon on Thursday July 09, @12:06PM (1 child)

        by Thexalon (636) on Thursday July 09, @12:06PM (#1447734)

        You're thinking about this all like a nerd. As in, your goal is to make the journal full of quality papers that advance human knowledge in your particular branch of engineering.

        That's not the goal of the people in charge of these things these days. Their goal is to make their institution look productive by maximizing the number of papers produced and their alleged impact on the field, without increasing the costs of those pesky professors and grad students always going about integrity and principles and the scientific process. For them, peer review is nothing more than annoying checkbox you have to click through to get the paper published that slows them down. In their ideal world, an AI bot writes a paper, a bunch of doctoral students pay for the privilege of putting their name and the name of their institution on it, another AI bot says that the first AI bot's paper is acceptable, the doctoral students pay for it to be published, and the paper is now published with the institution's name all over it. And then to make the work "high-impact", the next rounds of this make sure to program the AI bot that writes the paper to cite the work of the previous team. See, all the metrics look great!

        --
        "Think of how stupid the average person is. Then realize half of 'em are stupider than that." - George Carlin
        • (Score: 0) by Anonymous Coward on Friday July 10, @01:41AM

          by Anonymous Coward on Friday July 10, @01:41AM (#1447808)

          > You're thinking about this all like a nerd.

          Yup, I am thinking like a nerd (I'm the AC GP)--my engineering niche is my career and it's been good to me in many respects.

          If my "reject" reviews throw a few monkey wrenches into the machinery you describe, then I'll take the satisfaction of a job well done.

    • (Score: 2, Disagree) by VLM on Wednesday July 08, @02:54PM (2 children)

      by VLM (445) on Wednesday July 08, @02:54PM (#1447646)

      This is an end point of a stupid HR policy

      Not necessarily stupid but incredibly unaware about being backward facing.

      In the past, great scientists incidentally as a side effect published generally more heavily cited papers. Cool. Huh I wonder if we could hire great scientists using a numerical metric... and the road to hell is paved with those good intentions.

      The problem is its easily game-able so looking forward, scientists focused on publishing as many fake papers with fake citations as possible will get promoted further and faster, despite being really awful scientists.

      Its similar to how in the old days, hyper rich people sent their kids to college, so NPCs get it backwards and think if they force their kid to get an expensive college degree in basketweaving or early childhood education, their kid will inevitably as a result get hyper rich because thats how it worked in 1920. Nah, they just end up poorer in life long debt and dead career, worse off than even uneducated people. Thats the future of overpriced college education... not the past, no, but its the future.

      They just can't understand changing conditions, or the results of bad motivators being bad results.

      A simple way to cheat a complicated system usually results in the cheater being cheated themselves LOL in the long run.

      For another current example see "tokenmaxxxxxing" in AI LLM use.

      • (Score: 2) by Bentonite on Thursday July 09, @06:20AM (1 child)

        by Bentonite (56146) on Thursday July 09, @06:20AM (#1447717)

        It doesn't appear that a "scientist" that is gaming citations to the point of publishing fraudulent papers has any interest in science - rather it seems those are anti-scientists (as a paper that contains wrong information, is actively harmful, unlike no paper).

        AI LLM

        New oxymoron coined?

        • (Score: 2) by VLM on Thursday July 09, @04:29PM

          by VLM (445) on Thursday July 09, @04:29PM (#1447761)

          New oxymoron coined?

          Appropriately, for "artificial intelligence" LOL

  • (Score: 3, Interesting) by khallow on Wednesday July 08, @01:22PM (2 children)

    by khallow (3766) Subscriber Badge on Wednesday July 08, @01:22PM (#1447634) Journal

    The team also examined the citation patterns of suspicious papers and the journals in which they were published. They found that the suspect articles had been cited by other papers up to twice as many times as papers that had not been flagged. The difference between the citation rate of suspicious and non-suspicious papers was highest in the first few years after publication, but dropped off over time.

    The researchers also identified a pattern of suspicious papers citing other suspicious papers. The authors suggest that paper mills are citing their previous papers. "When a dishonest researcher buys a paper mill product, they not only get a published paper, but also get citations to that paper as part of the bargain," says Barnett.

    Sounds like that's what they are optimizing for. My guess is that we have substantial funding, hiring, or promotion decisions worldwide being made on the basis of this metric. That's what needs to be fixed.

    • (Score: 4, Insightful) by JoeMerchant on Wednesday July 08, @01:32PM (1 child)

      by JoeMerchant (3937) on Wednesday July 08, @01:32PM (#1447636) Journal

      I think this could be an artifact of sustainted "directed outcome" publication. When you're given an outcome to target, of course you cite papers which also target the same outcome.

      Real science is more random.

      --
      🌻🌻🌻🌻✌️ [google.com]
      • (Score: 1) by khallow on Wednesday July 08, @02:08PM

        by khallow (3766) Subscriber Badge on Wednesday July 08, @02:08PM (#1447640) Journal
        A dangerous warning sign is that papermilling doubles the weight of this metric. Whoever uses this in academic or funding decisions is much more likely to reward dishonesty.
  • (Score: 2) by Username on Wednesday July 08, @01:47PM

    by Username (4557) on Wednesday July 08, @01:47PM (#1447638)

    The condescending tone makes me think they might be trying to smear competition. Might just be a narcissist telling others they can't be as special as them. "They aren't a real journal." I bet thier research is on the same level.

  • (Score: 2) by VLM on Wednesday July 08, @02:30PM (4 children)

    by VLM (445) on Wednesday July 08, @02:30PM (#1447642)

    Two topics not mentioned in the article or I missed them:

    1) Do a country analysis on the data. This is endemic in China and India, less so in say Belgium. "Its part of their culture" at this point. Keep this in mind when hearing about various countries taking over in academia. There are countries that are now in the lead for slop production which results in being in the lead for hiring. But corruption makes academia no longer worth funding if its results aren't useful to the general public. You can already see the apathy when corrupt areas of academia are defunded "by orange man bad" but fundamentally if we don't have the money, our money is funding corruption in foreign countries, and there's no useful results other than encouraging more corruption in the field, going "cold turkey" is a viable plan.

    2) At this point there's little difference between Karma farms on Reddit and academia. I think academia people need to think VERY carefully about what this implies about the long term viability of their field. Mental illness always rises to the top, or so it seems. Successful orgs no matter if industry, social, academia, tend to get one brand of competent psychopaths. On the other hand the type of mentally ill that rise to the top of painfully unsuccessful orgs like legacy social media or soft science academia or overseas hard science academia tend to be absolute lunatics, like not even competent psychopaths. If you're in an org mostly ruled by corrupt karma farmers... you probably should get out and get away as fast as humanly possible. And of course rats deserting the sinking ship is a feedback loop, nothing is crazier than the most politically pure subreddits and corners of academia but the people left behind who couldn't be pushed out by the extremists are ... not really functional when interacting in the real world anymore.

    Look at nutrition in the USA as an example of a system kind of in between successful trillion dollar psychopaths who would poison their grannie with bad food if it makes the share price go up a penny, and on the other end its mentally ill people with strange fetishes giving advice.

    • (Score: 3, Insightful) by PiMuNu on Wednesday July 08, @03:14PM (2 children)

      by PiMuNu (3823) on Wednesday July 08, @03:14PM (#1447651)

      > defunded "by orange man bad"

      I agree with the sentiment, but there is devil in detail. Such a defunding needs to be supported by evidence, and "orange man bad" looks very much like he is defunding based on political needs, which is pretty much opposite to your claim.

      > little difference between Karma farms on Reddit and academia

      Yeah, this is a good point. Note that loads of good work is going on - we are moving forwards in terms of tech and a lot of that movement is coming from academia. There is quite some chaff in with the wheat though.

      • (Score: 2) by VLM on Wednesday July 08, @03:26PM (1 child)

        by VLM (445) on Wednesday July 08, @03:26PM (#1447654)

        Some of the hires were political so some of the fires being political seems "OK"? Its odd how its not causing any problems, kind of like when 90% of twitter's dead weight got fired LOL. Lots of people hiding in sinecures.

        There is quite some chaff in with the wheat though.

        The problem is we're actively defunding the wheat to pay for more chaff because the system is profoundly broken, some areas worse than others. It seems the easiest path points to an inevitable all-chaff product line LOL and the last of the wheat will be removed from production soon enough.

        At that point the general public will not mind pulling the plug entirely, which will REALLY set back scientific progress.

        • (Score: 2) by PiMuNu on Thursday July 09, @08:52AM

          by PiMuNu (3823) on Thursday July 09, @08:52AM (#1447727)

          > Its odd how its not causing any problems

          So far a lot (all?) of the funding cuts have been turned back by congress. But if the Trump cuts went through, the impact would be a degradation of US technology lead over decades. You don't see it until it's too late.

    • (Score: 2) by JoeMerchant on Thursday July 09, @04:14AM

      by JoeMerchant (3937) on Thursday July 09, @04:14AM (#1447712) Journal

      >corruption makes academia no longer worth funding if its results aren't useful to the general public.

      Funding by who?

      Corruption is exactly what industry funding of academia is all about. Last news I heard, the general public (federal government) funding routes in the US had taken a significant downturn.

      --
      🌻🌻🌻🌻✌️ [google.com]
  • (Score: 2) by VLM on Wednesday July 08, @03:19PM (4 children)

    by VLM (445) on Wednesday July 08, @03:19PM (#1447653)

    In many nations, having papers published in journals with high impact factors is taken into account when researchers apply for jobs and funding.

    I'm not sure what a realistic alternative is.

    Possibly, if you block off the technical path for management, management can't figure out who to hire without a spreadsheet telling them. If management (both private industry and academia) were made up of PHDs instead of MBAs, then as experts in their field they MIGHT actually know who to hire rather than rely on spreadsheets. All this requires is the MBAs deciding to fire themselves and be replaced by "the nerds" so I'm sure that'll happen real soon now LOL. In the very long run, orgs run my MBAs should have MUCH lower rate of returns than orgs run by technically skilled people, but it might be a very long run and given enough corruption (which I'm sure we have) it may never naturally compete and reset itself. In that case you have to wait for a civilization to be conquered and replaced to make progress again. So that might be our future. It certainly is the case in Europe. How does the 3rd world handle university appointments because thats replacing the Euros and the Euro system pretty soon.

    One option is lower pay. People will do anything for $500K/yr with lifetime tenure. Lower that to like $50K/yr absolute max no tenure and a lot of the toxic competitiveness will go elsewhere. I don't think anyone in the biz likes this. But if education sinecures kill organizations and civilizations, their feelings may not matter. And there's always industry/government if academia isn't a lottery winning sized paycheck. And if salaries were cheap and publish or perish disappeared the culture in academia might be more friendly making it a net win. So one individual won't become a millionaire if they're a college prof but maybe thats OK if everyone loves their job? Note that millionaire college profs could pay a very nice salary to grad students and post docs. I think maybe science would make more progress with 20 postdocs who can eat more than peanut butter sandwiches than 1 millionaire prof doing management (probably of the 20 postdocs actually making progress LOL). Essentially the publish or perish fake paper crisis is a cause of the income pyramid being too pointy in academia with enormous seas of poverty and a couple individuals making millions. Academia makes capitalism look practically socialistic or communistic in comparison sometimes. Its pretty bad. Slaves serving millionaires is an academia thing not capitalist industry, generally. The people who always psychologically project always sell it the other way around "of course"

    Random lotto is a hilarious idea. If the system we have is negatively dysfunctional then rolling dice would have better results. Every year, or couple years, apply for a grant, roll dice, if you don't get it go back to "work" or consulting/contracting or just manual labor. This might select for better candidates than the existing system. Which means the org would be better off firing the MBAs and HR ladies, so I'm sure they'll love being replaced by rolling dice.

    Aside from the above, I got nothing? Any other ideas how to hire scientists?

    • (Score: 0) by Anonymous Coward on Wednesday July 08, @04:20PM

      by Anonymous Coward on Wednesday July 08, @04:20PM (#1447659)

      > Any other ideas how to hire scientists?

      Sure, interview them, that's what I do (but my company is really tiny--I'm sure it gets tougher for larger orgs. Assuming you already have competent staff, an oral "exam" followed by a private debrief/discussion seems like a fairly reliable way to sort out the frauds. Just the notice that an oral interview is part of the process might scare away quite a few of the frauds?

      Oh, you say that's too expensive and time consuming? Well, then you get what you pay for, and imo, paying HR to do this is a real waste of money.

    • (Score: 2) by JoeMerchant on Thursday July 09, @04:26AM

      by JoeMerchant (3937) on Thursday July 09, @04:26AM (#1447713) Journal

      >I'm not sure what a realistic alternative is.

      Formalization of measures of reputation.

      People shouldn't be expected to "pass reputational judgement" on more than the 50-100 professionals and organizations they know best in the field, anything beyond that is going to be largely hear-say. But, if you are highly regarded by your local peers, and you and your local peers are highly regarded by remote peers, and the institutions they are associated with are also highly regarded, that all boosts your personal objective reputation score.

      If a corporation, say Volkswagen, develops a poor reputation regarding a certain field, say diesel emissions, then researchers associated with support of Volkswagon's hijinx would take a reputational hit, as would their associated institutions, local colleagues, etc. anybody who was being boosted by them before the VW association would be boosted a bit less after the VW scandal came out.

      This is all done rather informally and fuzzily today. A minor lingering reputational manifestation I see at the gas stations is: BP (the one who mishandled the drilling platform and coated most of the beaches fronting the "Gulf of America" with crude oil) charges less for gas than the other stations, not a lot less, but enough to notice - and that's attributed to their reputational hit from the still in recent memory oil spill(s). Apparently it's not enough to get them to close / rebrand the stations, in fact it might be a kind of competitive advantage - getting more volume of customers due to their lower prices... but, reputation is what the local reporters say about it.

      Formalizing the process, with some carefully considered measures of both transparency and anonymity, and publication of the resulting relative ratings would, I feel, encourage better behavior than the current publish or perish culture.

      --
      🌻🌻🌻🌻✌️ [google.com]
    • (Score: 2) by JoeMerchant on Thursday July 09, @04:42AM (1 child)

      by JoeMerchant (3937) on Thursday July 09, @04:42AM (#1447714) Journal

      > were made up of PHDs instead of MBAs

      Personally, I don't know which are worse. PhD means: puts up with the Academic bullshit long enough, and kisses the right rings with proper obesiance to get the title. Having worked with a mixture of doctor titled and non doctor titled engineers, the doctor title isn't a guarantee of a problem child - but it raises the odds. I'd say: in the top quartile of people who accomplish great things professionally, there's a higher concentration of PhDs, yes, but in the bottom quartile where Dilbert's Wally levels of productivity live, there are even more PhDs down there.

      Then, once upon a time, a PhD hired me into a startup. He had already hired 2 other PhDs and a Master's candidate, I hired a BS and PhD to go with them. All of those PhDs were odd birds, one was an extreme loner - working intently on his own thing, not integrating with the rest of us at all, but... his thing was good, except that it didn't always support the Chief PhD's narrative so that led to friction there... We had another who sat in the corner all year, researching high performance computing toys that we ultimately proved were un-necessary, but in an indirect way he helped to do that by prattling on endlessly about cache memory and the importance of cache coherency. During a code performance review, we discovered a way to improve cache coherency of the algorithm resulting in 100x speedup - meaning his high performance computing hardware was no longer required. This also angered and disappointed the Chief PhD because it disrupted his vision of how the product would be promoted - his visions of massively parallel processing towers of computing power became un-necessary - indirectly thanks to our PhD rendered lessons on cache coherency - although we might have stumbled into the solution without his guidance anyway, it was undeniable that his focus on cache coherency led us to have that eureka moment in the code review of: "if we structure the loop like this, we can accomplish the inner loop without any cache faults!" and we knew it should be a dramatic improvement before even testing it.

      As for MBAs - I often say I had a 12 year MBA experience working for a small company, filling most of the roles personally at one time or another during those 12 years. It was valuable experience, but there's no way in hell an actual MBA degree on their compressed 2 year timeframe in classroom settings can impart anywhere near the experiential knowledge that I acquired in 12 years of real world employment. Then you get into the "what type of person even pursues an MBA in the first place?" filter factor and... I'd steer MBAs away from the candidate pool at any company I had hiring control of and wanted to succeed competitively in the market.

      --
      🌻🌻🌻🌻✌️ [google.com]
      • (Score: 0) by Anonymous Coward on Friday July 10, @01:27PM

        by Anonymous Coward on Friday July 10, @01:27PM (#1447846)

        > PhD means: puts up with the Academic bullshit long enough, and kisses the right rings with proper obesiance to get the title.

        While I know what you mean and have seen this, I've also seen a PhD who went back for his degree after working in industry for several years. He was able to find his own grant money (we helped with letters of recommendation) and had a full boat ride with tuition and a stipend (a modest living wage). He wasn't reliant on the university profs to arrange research funding or to work on their funded projects as a serf. He was able to pick his own topic and avoid most of the academia ass kissing and bs that is common.

        Another way to do this is to keep a job, while taking a light course load for a longer number of years. Eventually you finally have to take some time off work, to write the dissertation. I've known several engineers that took this route to a PhD.

  • (Score: 0) by Anonymous Coward on Thursday July 09, @03:27PM (1 child)

    by Anonymous Coward on Thursday July 09, @03:27PM (#1447757)

    BERT is the first of a family of encoder only LLMs released by Google (IIRC). These researchers didn't create the model, they fine tuned it on a specific task. I may have used BERT for similar things in my own work....

    • (Score: 2) by jb on Saturday July 11, @08:47AM

      by jb (338) on Saturday July 11, @08:47AM (#1447899)

      BERT is the first of a family of encoder only LLMs released by Google

      Presumably named after the "Sesame Street" character, only with even less intelligence.

(1)