Hugging Face's chief science officer worries AI is becoming 'yes-men on servers':
AI company founders have a reputation for making bold claims about the technology's potential to reshape fields, particularly the sciences. But Thomas Wolf, Hugging Face's co-founder and chief science officer, has a more measured take.
In an essay published to X on Thursday, Wolf said that he feared AI becoming "yes-men on servers" absent a breakthrough in AI research. He elaborated that current AI development paradigms won't yield AI capable of outside-the-box, creative problem-solving — the kind of problem-solving that wins Nobel Prizes.
"The main mistake people usually make is thinking [people like] Newton or Einstein were just scaled-up good students, that a genius comes to life when you linearly extrapolate a top-10% student," Wolf wrote. "To create an Einstein in a data center, we don't just need a system that knows all the answers, but rather one that can ask questions nobody else has thought of or dared to ask."
Wolf's assertions stand in contrast to those from OpenAI CEO Sam Altman, who in an essay earlier this year said that "superintelligent" AI could "massively accelerate scientific discovery." Similarly, Anthropic CEO Dario Amodei has predicted AI could help formulate cures for most types of cancer.
Wolf's problem with AI today — and where he thinks the technology is heading — is that it doesn't generate any new knowledge by connecting previously unrelated facts. Even with most of the internet at its disposal, AI as we currently understand it mostly fills in the gaps between what humans already know, Wolf said.
Some AI experts, including ex-Google engineer François Chollet, have expressed similar views, arguing that while AI might be capable of memorizing reasoning patterns, it's unlikely it can generate "new reasoning" based on novel situations.
Wolf thinks that AI labs are building what are essentially "very obedient students" — not scientific revolutionaries in any sense of the phrase. AI today isn't incentivized to question and propose ideas that potentially go against its training data, he said, limiting it to answering known questions.
"To create an Einstein in a data center, we don't just need a system that knows all the answers, but rather one that can ask questions nobody else has thought of or dared to ask," Wolf said. "One that writes 'What if everyone is wrong about this?' when all textbooks, experts, and common knowledge suggest otherwise."
Wolf thinks that the "evaluation crisis" in AI is partly to blame for this disenchanting state of affairs. He points to benchmarks commonly used to measure AI system improvements, most of which consist of questions that have clear, obvious, and "closed-ended" answers.
As a solution, Wolf proposes that the AI industry "move to a measure of knowledge and reasoning" that's able to elucidate whether AI can take "bold counterfactual approaches," make general proposals based on "tiny hints," and ask "non-obvious questions" that lead to "new research paths."
The trick will be figuring out what this measure looks like, Wolf admits. But he thinks that it could be well worth the effort.
"[T]he most crucial aspect of science [is] the skill to ask the right questions and to challenge even what one has learned," Wolf said. "We don't need an A+ [AI] student who can answer every question with general knowledge. We need a B student who sees and questions what everyone else missed."
(Score: 4, Insightful) by chucky on Saturday March 08, @03:22PM (6 children)
I'm fine if AI becomes the fastest librarian in the world. And if a human librarian helps me to formulate the question better, so that the AI doesn't hallucinate...
(Score: 4, Insightful) by Unixnut on Saturday March 08, @05:32PM (2 children)
I was going to say something similar, quite frankly if AI provides me with "a very obedient student" then that is great. I am perfectly capable of reasoning and asking new questions myself. No doubt actual geniuses will be able to come up with even better questions than I can, and they can make use of AI to do the grunt work to come to workable theories much faster than present day.
I see a similar thing at work where they brought in their own chatGPT-like system. I can ask it to write code for me, and the code it spits out is equivalent to a newbie copy/pasting snippets from stackoverflow until they cobble together a working program. Sometimes (e.g. if I need a one off script to do a task) that is all I need from the code, and apart for cursory logic checks to make sure it does what I expect it to we are good to go. While I could whip up a similar program in round half an hour the AI can do it in under 15 seconds (+5 mins logic check).
For more serious programs I tend to have to re-write large chunks of the AI output, but the fact is the bulk of the gruntwork is already there, as are the basics (things like class/program execution structure) as well as hints at logic flow. I can re-write the bits that need it while leaving a lot of the boilerplate code as is. This results in a serious increase in my productivity.
I can imagine the same would happen for researchers, or even the lone genius who doesn't have the funding for their own research team. Suddenly everyone can have access to a virtual "Research assistant" to gather, collate (and even summarise) data for you. That in itself will improve the productivity of inventors, researchers and your everyday genius.
You don't need AI to draw conclusions for you, let alone philosophise upon questions not yet existing. I question if we should even try to make it possible. I really don't need a machine to think for me, I am perfectly capable of thinking for myself.
(Score: 4, Insightful) by c0lo on Sunday March 09, @04:09AM (1 child)
That's far from enough for a quality research assistant - a good one will know how to fudge the data in such a way that the published article receives max citations.
For anything else, the post-grad students are much cheaper.
https://www.youtube.com/@ProfSteveKeen https://soylentnews.org/~MichaelDavidCrawford
(Score: 0) by Anonymous Coward on Monday March 10, @03:17PM
you think you're joking but US universities have a huge influx of Chinese students who are obedient A-students and pre-trained in fudging the results to please Leadership. this does not good science make.
(Score: 2) by corey on Saturday March 08, @08:49PM (2 children)
I agree. The current use of AI seems to be an energy inefficient version of a search engine. But that aside, it’s good as an indexer.
What Wolf says is correct in my opinion, but he’s talking purely from a scientific research perspective. The system he proposes, if it were possible, could then also figure out new ways of stealthily killing off all humans or ways of making itself “smarter”. Maybe anyway.
(Score: 2, Insightful) by Anonymous Coward on Sunday March 09, @12:08AM (1 child)
Most are HORRIBLE as indexers if they can't tell me where they came up with what they parroted.
(Score: 2) by corey on Sunday March 09, @10:41PM
Yeah agree with that. But on Brave search (search.brave.com), it states where it got the AI widget info from, from memory, or at least where i can find more info on the topic.
(Score: 4, Interesting) by Thexalon on Saturday March 08, @05:58PM (4 children)
The fundamental mistake of a lot of would-be entrepreneurs and business executives is thinking that coming up with new ideas is hard. It isn't: Put 10 ordinary people in a room with a whiteboard, a problem to solve or an area to improve, and a rule of "there are no bad ideas" for approximately an hour, and you can easily generate at least a dozen decently good ideas.
The hard parts have always been:
1. Sifting out the fantastic ideas from the decently good ideas. Nobody has a reliable way of doing this, it's always going to be a bit of guesswork. Where software could help with this is providing a good way to at least rule out all the ones that have been tried and failed before.
2. Turning those ideas into reality. This is the really time-consuming part. Where software could help with this is in automating more of what had previously been manual.
"Think of how stupid the average person is. Then realize half of 'em are stupider than that." - George Carlin
(Score: 0) by Anonymous Coward on Sunday March 09, @01:55AM (1 child)
"automating more of what had previously been manual."
And why, exactly, does this require large language models or general artificial intellegence?
If nothing produced by them can be trusted, where is the efficency gain?
I don't think you're going to have a whole lot of luck figuring out something as complicated
or poorly documented as "solutions to problems that have been tried and failed" if you aren't
lucky enough to come across something written down about it somewhere in the past that
was somehow ingested and put into a form that could be retrieved
(Score: 3, Touché) by Thexalon on Sunday March 09, @04:11AM
You'll notice I wrote "software", not "LLMs" or "AI" or "generative ML" or anything like that.
"Think of how stupid the average person is. Then realize half of 'em are stupider than that." - George Carlin
(Score: 2) by c0lo on Sunday March 09, @04:13AM
Keep your buggy software away from my... ummm... hands, I don't want a Software Transmitted Disease.
https://www.youtube.com/@ProfSteveKeen https://soylentnews.org/~MichaelDavidCrawford
(Score: 0) by Anonymous Coward on Monday March 10, @03:20PM
You know what good and coming up with solution to problems? Giving the people facing those problems a say in what the solution needs to be. All this fucking top-down visionary leadership bullshit is just rich-people masturbation. Flying to fucking Mars to save humanity... gimme a fucking break.
(Score: 1, Interesting) by Anonymous Coward on Sunday March 09, @12:12AM
https://en.wikipedia.org/wiki/Gestalt_psychology [wikipedia.org]