SoylentNews is people
SoylentNews
Hugging Face's chief science officer worries AI is becoming 'yes-men on servers':
AI company founders have a reputation for making bold claims about the technology's potential to reshape fields, particularly the sciences. But Thomas Wolf, Hugging Face's co-founder and chief science officer, has a more measured take.
In an essay published to X on Thursday, Wolf said that he feared AI becoming "yes-men on servers" absent a breakthrough in AI research. He elaborated that current AI development paradigms won't yield AI capable of outside-the-box, creative problem-solving — the kind of problem-solving that wins Nobel Prizes.
"The main mistake people usually make is thinking [people like] Newton or Einstein were just scaled-up good students, that a genius comes to life when you linearly extrapolate a top-10% student," Wolf wrote. "To create an Einstein in a data center, we don't just need a system that knows all the answers, but rather one that can ask questions nobody else has thought of or dared to ask."
Wolf's assertions stand in contrast to those from OpenAI CEO Sam Altman, who in an essay earlier this year said that "superintelligent" AI could "massively accelerate scientific discovery." Similarly, Anthropic CEO Dario Amodei has predicted AI could help formulate cures for most types of cancer.
Wolf's problem with AI today — and where he thinks the technology is heading — is that it doesn't generate any new knowledge by connecting previously unrelated facts. Even with most of the internet at its disposal, AI as we currently understand it mostly fills in the gaps between what humans already know, Wolf said.
Some AI experts, including ex-Google engineer François Chollet, have expressed similar views, arguing that while AI might be capable of memorizing reasoning patterns, it's unlikely it can generate "new reasoning" based on novel situations.
Wolf thinks that AI labs are building what are essentially "very obedient students" — not scientific revolutionaries in any sense of the phrase. AI today isn't incentivized to question and propose ideas that potentially go against its training data, he said, limiting it to answering known questions.
"To create an Einstein in a data center, we don't just need a system that knows all the answers, but rather one that can ask questions nobody else has thought of or dared to ask," Wolf said. "One that writes 'What if everyone is wrong about this?' when all textbooks, experts, and common knowledge suggest otherwise."
Wolf thinks that the "evaluation crisis" in AI is partly to blame for this disenchanting state of affairs. He points to benchmarks commonly used to measure AI system improvements, most of which consist of questions that have clear, obvious, and "closed-ended" answers.
As a solution, Wolf proposes that the AI industry "move to a measure of knowledge and reasoning" that's able to elucidate whether AI can take "bold counterfactual approaches," make general proposals based on "tiny hints," and ask "non-obvious questions" that lead to "new research paths."
The trick will be figuring out what this measure looks like, Wolf admits. But he thinks that it could be well worth the effort.
"[T]he most crucial aspect of science [is] the skill to ask the right questions and to challenge even what one has learned," Wolf said. "We don't need an A+ [AI] student who can answer every question with general knowledge. We need a B student who sees and questions what everyone else missed."
(Score: 4, Insightful) by chucky on Saturday March 08, @03:22PM (6 children)
I'm fine if AI becomes the fastest librarian in the world. And if a human librarian helps me to formulate the question better, so that the AI doesn't hallucinate...
(Score: 4, Insightful) by Unixnut on Saturday March 08, @05:32PM (2 children)
I was going to say something similar, quite frankly if AI provides me with "a very obedient student" then that is great. I am perfectly capable of reasoning and asking new questions myself. No doubt actual geniuses will be able to come up with even better questions than I can, and they can make use of AI to do the grunt work to come to workable theories much faster than present day.
I see a similar thing at work where they brought in their own chatGPT-like system. I can ask it to write code for me, and the code it spits out is equivalent to a newbie copy/pasting snippets from stackoverflow until they cobble together a working program. Sometimes (e.g. if I need a one off script to do a task) that is all I need from the code, and apart for cursory logic checks to make sure it does what I expect it to we are good to go. While I could whip up a similar program in round half an hour the AI can do it in under 15 seconds (+5 mins logic check).
For more serious programs I tend to have to re-write large chunks of the AI output, but the fact is the bulk of the gruntwork is already there, as are the basics (things like class/program execution structure) as well as hints at logic flow. I can re-write the bits that need it while leaving a lot of the boilerplate code as is. This results in a serious increase in my productivity.
I can imagine the same would happen for researchers, or even the lone genius who doesn't have the funding for their own research team. Suddenly everyone can have access to a virtual "Research assistant" to gather, collate (and even summarise) data for you. That in itself will improve the productivity of inventors, researchers and your everyday genius.
You don't need AI to draw conclusions for you, let alone philosophise upon questions not yet existing. I question if we should even try to make it possible. I really don't need a machine to think for me, I am perfectly capable of thinking for myself.
(Score: 4, Insightful) by c0lo on Sunday March 09, @04:09AM (1 child)
That's far from enough for a quality research assistant - a good one will know how to fudge the data in such a way that the published article receives max citations.
For anything else, the post-grad students are much cheaper.
https://www.youtube.com/@ProfSteveKeen https://soylentnews.org/~MichaelDavidCrawford
(Score: 0) by Anonymous Coward on Monday March 10, @03:17PM
you think you're joking but US universities have a huge influx of Chinese students who are obedient A-students and pre-trained in fudging the results to please Leadership. this does not good science make.
(Score: 2) by corey on Saturday March 08, @08:49PM (2 children)
I agree. The current use of AI seems to be an energy inefficient version of a search engine. But that aside, it’s good as an indexer.
What Wolf says is correct in my opinion, but he’s talking purely from a scientific research perspective. The system he proposes, if it were possible, could then also figure out new ways of stealthily killing off all humans or ways of making itself “smarter”. Maybe anyway.
(Score: 2, Insightful) by Anonymous Coward on Sunday March 09, @12:08AM (1 child)
Most are HORRIBLE as indexers if they can't tell me where they came up with what they parroted.
(Score: 2) by corey on Sunday March 09, @10:41PM
Yeah agree with that. But on Brave search (search.brave.com), it states where it got the AI widget info from, from memory, or at least where i can find more info on the topic.