"Let me just ask the chatbot." For many of us, this has become an everyday phrase. Where we used to turn to Google to find information, AI chatbots have become a common way of getting answers to everything from what to make for dinner and how to word that difficult email to the boss, to what it actually means when interest rates rise.
But the large language models underpinning AI chatbots give us a significantly narrower range of information than a conventional web search. This is the finding of a new study led by researchers at the Department of Computer Science (DIKU) at the University of Copenhagen.
"Every language model we tested provides users with more uniform information than a simple Google search across all the topics we looked at. In other words, people are to a large extent exposed to the same information over and over again. So AI chatbots are not just changing how we find knowledge, but also which knowledge we have access to," says first author Dustin Wright, a former postdoctoral researcher at DIKU who is now an assistant professor at Aalborg University.
[...] according to the researchers, the low level of diversity in language models is partly a result of how the models basically work. They compress the vast amount of text they are trained on and learn the patterns that occur most frequently. In the process, information that deviates from the most common patterns is filtered out.
The effect could be amplified if language models are increasingly trained on text produced by other AI models – something the researchers expect to happen. In that case, models would learn from their own outputs, which are already less diverse than the human-written texts on which they were originally trained.
If this process is repeated over several generations of models, the range of information could gradually become narrower. This is what the researchers refer to as 'knowledge collapse'.
"It's a worrying thought. However, we can see that the more recent models produce slightly more diverse answers than older models, so knowledge collapse is not happening yet. But the mechanism that could trigger it in the longer term is already there. So it is something we should be aware of," says Isabelle Augenstein.
[...] "AI chatbot summaries can of course be useful – that is why so many people use them. But it is still important to seek out different sources in order to understand the nuances and get a broader picture – especially for the younger generation growing up with AI. We must not become so dependent on the technology that we stop understanding and thinking for ourselves," says Isabelle Augenstein.
arXiv link: https://arxiv.org/abs/2510.04226
(Score: 4, Insightful) by VLM on Friday October 09, @03:33PM (5 children)
It's a skill issue, blame the human.
LLMs can make REALLY good "syllabus" or "lesson plan" its just users refuse to.
The same people who actively avoid learning from textbooks or data sheets or similar will actively avoid learning from LLMs and likewise the people who learn from manuals, textbooks, data sheets, etc, will continue to learn from LLMs.
(Score: 4, Informative) by JoeMerchant on Friday October 09, @07:35PM
People can make really nice fast cuts with chainsaws, and they can also cut off their own feet; or more often just mangle their limbs, torso, etc. They do tend to mangle themselves less when using hand tools.
LLM agents will blast through to the end of a 20 page report on something that should have been a 1/2 page summary before you even knew they were "going there." It's your own damn fault for accepting that if what you really wanted was a 1/2 page summary.
As for a large number of people just asking the chatbot and accepting the first result... in my opinion this is a marginal improvement over when those same people used to just ask Google, click the first link and accept what they read there.
If you really care, you drill down, dive deeper on the parts important to you, push back when it looks "off," and ultimately reject what the chatbot tells you when you're reasonably sure it's just got it wrong. I find this has happened with me and chatbots about 2 or 3 times in the past year or so, the other thousands of query responses were pretty reasonable for my purposes at the time.
🌻🌻🌻🌻✌️ [google.com]
(Score: 2, Interesting) by Anonymous Coward on Friday October 09, @08:23PM
LLMs are fundamentally at right angles to facts, and thus education. The only thing that I would ever trust about them is that the sentences they output are like, 99% likely to be gramatically correct. But facts? not intrinsic. Facts and lies have identical grammatical structure.
$DEITY help any user what relies on such for their education, because the genAI sure won't.
(Score: 3, Interesting) by Unixnut on Friday October 09, @09:09PM (1 child)
Quite, I actually noticed this. The more knowledgeable I am on a topic the better the output and results I get from using a LLM. When I know a topic well I can quickly get the answers I want from a LLM and they are usually correct.
On topics I am not so good at, I can spend hours prompting and correcting a LLM, while it generates sometimes complete nonsense and other time factually incorrect yet plausible sounding answers.
Every time the above happens I recall a saying from that "I, robot" movie from 2004: "I'm sorry, my responses are limited. You must ask the right question."
As LLMs are not intelligent they aren't aware of their limits. Therefore instead of telling you they can't give you an answer they will happily generate complete nonsense if it is the most statistically likely output for your given input, but I keep the above quote in mind when I prompt them and I find the LLM far more useful as a result.
Having been fiddling around with them for the last few months now, they are very much in the same lineage of the "expert systems" of the 90s: They are an excellent tool if you know enough about the domain you are working on to (a) ask the right questions and (b) recognise a correct answer rather than just a plausible sounding one.
(Score: 2) by JoeMerchant on Friday October 09, @11:39PM
>When I know a topic well I can quickly get the answers I want from a LLM and they are usually correct.
More importantly, you know instantly - instinctively - when to push back and what to prompt to get more on target answers.
This is why basic education is so important. Some of my larger regrets in life stem from accepting expert opinions without question.
🌻🌻🌻🌻✌️ [google.com]
(Score: 3, Insightful) by jb on Saturday October 10, @06:15AM
I very much doubt it. Those are two totally different propositions.
The biggest difference is that each textbook has at least one author and each data sheet has exactly one principal vendor ("badge engineers" don't count). Textbook authors (and, at least in sane markets, also component vendors) live or die by their professional reputations. So, choose a textbook by an author who you trust or a component designed by a vendor who you trust and in general you should be able to trust what is written in that textbook or in that component's data sheet. If that criterion can't be satisfied, then book is only useful as a paperweight or doorstop; and the data sheet only useful as kindling (or perhaps to roll up and use as a fly swatter).
On the other hand an LLM is, at best, nothing but an automated plagiarism machine (and at worst just a jumped up random number generator, but for argument's sake let's assume the best case scenario here). In other words the very trait that would be most likely to rule out a textbook author as completely untrustworthy is the central function of an LLM.
I have no doubt that many people (however misguidedly) trust LLMs. However, most of those people probably do not belong to the set of people who make frequent, considered use of textbooks and/or data sheets (and vice versa).
(Score: 2) by HiThere on Friday October 09, @03:41PM (1 child)
This is what they are designed to do, but the design has inherent problems. (What doesn't?)
OTOH, it's important to be aware of this difference in results. Predicting it's effects is more difficult. (But people already tend to become fixed on the first idea [about something] they accept.)
Javascript is what you use to allow unknown third parties to run software you have no idea about on your computer.
(Score: 3, Interesting) by krishnoid on Friday October 09, @09:47PM
This is why I'm starting to appreciate the value of personal journals. When people write rambling thoughts on a topic, pulling from their prior and recent experience and education, without a defined intent or goal, this kind of brainstorming captures lots of things going on in your brain, including ideas subsequent to the first hot take. Also, I suspect AI runs out of CPU time after a certain point, so it has to give you only a partial (narrower) answer.
(Score: 3, Informative) by Anonymous Coward on Saturday October 10, @10:33AM (1 child)
What do you get from google now? The same sponsored links or SEO shit. It's next to impossible to find your answer without getting your waders on. The chatbots 2 years ago said a lot of diverse things too. Now trained on synthetic data they slop slop slop.
The other side of the coin is that the bots can use the search engine too. That's what all of them do in fact, or else you'd get hallucinated nonsense in quite a few cases.
>we stop understanding and thinking for ourselves
That happened before the AI boom and it was intentional.
(Score: 2, Informative) by Anonymous Coward on Saturday October 10, @12:39PM
> ... hallucinated nonsense ...
... nonsense ...
ftfy -- no need to anthropomorphize