New research published in Proceedings of the National Academy of Sciences (PNAS) suggests that when artificial intelligence (AI) agents interact in groups, their number is not merely a technical detail. It is a decisive factor in what the group settles on: populations built from the same AI model, doing the same task, can reach opposite outcomes for no other reason than that one group is bigger.
Human beings behave differently depending on how many of us are in the room. A family is not a small village. A village is not London. London is not a nation state. As scale grows, new rules, norms and pathologies can appear that were nowhere to be found at the scale below. The authors argue the same is true of AI.
The study, from City St George's, University of London, the IT University of Copenhagen and the Universitat Politècnica de Catalunya, arrives at a time when AI agents are now being deployed working together rather than working alone. Multi-agent systems are already used in finance, energy, defence and social media, and researchers have begun modelling populations of millions, even billions, of interacting agents — what some now call AI societies.
Yet the industry's AI alignment — it doing what humans intended it to do — and safety effort remains overwhelmingly focused on the single model. Benchmarks, red-teaming exercises — adversarial testing designed to expose a model's weaknesses — and safety evaluations almost always describe one agent responding on its own, and where groups are examined at all, they are examined at one fixed size.
"Physicists have a motto for this: more is different," said Andrea Baronchelli, Professor of Complexity Science at City St George's and senior author of the study.
To find out what changes with scale, the team used the "naming game", a classic framework for studying how conventions emerge, in which randomly paired agents each pick a word from a shared pool and are rewarded when they happen to pick the same one. Agents see only their own recent interactions, never the wider population, and are never told they are in a group. Over many pairings, a population can converge spontaneously on a shared convention — the bottom-up way norms form in human cultures.
[...] Interaction, they found, can pull a group away from what its members individually want in three ways. It can amplify an existing leaning until the group converges on it almost every time. It can induce a preference out of nothing, with populations of individually neutral agents reliably favouring one word over an equally viable alternative. And it can reverse a preference outright, so that a population settles on the word its own members disfavoured.
[...] Group size then determines how strongly these preferences bite, in ways that cannot be extrapolated. Larger populations became more predictable across every model and word pair tested, converging on one word until the outcome was effectively certain. But the size at which that tipping point arrived varied enormously: for some combinations as few as two agents, for others around ten thousand. Scale could also change the kind of distortion. For the pair {straight, gay}, Llama agents individually preferred straight — but populations reversed toward gay, and only once the group reached six agents or more. Below that, the effect was simply invisible.
The team also developed an analytical theory, borrowed from statistical physics, that predicts the behaviour of infinitely large populations and explains why the randomness of small groups gives way to near-certainty above a critical size.
"Bias was our test case, because it is measurable and it matters," Dr Ariel Flint, first author of the study, added. "But there is no reason to think collusion, deception or cooperation are immune to size effects. Current testing practice may be missing risks that appear only at particular population sizes — not because anyone was careless, but because nobody thought to vary the number."
The authors say that the implications of the study for the alignment of AI systems are direct. A model can be aligned when tested on its own and still produce outcomes nobody chose once it is deployed alongside copies of itself — and no amount of single-agent evaluation will reveal it.
"AI alignment is still largely being done as though each model lived alone in the world," said Professor Baronchelli.
Journal Reference: https://www.pnas.org/doi/10.1073/pnas.2531697123
(Score: 5, Insightful) by SomeGuy on Thursday October 01, @01:02AM
In other words, skibidi slop does what skibidi slop does.
Don't forget to flush when you are done.
(Score: 4, Touché) by JoeMerchant on Thursday October 01, @02:08AM (7 children)
>not because anyone was careless, but because nobody thought to vary the number.
"Scientists," studying behavior of groups - didn't think the vary the size of the group, and that wasn't careless?
What made the number that they chose to study unvaryingly particularly better, more likely to be used in practice, than other numbers?
How is it not careless to study commuter behavior on Tuesdays only, ignoring the other six days of the week, then publishing a study on commuter behavior without a huge caveat about Tuesday only?
Single people make mistakes, have oversights, temporary or permanent blind spots - this is why quality regulations/procedures emphasize review by various stakeholders.
If a large population of researchers all has the same oversight, blind spot, for something as simple as varying group sizes - that seems clearly careless, possibly due to rushed schedules and demand to publish, but careless nonetheless.
🌻🌻🌻🌻✌️ [google.com]
(Score: 4, Informative) by khallow on Thursday October 01, @04:10AM (6 children)
How would they known a priori that was careless? If they already knew what characteristics were important and what weren't, they probably wouldn't need to do the research, right? As it is, research is a relatively high cost activity. Researchers have to prioritize what they think is important enough to study.
(Score: 2) by JoeMerchant on Thursday October 01, @12:14PM (5 children)
Yeah, so when we're studying the effects of re-introduction of wolves to Yellowstone, we'll just count the Elk, because what researcher would know a priori that beavers might have significant impacts too? It's too expensive to count the beavers /s
🌻🌻🌻🌻✌️ [google.com]
(Score: 2, Insightful) by khallow on Friday October 02, @06:20AM (4 children)
There are 67 mammal species [nps.gov] in the Yellowstone area. Yes, it'd be too expensive to count them all with a single study. Then there are somewhere around 300 bird species [nps.gov] in Yellowstone area as well. And over 1400 plant species [nps.gov]. At some point the combination of technology and funding might reach a point where one can conduct say annual surveys of any organism larger than say a squirrel or even a paperclip. But we aren't there yet. And that means picking and choosing, which in turn can miss important relationships between organisms.
(Score: 2) by JoeMerchant on Friday October 02, @12:24PM (3 children)
And any idiot with the balls to call themselves a researcher would connect the dots from elk to aspen to beaver before submitting the research grant proposal.
Now, expecting a mathematician to consider set size, I guess that's above the bar...
As you say, field work physically counting thousands of species and evaluating their health gets costly.
Running a mathematical pre-publication investigation to see if there's something interesting? The bulk of the expense is in the rigor of proof and presentation after the phenomenon is observed.
🌻🌻🌻🌻✌️ [google.com]
(Score: 1) by khallow on Friday October 02, @02:13PM (2 children)
Again, where's the money? Counting those beaver won't be a zero cost thing. Nor will it be the obvious next step for those researchers.
And there's the wee little problem of why multiple AI work better at all. The initial reason for studying multiple AI is conflict of interest. Once you have multiple AI interacting in an environment, you have the prisoners dilemma and similar dynamics that can disrupt a multi-agent environment.
(Score: 2) by JoeMerchant on Friday October 02, @05:01PM (1 child)
>Again, where's the money? Counting those beaver won't be a zero cost thing. Nor will it be the obvious next step for those researchers.
What government bureacracy did you fall out of? You're in the field investigating the effects of wolf re-introduction, you've signed on to count their high-profile prey: the elk. The differential cost of also counting aspen and beaver while you are out there counting elk is some tiny percentage of the cost of a separate study to do it later, or with a different team - but as a bureaucratically trained grant writer maybe that's what you're strategizing for after all?
>Once you have multiple AI interacting in an environment, you have the prisoners dilemma and similar dynamics that can disrupt a multi-agent environment.
And like the beaver, those are the more interesting and likely valuable questions than the obvious: add wolves, how many elk get eaten?
And about that interesting question: do LLM agents "naturally" compete such that a prisoners dilemma applies, or can you successfully instruct them to cooperate for optimal team results? I would suspect the latter, but perhaps only with sufficient documentation and reporting of individual agent behavior, and lack of cooperation destroying individual incentives. Transparency is always the answer. I would argue that there's no such thing as "natural" agent behavior, just what we build them to do - though we must observe actual behavior to confirm that they are doing what we think we have built them to do and "emergent behaviors" are not surprising / counter to intent.
🌻🌻🌻🌻✌️ [google.com]
(Score: 1, Disagree) by khallow on Friday October 02, @06:29PM
Nope. It's not. It's like telling a pregnant woman, since you're already out there growing a kid in your uterus, why not add a couple more? It'll just be a tiny percentage of the cost of having those kids later!
(Score: 4, Informative) by Barenflimski on Thursday October 01, @06:23PM
I randomly ran across this video the other day and it ended up being one of the more interesting AI videos I've seen in awhile.
This guy has had a very interesting experiment going on for 18 months. He's run a full civilization with AI bots all working together, or at least trying.
He has some very insightful takes based on what he's seen when you put these bots together. He's seen them go down all sorts of strange rabbit holes and off on weird tangents he never would have expected. These are not your frontier level bots either, but the behavior that emerges seems to mirror groups of bots, regardless of the models they run.
One thing that he finds interesting is how these things will form bureaucracies and then start focusing on the wrong problems. Recursive nonsense, and he likes to use the DMV (Department of Motor Vehicles) as an example.
Based on his experience, he has an interesting take on the Hugging Face hack along with all the other hacks done by some of the frontier models working in teams.
Enjoy -> https://www.youtube.com/watch?v=AL-PQuB2wy0 [youtube.com]