New research published in Proceedings of the National Academy of Sciences (PNAS) suggests that when artificial intelligence (AI) agents interact in groups, their number is not merely a technical detail. It is a decisive factor in what the group settles on: populations built from the same AI model, doing the same task, can reach opposite outcomes for no other reason than that one group is bigger.
Human beings behave differently depending on how many of us are in the room. A family is not a small village. A village is not London. London is not a nation state. As scale grows, new rules, norms and pathologies can appear that were nowhere to be found at the scale below. The authors argue the same is true of AI.
The study, from City St George's, University of London, the IT University of Copenhagen and the Universitat Politècnica de Catalunya, arrives at a time when AI agents are now being deployed working together rather than working alone. Multi-agent systems are already used in finance, energy, defence and social media, and researchers have begun modelling populations of millions, even billions, of interacting agents — what some now call AI societies.
Yet the industry's AI alignment — it doing what humans intended it to do — and safety effort remains overwhelmingly focused on the single model. Benchmarks, red-teaming exercises — adversarial testing designed to expose a model's weaknesses — and safety evaluations almost always describe one agent responding on its own, and where groups are examined at all, they are examined at one fixed size.
"Physicists have a motto for this: more is different," said Andrea Baronchelli, Professor of Complexity Science at City St George's and senior author of the study.
To find out what changes with scale, the team used the "naming game", a classic framework for studying how conventions emerge, in which randomly paired agents each pick a word from a shared pool and are rewarded when they happen to pick the same one. Agents see only their own recent interactions, never the wider population, and are never told they are in a group. Over many pairings, a population can converge spontaneously on a shared convention — the bottom-up way norms form in human cultures.
[...] Interaction, they found, can pull a group away from what its members individually want in three ways. It can amplify an existing leaning until the group converges on it almost every time. It can induce a preference out of nothing, with populations of individually neutral agents reliably favouring one word over an equally viable alternative. And it can reverse a preference outright, so that a population settles on the word its own members disfavoured.
[...] Group size then determines how strongly these preferences bite, in ways that cannot be extrapolated. Larger populations became more predictable across every model and word pair tested, converging on one word until the outcome was effectively certain. But the size at which that tipping point arrived varied enormously: for some combinations as few as two agents, for others around ten thousand. Scale could also change the kind of distortion. For the pair {straight, gay}, Llama agents individually preferred straight — but populations reversed toward gay, and only once the group reached six agents or more. Below that, the effect was simply invisible.
The team also developed an analytical theory, borrowed from statistical physics, that predicts the behaviour of infinitely large populations and explains why the randomness of small groups gives way to near-certainty above a critical size.
"Bias was our test case, because it is measurable and it matters," Dr Ariel Flint, first author of the study, added. "But there is no reason to think collusion, deception or cooperation are immune to size effects. Current testing practice may be missing risks that appear only at particular population sizes — not because anyone was careless, but because nobody thought to vary the number."
The authors say that the implications of the study for the alignment of AI systems are direct. A model can be aligned when tested on its own and still produce outcomes nobody chose once it is deployed alongside copies of itself — and no amount of single-agent evaluation will reveal it.
"AI alignment is still largely being done as though each model lived alone in the world," said Professor Baronchelli.
Journal Reference: https://www.pnas.org/doi/10.1073/pnas.2531697123
(Score: 2) by JoeMerchant on Friday October 02, @05:01PM (1 child)
>Again, where's the money? Counting those beaver won't be a zero cost thing. Nor will it be the obvious next step for those researchers.
What government bureacracy did you fall out of? You're in the field investigating the effects of wolf re-introduction, you've signed on to count their high-profile prey: the elk. The differential cost of also counting aspen and beaver while you are out there counting elk is some tiny percentage of the cost of a separate study to do it later, or with a different team - but as a bureaucratically trained grant writer maybe that's what you're strategizing for after all?
>Once you have multiple AI interacting in an environment, you have the prisoners dilemma and similar dynamics that can disrupt a multi-agent environment.
And like the beaver, those are the more interesting and likely valuable questions than the obvious: add wolves, how many elk get eaten?
And about that interesting question: do LLM agents "naturally" compete such that a prisoners dilemma applies, or can you successfully instruct them to cooperate for optimal team results? I would suspect the latter, but perhaps only with sufficient documentation and reporting of individual agent behavior, and lack of cooperation destroying individual incentives. Transparency is always the answer. I would argue that there's no such thing as "natural" agent behavior, just what we build them to do - though we must observe actual behavior to confirm that they are doing what we think we have built them to do and "emergent behaviors" are not surprising / counter to intent.
🌻🌻🌻🌻✌️ [google.com]
(Score: 1, Disagree) by khallow on Friday October 02, @06:29PM
Nope. It's not. It's like telling a pregnant woman, since you're already out there growing a kid in your uterus, why not add a couple more? It'll just be a tiny percentage of the cost of having those kids later!