1. What did the DeepMind researchers find?
Google DeepMind researchers have published a paper arguing that large language models cannot be trusted to produce genuinely random choices on request. The preprint, titled "The Illusion of Stochasticity in LLMs" and posted to arXiv on April 8, 2026, holds that reliable sampling from a probability distribution is a basic requirement for AI agents, and that today's models do not meet it. Authors include Xiangming Gu, Soham De, Michalis Titsias, Larisa Markeeva, Petar Veličković and Razvan Pascanu.
The central finding is a gap between understanding and execution. A model can describe a probability distribution, reason about it and handle the arithmetic involved. Yet when asked to draw samples that actually follow that distribution, its outputs drift away from what the target requires. The authors describe the apparent randomness of model answers as an illusion: responses vary from one run to the next, but variation alone is not the same as faithful sampling.
The study reached its conclusions through formal statistical testing rather than anecdotes about a model repeating a favorite number. That distinction matters for anyone building software on top of these systems, because a weakness shown by controlled tests is easier to reproduce and to measure than a one-off observation.
2. What is random sampling, and why does it matter for AI?
Random sampling means producing outcomes in proportion to a stated set of probabilities. If an agent is told to pick option A 70 percent of the time and option B 30 percent of the time, then over many trials the counts should land close to that split. The same idea covers rolling a fair die, drawing a number uniformly between zero and one, or picking from a bell-shaped Gaussian curve in which values near the middle appear more often than values at the extremes.
Many practical tasks depend on this ability. Simulations draw from distributions to model uncertainty. Games and negotiations call for mixed strategies, in which a player deliberately varies its moves so an opponent cannot predict them. Security routines rely on unpredictable values. As language models are placed in the role of agents that make decisions, plan actions and interact with other parties, they are increasingly asked to behave in these ways.
The paper therefore frames sampling as a foundation rather than a curiosity. If an agent infers a distribution from data and then acts on it, a skewed draw can quietly distort the decision that follows, and the error may be hard to spot from the outside because each individual output looks plausible.
3. Why do LLM outputs only look random?
Language models generate text one token at a time, choosing each token from a probability distribution that the model computes internally. That built-in variability explains why the same prompt can yield different answers on different attempts. It can create the impression that a model is rolling dice behind the scenes.
The researchers stress that this token-level variation is not controllable sampling. A model that wanders among several possible phrasings has not necessarily learned to follow a particular distribution when told to. Asking a model for a random number from one to ten, for example, invites it to reproduce patterns from its training text, where certain numbers appear more often than others, rather than to choose evenly.
The study reports that the sampling errors it found were systematic. Models showed preferences for particular numbers, and the sequence in which information appeared in a prompt also shaped what came back. A fair random source should be indifferent to both. These two patterns, favored values and sensitivity to ordering, are the sort of bias that a formal goodness-of-fit test is designed to expose.
4. How can models be made to sample correctly?
The authors report a workable remedy. When a model is handed random seeds generated outside itself, for instance numbers produced by ordinary code, it can map those seeds onto a target distribution much more faithfully. In this setup the supplied seeds supply the unpredictability, and the model contributes only the deterministic transformation that reshapes raw randomness into the requested form.
That division of labor reflects what the paper identifies as the real weakness. Frontier models can perform the arithmetic that converts a uniform random input into, say, a skewed or bell-shaped outcome. What they struggle to do is generate the raw randomness on their own. In the authors' framing, the model is competent at the math and unreliable as the source of chance.
The practical advice for developers follows directly: do not ask the model to be random. Generate the random values in code, pass them in as inputs, and let the model use them. This keeps the unpredictable element in a component that is built to deliver it, and keeps the language model where it performs well.
5. Why can biased randomness become a security problem?
Predictability is a liability whenever an agent faces an opponent. In competitive settings, a strategy that depends on unpredictable choices only works if the choices are in fact unpredictable. An agent whose supposedly random moves lean toward certain values hands an adversary a pattern to exploit, and an attacker who learns the tendency can adjust accordingly.
The paper treats unfaithful sampling as a distinct failure point for agentic systems rather than a minor quirk. Because agents often act on distributions inferred from data, a bias in the draw can tilt outcomes in ways that external observers cannot easily detect. Standard checks that only confirm the output is varied would miss the problem entirely.
This is also why the issue extends beyond games. Any workflow that relies on an agent to vary its behavior, whether to split traffic, to select test cases, to randomize an audit or to avoid being anticipated, inherits the same limitation unless randomness is supplied externally.
6. What does this mean for companies using AI agents?
For organizations evaluating or deploying agents, the study suggests testing statistical fidelity directly instead of assuming it. A model that gives different answers each time may appear unpredictable while remaining biased in a way that is measurable and exploitable. Checking the frequency of outputs over many trials against the intended distribution is a more reliable test than inspecting a handful of responses.
The finding also bears on a broader debate about what language models can and cannot do natively. The paper adds to a growing body of research that examines whether these systems handle statistical tasks reliably, and it cleanly separates one capability that works, transforming supplied randomness, from one that does not, producing randomness. That separation gives engineers a concrete design rule rather than a vague warning.
Builders can apply the lesson without waiting for better models. Route every decision that needs real chance through a conventional random number generator, log the values, and give the model only the job of interpreting them. Doing so makes behavior easier to audit, because the random inputs can be inspected and replayed.
7. How do statistical tests expose a biased sampler?
A goodness-of-fit test asks a simple question: if the outputs really came from the stated distribution, how likely is it that the observed counts would look like this? The answer is reported as a p-value. A very small value means the observed pattern would be extremely unlikely under the claimed distribution, which is strong evidence that the sampler is not doing what it says. Testing a model this way turns a vague impression of bias into a number that can be compared across models and settings.
The approach also avoids a common trap. A person looking at ten or twenty outputs from a model may see no obvious pattern, because small samples hide skew. Only by collecting many draws and comparing their overall shape with the target can a reviewer tell whether the model is honoring the distribution. That is why the paper's reliance on formal testing makes its conclusions more persuasive than informal demonstrations.
For readers without a statistics background, a useful comparison is a die that looks normal but lands on six a little too often. No single roll proves anything, yet across thousands of rolls the imbalance becomes unmistakable. The researchers applied the same logic to language models asked to behave like fair dice and similar sources of chance.
8. Where does this leave agent design?
Taken together, the findings point toward a hybrid design in which each component does what it does best. Deterministic code supplies chance, and the language model supplies judgment, language understanding and the ability to reshape inputs into the form a task requires. Neither part is asked to imitate the other.
The same thinking applies to evaluation. Teams that measure agents on accuracy or helpfulness alone may never notice a sampling flaw, since the flaw only shows up in aggregate behavior over many runs. Adding distribution-level checks to an evaluation suite is a modest cost compared with discovering the problem after an adversary has already learned the agent's habits.
Finally, the result is a reminder that fluent output is not proof of an underlying capability. A model can discuss probability convincingly while failing to act on it, and the gap between describing a skill and performing it is exactly what the researchers set out to measure.
9. What comes next?
The paper is a research preprint, and the immediate question is whether future models close the gap or whether external randomness becomes a standard part of agent design. Because the authors show that supplying seeds restores accurate sampling, the fix is available today and does not depend on retraining.
Readers following the work can watch for follow-up studies that apply the same statistical tests to newer model releases, and for agent frameworks that add a built-in random source as a default tool. Until then, the DeepMind result stands as a clear caution: when an AI agent is told to roll the dice, the dice should come from somewhere else.