1. Coinbase fraud detection model beats a frontier system
Coinbase reported on October 8, 2026 that a fine-tuned version of Alibaba's open Qwen3.5-9B language model outperformed Anthropic's Opus 4.5 on the exchange's internal fraud benchmark for Onramp, its service for buying crypto. The smaller model led on all four detection metrics, posted an F1 score 9.6% higher, and cut end-to-end latency by 55% during online inference.
The results come from the third and final installment of a Coinbase blog series called Owning Intelligence, published across October 7 and 8. The series describes how the company built, tested and then post-trained an artificial intelligence agent to review transactions for signs of fraud, and it argues that a model tailored to one narrow job can beat much larger general-purpose systems at that job.
All of the performance figures are self-reported. They were measured on a benchmark that Coinbase assembled from its own transaction data, so outside researchers cannot rerun the comparison. That limitation matters when weighing the headline claim, and it is returned to below.
2. What did Coinbase actually build?
The project is a fraud risk agent powered by a large language model and designed for Onramp, the product that lets customers purchase cryptocurrency, typically with a card or bank payment. Fraud is a persistent problem for any service of this kind, because stolen payment credentials are often used to buy digital assets that can be moved out quickly and are hard to recover.
The agent does not replace the company's existing defenses. It sits downstream of the machine-learning models Coinbase already operates and examines transactions that have passed through those earlier checks. In effect it acts as an additional layer of contextual review, looking at cases where the first line of automated screening has not already settled the question.
Its output is deliberately narrow. The agent returns a risk level and nothing more, which means the systems around it can keep working as they did before. Coinbase did not have to rebuild its fraud pipeline to accommodate the new component, and the surrounding rules and models treat the agent's answer as one further signal.
Layering is common in payment security. Inexpensive statistical models screen every transaction quickly and clear the great majority of them. Slower and more expensive reviewers are reserved for the ambiguous remainder, where context such as account history, device details and purchase patterns can change the verdict. Coinbase's agent fits that second tier, which is why its per-request cost and response time matter as much as its accuracy.
3. How did the agent perform in live testing?
Before the model comparison, Coinbase described the agent's effect on real traffic. In an online A/B test, the group of transactions reviewed by the agent showed 30% fewer fraudulent transactions and a 22% reduction in the dollar value of fraud compared with the control group.
The two figures measure different things. A drop in the count of fraudulent transactions means fewer stolen-credential purchases are being completed, while a drop in dollar value shows that the losses that remain are smaller. A system can reduce the number of incidents without reducing the money lost if it only catches low-value attempts, so the pairing of the two numbers gives a fuller picture of the improvement.
In practical terms, fewer fraudulent purchases getting through usually means fewer disputed card payments and fewer chargebacks for the platform to absorb. Those savings are a large part of the business case for deploying a language-model agent, which costs more to run per transaction than a conventional scoring model.
Fraud rates differ by season, region and payment method, so a result from one testing window does not guarantee the same reduction later, and the figures describe a test and not a permanent outcome.
4. How was the Qwen model trained and tested?
Coinbase started from Qwen3.5-9B, an open-weight model from Alibaba with roughly nine billion parameters. That is small by the standards of today's flagship systems, which are generally far larger and are accessed through a paid interface rather than run on a company's own hardware.
The team post-trained the model for Onramp fraud detection using reinforcement learning, a method in which a model is rewarded for producing answers that match known outcomes instead of simply imitating example text. In this case the rewards were tied to outcomes in Coinbase's proprietary fraud data. The company used SkyRL, a reinforcement-learning framework, running on the Anyscale platform, and it served the finished model with Ray Serve LLM and vLLM, two tools for running language models in production.
The model was then measured against frontier systems on a benchmark built from Coinbase's own Onramp transactions, including Anthropic's Opus 4.5. The detection metrics used in such a comparison typically include precision, which is the share of flagged transactions that were truly fraudulent, and recall, which is the share of all fraud that was caught. F1 combines the two into a single score, and a dollar-weighted version of recall counts large fraudulent purchases more heavily than small ones.
According to Coinbase, the fine-tuned Qwen model exceeded Opus 4.5 on every one of those detection measures. Its F1 score was 9.6% higher, and it also finished its work faster, with end-to-end latency in online inference 55% lower.
The appeal of reinforcement learning for this kind of work is that fraud outcomes are eventually known. Chargebacks and confirmed cases arrive after the fact, giving the company a stream of labeled results it can use as a training signal. A general-purpose model has never seen that signal for Coinbase's customers, which is the advantage a post-trained model holds.
5. Why does speed matter for a fraud check?
A fraud review happens while a customer is waiting for a purchase to go through, so every additional moment of delay is felt in the checkout experience. A model that answers in well under the time a larger system needs allows the company to run the review on more transactions without slowing payments down.
Speed also interacts with cost. A smaller model that the company hosts itself generally needs less computing power for each request than a very large model reached through an outside provider. Coinbase's results suggest that the extra capability of a frontier system does not automatically translate into better performance on a task as specific as judging whether an Onramp purchase is fraudulent.
Hosting a model in-house also gives a company control over data handling. Sensitive transaction details do not need to be sent to an outside provider for every review, which is a consideration for a regulated financial firm.
6. Are newer frontier models always better?
No, at least not on this task. The second installment of the series compared successive generations of general-purpose models on the same Onramp benchmark and found that newer releases from Anthropic and OpenAI scored lower than the versions they replaced on recall, F1 and dollar-weighted recall, even in cases where their precision improved.
In other words, the newer systems tended to be more careful about what they flagged but missed more of the fraud that was actually present. For a fraud team, missing fraud is the costlier error, so a model that gains precision while losing recall can be a step backward even though it looks stronger on general-purpose leaderboards.
Coinbase draws a strategic conclusion from this. It favors post-trained models built for a single, well-defined problem over general models that rank highly on public rankings, on the reasoning that leaderboard strength measures broad ability rather than fitness for one operational job. The company's title for the series, Owning Intelligence, reflects the same idea: building and controlling the model that does the work instead of renting a general one.
7. What do the results mean for users and the wider industry?
For people who buy crypto through Onramp, the practical effect is fewer stolen-card purchases reaching completion and fewer losses when something does slip through. The platform bears much of that cost, but the pattern of fraud that fuels chargebacks can also lead to tighter limits and more friction for legitimate customers, so better detection can reduce pressure in that direction.
For the broader industry, the findings add to a growing argument that smaller, specialized models can be competitive with far larger ones when they are trained on a company's own data and measured against the company's own definition of success. Exchanges, payment processors and banks all hold large records of confirmed fraud, and that kind of labeled history is what makes reinforcement learning on a narrow task possible.
The approach also changes the dependence picture. A firm that runs its own post-trained open model is less exposed to changes in a third party's model releases, including the kind of regression Coinbase observed when newer general models performed worse on its benchmark.
8. What are the limits of these findings?
The benchmark is proprietary, so the comparison cannot be independently verified. The results also cover one task on one product. A model that excels at judging Onramp purchases has not necessarily been shown to work for other kinds of fraud, other payment flows or other companies.
Fraud also changes over time. Attackers adapt to the defenses they meet, and a model trained on past patterns can lose accuracy as tactics shift. Keeping a specialized model effective generally requires continued monitoring and periodic retraining on fresh outcomes.
The finding that newer general-purpose models underperformed their predecessors deserves further study. The difference may come from how those systems handle structured risk judgments, or it may change with future updates. Coinbase's data does not settle which explanation is correct, and the result applies to the specific benchmark it used.
Independent replication would make the claim considerably stronger. Until other teams publish comparable tests on their own data, the result is best read as a documented case study from one company, with encouraging numbers, and not as a general ranking of AI models.