1. MKA benchmark launches with 200 private tasks
Google DeepMind and the independent evaluation firm Artificial Analysis released the MKA benchmark, short for Multimodal Knowledge Agent, on November 10, 2026. The test measures how well autonomous AI agents handle visually dense professional work, using 200 private tasks drawn from 10 industry verticals and shaped by specialists from 83 occupations.
The release was accompanied by a DeepMind blog post and a technical report that sets out the methodology and the first set of results. Together they present MKA as a different kind of measurement from the question-and-answer tests that have dominated the field: closer to a review of finished work than to an exam.
An AI agent, in this context, is a system built on a large language model that can plan and carry out a sequence of actions with limited human direction, such as opening files, running calculations and producing documents. Evaluating such systems is harder than evaluating a chatbot, because the output is a piece of work rather than a single answer.
The timing reflects where the industry's attention has moved. Developers increasingly market their models as agents capable of completing extended assignments, and customers want evidence that those claims hold up on work resembling their own. A benchmark built around hours-long professional projects is an attempt to supply that evidence in a form that can be compared across models.
2. What is the MKA benchmark?
MKA is an evaluation that asks AI agents to complete realistic professional assignments built around images and diagrams, and then grades the finished product. It deliberately leaves out trivia and puzzle-solving. The tasks are meant to resemble what a working specialist would actually be handed.
Each assignment requires the agent to interpret demanding visual material, including architectural blueprints and engineering schematics. Reading these correctly involves more than recognising objects in a picture. A blueprint encodes dimensions, relationships between spaces and conventions that a trained professional learns to decode, and a schematic conveys how the parts of a system connect.
Interpretation is only half the job. The agent must then produce a deliverable, such as a spreadsheet or a slide deck. The tasks are modelled on multi-step projects that would occupy an experienced human professional for roughly three to eight hours, which places them well beyond the short prompts used in many earlier tests.
The breadth of input is one of the benchmark's selling points. Specialists from 83 different occupations contributed to the design of the tasks, which span 10 verticals. That involvement is intended to keep the assignments faithful to real practice rather than to a researcher's impression of what a given profession does.
3. How are the agents scored?
Scoring rests on rubrics written by experts, and those rubrics judge two things: whether the content is factually accurate and whether the deliverable is well designed. Both count toward the final mark, so an agent cannot score well on substance alone.
The design component is unusual. Under MKA, an agent that reaches the right conclusion but presents it in a cluttered or unreadable slide deck still loses points. The reasoning is that a deliverable has value only if the person receiving it can use it. A correct figure buried in a confusing layout fails the client just as an incorrect figure would, even if the failure is of a different kind.
Applying detailed rubrics to hundreds of long-form outputs by hand would be slow and expensive, so the benchmark uses what its authors call an agentic autorater. This is an AI system whose job is to assess the work of other AI systems against the rubric criteria. The research team checked the autorater's judgements against those of human domain experts and reported a high rate of agreement between the two.
Using one AI system to grade another raises a natural question about reliability, which is why the comparison with human experts matters. An automated grader is useful only if its verdicts track those a qualified person would reach. The check against domain specialists is the benchmark's answer to that concern, and it is also what allows the test to be run repeatedly on new models without convening a panel of professionals each time.
4. Why are the tasks kept private?
Secrecy is a defence against a well-known problem in AI evaluation. The 200 tasks in the main evaluation set are not public, which means developers cannot easily train their models on the answers ahead of time.
Public benchmarks tend to lose their value over time. Once test questions circulate on the internet, they can end up in the data used to train new models, whether deliberately or by accident. A model that has effectively seen the exam can post a high score without possessing the underlying skill. Holding the tasks back is a way to keep the measurement meaningful for longer.
The cost of that choice is transparency. Outside researchers cannot inspect the full set of tasks to judge for themselves whether they are fair, representative or well constructed. They must rely to a greater extent on the description provided by the organisations that built the test.
5. Early results: Gemini 4 Carbon second on quality, far cheaper to run
The first leaderboard puts Google's own Gemini 4 Carbon model in second place for quality. The results also include a cost comparison: Gemini 4 Carbon completed the tasks at roughly six times lower cost on average than the top-scoring model.
Cost has become an important dimension of agent evaluation. A task that takes a professional several hours can require an AI agent to process a great deal of material and take many steps, and each step consumes computing resources that the user pays for. Two models of similar quality can therefore differ sharply in what it costs to finish the same piece of work, and buyers deploying agents at scale care about both numbers.
The pairing of a quality rank with a cost figure points to how the developers want the results read. Coming second on quality while costing a fraction as much to run is presented as a favourable trade for many practical uses, where a small difference in output quality may matter less than a large difference in the bill.
The results come with an obvious caveat. DeepMind helped build the benchmark, and a DeepMind model performed strongly on it once cost is taken into account. That relationship does not make the findings wrong, but readers have reason to bear it in mind when weighing them.
6. The gap MKA is trying to fill
Most established benchmarks for language models are built entirely from text. They test whether a model can answer questions, write code or reason through a written problem. MKA sets out to add visual reasoning to that picture.
The argument for doing so is that professional work is rarely made of text alone. Architects work from drawings. Engineers work from schematics. Analysts work from charts, tables and page layouts. A system that performs well on written questions may still struggle when the essential information is carried by a diagram.
MKA is also designed to test the whole cycle of knowledge work rather than one part of it. An agent has to take in complicated source material, reason about what it means and then deliver something usable. Weakness at any of those stages shows up in the final score, which is closer to how a human employee's output would be judged.
That end-to-end framing separates MKA from tests that isolate a single skill. A model can be excellent at describing an image and still fail to turn what it sees into a coherent analysis, or produce a sound analysis and then present it poorly. By grading only the finished deliverable, the benchmark rewards systems that hold together across every step and exposes those that break down along the way.
7. What it means for developers and buyers
For companies building AI models, the benchmark raises the importance of multimodal ability, meaning the capacity to work across text and images together. It also places presentation on the scorecard. If rubrics penalise cluttered output, formatting can no longer be treated as an afterthought to be fixed by the user.
For organisations deciding which agent to deploy, a test built on realistic assignments offers a more direct signal than abstract scores. A ranking that reflects several hours of simulated professional work, together with the cost of producing it, speaks more plainly to the question a buyer actually has: whether the system can do the job, and at what price.
The limitations are worth noting. Two hundred tasks across 10 sectors is substantial for a benchmark assembled by hand, but it covers only a portion of professional activity. A strong result in one vertical may not carry over to another. The automated grading, however closely it tracks human experts overall, may also diverge from them most often on the hardest or most subjective tasks, where a wrong call matters most.
8. What comes next?
Whether MKA becomes a common yardstick now depends on the rest of the industry. The partnership pairs Artificial Analysis, which brings the standing of a third-party evaluator, with DeepMind, which brings research resources along with a direct competitive interest in the outcome.
The deciding factor will be whether other AI laboratories submit their models and accept the methodology. A benchmark gains authority when rival developers treat its results as legitimate, including when those results are unflattering. If competitors engage with it, MKA could become a standard reference for visually demanding agent work. If they do not, it risks being seen mainly as a showcase for its co-creator's models.
Further results as more models are tested will provide the first indication of which way that goes.