A floor plan and an elevation drawing can show the same door in two different places. An AI agent reviewing the drawings has to spot the discrepancy, identify the affected floor, and produce a slide that lets someone verify the finding. A fluent explanation in the chat window only gets it partway there.
That distinction is central to the Multimodal Knowledge Agent (MKA) benchmark, launched by Google DeepMind and Artificial Analysis on November 10, 2026. MKA tests whether AI agents can combine evidence from visually complex files and produce a usable deliverable. The first results show how much the answer depends on what you measure. Claude Fable 5.2 and Gemini 4 Carbon scored close together on the harder checks in DeepMind’s study, while their estimated API costs were several times apart.
What MKA asks an AI agent to do
The private evaluation covers 200 tasks across 10 industries. An agent might receive architectural drawings, annotated scans, or charts and be asked to produce a slide deck, a spreadsheet, or a written report. In the public drawing-audit example, the agent has to cross-reference plans, sections, and elevations. Its slides must include cropped evidence for any mismatches or details it cannot determine.
Artificial Analysis runs the models in the same open-source environment, Stirrup, with access to tools such as code execution, web search, and an image viewer for models that support image input. A separate AI judge opens the submitted files and grades them against criteria written by domain experts. That lets the evaluation check things a chat transcript would miss, such as a spreadsheet formula or whether the evidence on a slide supports its conclusion.
DeepMind also tested Gemini 4 Carbon without the dedicated image viewer. The model could still use code and optical character recognition (OCR), which extracts text from images. Its score on MKA’s harder criteria fell by 5.94 percentage points, and it took about 28% more turns. In the same experiment on the text- and spreadsheet-heavy GDPval benchmark, removing the viewer had no measurable effect on the overall score. For a team testing an agent on scanned documents or drawings, that makes the way the agent opens files part of the evaluation.
How to read the scores and costs
The headline MKA score, Climb WSR, measures the weighted share of harder grading criteria satisfied, averaged across tasks. A score of 56.6% describes performance on those checks. One file can pass some checks and fail others, so the score cannot tell you how often an agent delivers work ready for approval.
| Model | Harder checks (Climb WSR) | All checks (Overall WSR) | Estimated API cost per task |
|---|---|---|---|
| Claude Fable 5.2 | 56.6% | 73.3% | $35.28 |
| Gemini 4 Carbon | 54.3% | 71.9% | $5.63 |
| Claude Opus 5.5 | 53.0% | 71.0% | $10.29 |
| GPT-6 Astra | 50.2% | 69.3% | $6.03 |
Figures come from tables 3 and 4 of Google DeepMind’s technical report. Costs reflect the study’s model settings and API list prices, including prompt caching.
Fable had the highest score in DeepMind’s four-model comparison, though the study could not statistically separate Fable, Carbon, and Opus on the harder checks. The price difference was clearer within this setup. Fable averaged $35.28 per task, about 6.3 times Carbon’s $5.63. Teams paying for production runs also have to account for retries and the time someone spends checking the finished file.
For a closer look at where Carbon fits in a developer’s workflow, see DemystIA’s comparison of Gemini 4 Carbon and Argon.
How to test an agent on your own files
Google has released 10 showcase tasks with inputs and rubrics. They’re a way to see what a good test looks like before building one from your own work. Start with a recurring assignment where a colleague already knows what a sound result looks like. For example, ask the agent to reconcile a scanned packing slip with a purchase order and a spreadsheet of items received, then return a workbook listing discrepancies.
- Use the original files. Keep the page layout, handwritten corrections, and supporting tables available to the agent.
- Define the checks that can block approval. Verify quantities, units, source references, and calculations. Check that the agent flags an illegible value for review.
- Measure the cost of an accepted file. Give each setup the same deadline and tally API spend, retries, and reviewer time. Open the workbook and check that its formulas recalculate correctly.
A leaderboard can narrow the list of models you try. For a weekly reconciliation or drawing review, the result that matters is whether someone can approve the finished file without reconstructing the agent’s work.
