Jul, 134 min to read
Frontend
News · Google DeepMind splits general intelligence into 10 cognitive abilities and crowdsources the missing evaluations
Mar, 174 min to read
FrontendGoogle DeepMind splits general intelligence into 10 cognitive abilities and crowdsources the missing evaluations
A new cognitive taxonomy names what AGI progress should be measured against — and a $200,000 Kaggle hackathon admits five of those abilities have no good tests yet.
What DeepMind put on the table on March 17
Google DeepMind released a paper, "Measuring Progress Toward AGI: A Cognitive Taxonomy," that breaks general intelligence into 10 named abilities: perception, generation, attention, learning, memory, reasoning, metacognition, executive functions, problem solving, and social cognition.
Rather than propose a single AGI score, the paper attaches a three-stage protocol to each ability: run systems across a broad task suite using held-out test sets to avoid contamination, collect human baselines from a demographically representative sample of adults, then map the model's performance against the distribution of human performance.
The framing is explicit that this is one piece, not the whole picture. The authors say cognitive science 'provides one important piece of the puzzle' — a hedge worth taking at face value, because the taxonomy is a vocabulary for measurement, not a claim that measurement exists.
The five abilities they admit they can't measure yet
The most honest detail is in the hackathon scope. Of the 10 abilities, the Kaggle challenge targets exactly the five 'where the evaluation gap is the largest': learning, metacognition, attention, executive functions, and social cognition.
That list is telling. The abilities with mature benchmarks — reasoning, problem solving, generation — are the ones already saturating leaderboards. The five with no good tests are the ones that determine whether an AI product feels reliable in a real interface, not just capable on a benchmark.
Metacognition is knowing when you don't know. Attention is focusing on what matters in a crowded context. Executive functions cover planning and inhibition. These are precisely the failure modes users hit at the frontend — a confident wrong answer, an agent that loses the thread mid-task, a tool that acts before it should. DeepMind is naming those gaps as measurement gaps, not solved problems.
Why human-baselined, per-ability scoring changes the reporting
The protocol's insistence on mapping model performance against a human distribution per ability pushes away from the single-number reporting that dominates model launches. A system could sit above the human median on generation and below it on metacognition, and this framework would keep those separate.
For anyone building on frontier models, that decomposition is more useful than an aggregate. It suggests evaluating a candidate model against the specific ability your product leans on — say, executive function for an agent workflow, social cognition for a support surface — rather than trusting a composite ranking.
The mechanics are concrete: submissions run on Kaggle's newly launched Community Benchmarks platform against a lineup of frontier models. The prize structure is $10,000 for the top two in each of the five tracks and $25,000 grand prizes for the four best overall, a $200,000 pool, with submissions open March 17 to April 16 and results on June 1.
What the outsourced evaluations mean for teams shipping AI features
The specific implication is that Google is asking the outside research community to define the tests for the very capabilities that govern user-facing reliability — and doing it in the open on a public benchmarking platform.
If the June 1 results produce usable evaluations for attention, metacognition, and executive function, teams get a shared, contamination-resistant way to check the behaviors that break products in front of users, not just the ones that top research benchmarks.
The gap the hackathon exists to close is the same gap frontend teams paper over today with guardrails, retries, and confidence heuristics. Watching which of the five tracks yields durable evaluations — and which stay unsolved after April 16 — is a more grounded signal of AGI progress than any single model release this year.
Find this story relevant to you?
Contact us to find a unique solution
Need an AI engineering partner that can actually build?
We help businesses integrate AI, build AI-powered products, automate high-value workflows, and modernize the software systems behind them.
Related reading
More analysis around product delivery, operational AI, and the systems work that makes deployment hold up in reality.