Promptea.
PolicyMajor

Anthropic says Claude now leads 26% of its own AI research

Anthropic published three measurements of how AI gets built inside its own walls: the share of its AI research Claude leads, how its 30,000-agent fleet is monitored, and how much compute goes to safety. The figures are the company's own, and nobody outside has audited them.

Promptea Editorial6 min read

Anthropic published three measurements on Thursday that try to put numbers on something the industry has so far argued about in adjectives: how fast frontier AI development is actually moving, and how much of it is now being done by AI. The figure that will travel is that Claude "leads" 26% of Anthropic's AI R&D work as of August 2026, up from under 1% in February.

The publication is a proposal as much as a disclosure. Anthropic is arguing that these three things should be measured and published regularly by every frontier lab, and it went first. It follows CEO Dario Amodei's essay We Must Pace the Frontier earlier this month, which called for the industry to slow capability gains and committed Anthropic to giving third-party evaluators employee-level access.

The three measurements

In Anthropic's own framing: "We're sharing three measurements that will help the public track AI development inside frontier AI labs: how much of AI R&D is performed by AI itself, how well the actions of AI agents are overseen, and how compute is allocated."

  • AI-led AI R&D — what share of the company's own research work is now performed by its models rather than its staff.
  • Agent oversight — how completely the actions of internal AI agents are monitored, and how often a monitor intervenes.
  • Compute allocation — how the company's computing budget splits between safety work and everything else.

The stated purpose is to shrink the gap between what labs know and what everyone else does. Capability benchmarks describe what models can do; these are meant to describe how the models get built.

Where the 26% comes from

The number sits on a scale Anthropic borrowed rather than invented. It uses an automation rating scale developed by Epoch AI that measures an "Automation Level," or AL, running from AL0 (no AI involvement) to AL5 (fully autonomous). At AL3, AI "collaborates" — substantial work under close human direction. At AL4, AI "leads": it completes most of a task end to end from a high-level prompt while a human supervises.

The 26% is the share of work at AL4 or above. The broader figure is less quoted and arguably more telling: "The share of work at or above 'AI collaborates' is above 90%." On Anthropic's own measure, nearly all of its AI research now has a model materially involved; the contested part is how much of it the model is running.

Anthropic calls the construct the Anthropic R&D Automation Index, and describes it as a prototype. Building it involved cataloguing the kinds of work done at the company into a frozen hierarchical task tree — "The resulting tree has 542 nodes at different depths, of which 378 are leaves like 'eval platform defect diagnosis and fixes,' 'RL sandbox egress and network policy,' and 'serving incident postmortems.'" Claude research agents then sample staff work records to list tasks, an independent Claude judge model assigns an automation level to each from the gathered evidence, and the ratings are aggregated by person-time weighting, with each person's week split evenly across their assigned tasks.

Two things are worth holding onto about that pipeline. Claude is doing much of the rating of how much work Claude does, which Anthropic is open about but which is a real dependency. And person-time weighting measures the share of *task categories* covered, not the share of output, value or difficulty — a task where Claude leads and a task where it cannot help are weighted by how much human time they consume, not by how much they matter.

30,000 agents, and what actually stops them

The oversight section is the part with the most transferable detail. As of August 2026, Anthropic reports roughly 30,000 agents doing research and engineering work at any one time on its most-used internal platform. Two layers watch them.

Online monitors check agent actions in real time and either block dangerous actions or steer misbehaviour automatically. Anthropic analysed over a billion decisions from its research and engineering agents over August 2026, and found that 0.002% of them — about 1 in 47,000 — were blocked by that monitor.

Offline monitors read agent activity after the fact. That layer "flags roughly one to two transcripts in every thousand for further review," roughly 100,000 transcripts a week by Anthropic's count. Most are resolved automatically by a bank of classifiers checking for false positives, and only the highest-priority flags — "approximately ~50 per week" — reach a human.

A block rate of 1 in 47,000 can be read two ways, and Anthropic does not pick one for you. It could mean the agents rarely try anything dangerous. It could mean the monitor catches a narrow band of things. The company's own caution is that at this scale "it is hard to be certain we have a complete picture of all possible kinds of behaviors," and it argues developers should publish how often *known* misbehaviour gets caught, with third parties running that test independently.

Six percent to safety

Over the week Anthropic examined — 13 to 20 July 2026 — about 6% of the compute that went to AI R&D was allocated toward safety, and about 12% of the compute that went specifically to AI-driven AI R&D went to safety.

Anthropic states the obvious problem with this metric itself, which is to its credit: "Safety research is hard to distinguish from capabilities research, and each developer will be tempted to draw the line generously." It also notes that a compute share measures only what is spent, and says nothing about efficiency or how much work that spending produced. A lab could halve its safety compute by making safety research more efficient and look worse for it.

Self-reported, and so far unmatched

None of these numbers has been audited by anyone outside Anthropic. The company says it intends to change that: "We plan to embed independent third-party evaluators from multiple organizations at Anthropic, and give them access to internal processes, systems, and data comparable to what internal risk assessment teams have." That is a plan, not a completed arrangement, and until it exists the figures are a company's account of itself.

The proposal's weight depends entirely on whether anyone else follows. Anthropic's pitch is that "any frontier developer could publish these measures regularly, using a public methodology." No other lab has published comparable figures, and a measurement regime one company reports on itself is closer to disclosure than to accountability.

It also lands in a genuinely contested argument. Reporting the same day in TechCrunch set out the case that the pacing push serves incumbents — Cohere's CEO is quoted describing major labs as creating "a cartel" to write rules that suit them, with the real dispute being "who writes them, who gets to participate." Publishing your own metrics is a reasonable answer to that charge, or an extension of it, depending on who ends up defining the categories.

If you run agents yourself

There is no prompting change here, but there is a reference architecture, and it is unusually specific for something a frontier lab has put in public. The shape is full coverage at both layers — a real-time monitor with authority to block or redirect, plus an after-the-fact reader over everything — with a cheap classifier funnel in between whose whole job is to keep the human queue small. The ratio is the interesting part: roughly 100,000 automated flags per week compressed down to about 50 items a person actually looks at.

If you are running coding agents at any scale, that is the trade being made explicit. Review capacity is the fixed input, and everything upstream exists to spend it well.

Why this matters

  • It is the first time a frontier lab has put a public number on how much of its own research its models perform, which is the quantity the entire argument about pacing AI development has been missing.
  • The oversight figures describe a working monitoring setup for a 30,000-agent fleet in concrete terms — coverage, flag rates and how much reaches a human — which is rare public detail for anyone building on agents.
  • The proposal only becomes accountability if rivals adopt it and outside evaluators verify it. Right now it is a company reporting on itself, and Anthropic names the ways that can go wrong.

Key takeaways

  • Claude 'leads' 26% of Anthropic's AI R&D work as of August 2026, up from under 1% in February; over 90% of that work is at 'AI collaborates' or above on Epoch AI's automation scale.
  • The index is a prototype built from a 542-node task tree, with a Claude judge model assigning automation levels and results weighted by person-time — so Claude is largely rating Claude.
  • Across over a billion agent decisions in August 2026, real-time monitors blocked 0.002% (about 1 in 47,000); offline monitors flag 1-2 transcripts per thousand, of which about 50 per week reach a human.
  • About 6% of AI R&D compute went to safety in the week of 13-20 July 2026; Anthropic warns that the safety-versus-capabilities line is one every developer is tempted to draw generously.
  • Anthropic says it plans to embed independent third-party evaluators with internal-level access, but that arrangement does not exist yet and no other lab has published comparable measures.

Sources

  1. AnthropicPrimary
    Measurements for understanding the pace of AI development inside frontier labs
    anthropic.com
  2. Dario AmodeiPrimary
    We Must Pace the Frontier
    darioamodei.com
  3. TechCrunch
    Is the AI safety debate about safety or control?
    techcrunch.com
Tags:
  • transparency
  • ai-r-and-d-automation
  • agent-oversight
  • compute-allocation
  • pacing-the-frontier
  • third-party-evaluators
  • automation-levels
Companies:
  • Anthropic
  • Epoch AI
  • Cohere
Models:
  • Claude

Get Promptea Weekly in your inbox

One email every Monday — the best AI stories of the week, verified and summarized.