AI is now building AI: Anthropic says Claude leads 26% of its R&D work

The share of artificial intelligence (AI) doing work to develop itself is surging. On Thursday, Anthropic said its AI model Claude now leads 26 per cent of the company's AI research and development (R&D) work, up from less than 1 per cent in February.

 

More than 90 per cent of Anthropic’s AI R&D work is now performed at a level where AI either collaborates with humans or leads the work, the company said in a blog post. However, Claude is not yet operating fully autonomously in any measured area of its R&D.

 

How much of AI R&D is now being done by Claude?

 

Anthropic said it has created the Anthropic R&D Automation Index to measure how much of its AI research and development is performed by Claude. The index rates different R&D tasks using an automation scale developed by Epoch AI that measures “Automation Level”, or AL, ranging from AL0, where there is no AI involvement, to AL5, where AI operates fully autonomously without a human in the loop.

 

 

At AL3, AI “collaborates,” meaning it can do large chunks of work under close human direction. At AL4, or the “leads” level, AI can complete most of the task end-to-end from a high-level prompt, while the human supervises.

 

As of August, Claude had reached the AL4 level for 26 per cent of Anthropic’s AI R&D work. The share of work at or above the “AI collaborates” level was more than 90 per cent.

 

Anthropic said the growing role of AI in developing future AI systems could help labs build more capable models faster, but could also make it harder for humans to understand or control increasingly capable systems. The company said the measurements are intended to show how close the industry is to recursive self-improvement, where a model fully autonomously builds its successor.

 

30,000 AI agents doing R&D at Anthropic

 

Anthropic also disclosed that around 30,000 AI agents were carrying out research and engineering work on its most-used internal platform at any given time, as of August.

 

The company said 100 per cent of these agents’ actions pass through an online monitor before execution, usually within seconds, while their activities are also ingested by an offline monitoring system after the actions are taken.

 

Anthropic analysed more than 1 billion agent decisions from August and found that 0.002 per cent, or about one in 47,000, were blocked by its online monitor. The system also flags roughly 100,000 transcripts a week for further review, with about 50 of the highest-priority cases escalated to humans.

 

How much does Anthropic spend on AI safety?

 

Around 6 per cent of the computing power used for AI R&D was allocated to safety work during a sample week from July 13 to July 20, Anthropic said, adding that for AI-driven AI R&D, the figure was about 12 per cent.

 

The company cautioned that compute is an imperfect measure of how much a lab focuses on safety, since safety research can be time-intensive without requiring as much computing power as frontier model training.

 

Anthropic said it plans to continue publishing these measurements to reduce the gap between what frontier AI labs know about their development processes and what the public knows. It also plans to bring in independent third-party evaluators to verify safety practices, report incidents, and monitor key metrics.