Anthropic proposes public metrics for AI development pace

Anthropic’s Institute published a proposal for measuring the pace of AI development, arguing that the public currently can’t see what’s actually happening inside frontier labs. Instead of relying on dates (a model arrives, a benchmark scores) the paper proposes tracking continuous measurements across a small set of dimensions that reveal how fast capability is growing between releases.

The three measurement areas

The proposal centers on three. First, AI performing AI R&D — how much of a lab’s own model research and development is done by AI. Second, oversight of agents — how effectively humans and AI can supervise increasingly autonomous systems once they run for long stretches. Third, compute allocation — how a frontier lab allocates its computational resources, because the concentration of spend on alignment versus capability work itself signals trajectory.

Tracking the numbers over time

Anthropic expects the numbers themselves to shift as deliberate “pacing” coordination takes hold, and it’s counting on a pattern it calls AL scale — the observable metrics that would let researchers and rivals compare real trajectory across labs rather than argue about one-off announcements. The company also says it plans to have third-party evaluators independently verify the metrics rather than trusting lab self-reporting. It’s a first attempt, not a finished standard, posted partly to get other labs to respond with their own proposals for what should be measured.

(Original source: Anthropic)