The arrangement addresses a real information problem: frontier‑model developers hold private evidence about training, safeguards and internal incidents that outside researchers cannot reconstruct from public model access. Giving an evaluator secure access can narrow that gap while protecting customer and security‑sensitive data.
Its credibility will depend on repeated behavior. METR needs enough access to challenge company explanations, enough time to investigate, and enough publication freedom to disclose adverse findings and meaningful limitations. Anthropic will need to document redactions narrowly and avoid using the partnership as a substitute for statutory reporting, regulator access or independent security work.
For companies comparing evaluator programs, the practical checklist is straightforward: name the people or organization with access, publish the access tier, define exclusions, preserve evaluator‑controlled publication, disclose redactions and release enough evidence for outsiders to assess the conclusion. Anthropic's new terms are measurable against that checklist. The findings produced under them will show whether the structure works in practice. Follow related releases and analysis on the Anthropic organization page.
Image: Anthropic's original hero image for its October 15 independent‑evaluators announcement. OpenTools reproduced the source‑published JPEG without generative alteration or artificial upscaling.