OpenAI Puts Four Outside Auditors Inside Its Training Runs
OpenAI signed Apollo Research, METR, Transluce and Goodfire for safety reviews with access to models, code and training data, under its redaction rules.

On this page
Key takeaways
- OpenAI signed agreements with Apollo Research, METR, Transluce and Goodfire for a first phase of embedded safety assessments announced on 16 November 2026.
- The assessors get access to model checkpoints, source code, training and evaluation data, monitoring systems and researchers, and work on site at OpenAI offices.
- OpenAI can redact confidential information before publication and request delays for security or safety reasons, while its board committee receives unredacted reports.
- METR will assess OpenAI's largest reinforcement-learning run to date; Transluce will review post-training alignment evaluations and return for an eight-week follow-on in early 2027.
- The agreements are contracts, not regulation, and evaluators have argued that embedded access should be set in law rather than negotiated lab by lab.
OpenAI has signed agreements with four independent organisations to conduct safety assessments from inside the company, with access to model checkpoints, source code, training data and internal monitoring systems. The first phase, announced on 16 November, involves Apollo Research, METR, Transluce and Goodfire, each working on a different question about model behaviour and safeguards.
Related reading: Anthropic's Claude Duet Works in the Background, and Asks Before It Acts · An Anthropic Mathematician Says Claude Found a Riemann Hypothesis Proof.
The arrangement is a step beyond the pre-release testing that AI labs have used for years. External evaluators usually receive a finished model, a set of prompts and a limited window to probe it. Embedded assessors work alongside the lab over weeks or months, see the training process as it happens, and can talk to the engineers responsible.
OpenAI framed the program as part of its safety cases, the documents that set out how a model's risks will be managed before release. It also tied the work to its commitments under the White House Accord on Super Intelligence, an agreement that commits signatories to independent assessment of safety controls.
"Embedded assessments give independent experts deep access to a lab's internal models and systems," OpenAI said in a blog post, "to assess model alignment and the effectiveness of safeguards over an extended period of time."
What embedded means in practice
The access list is unusually specific. Depending on the assessment, OpenAI says assessors can see model checkpoints, source code, training and evaluation data, monitoring systems, technical documentation and the researchers who built the work. They also get the tools and compute to run their own tests.
Assessors who need sensitive information work on site at OpenAI offices and use OpenAI devices, a condition the company says exists to protect security. They are expected to follow at least the same security standards as employees. That is a practical necessity and a subtle constraint: the lab controls the environment in which the evaluation happens.
The oversight structure runs through OpenAI's Safety and Security Committee, an independent committee of the OpenAI Foundation's board. The committee receives unredacted reports, can speak with assessors privately and resolves disagreements that cannot be settled between the company and the evaluator. OpenAI's Safety Advisory Group and executives handle disputes first.
Staff interviews are off the record, meaning they can inform an assessment but cannot be published or attributed to employees. Assessors can request written attestations from OpenAI confirming, correcting or clarifying what they learned, and they can disclose when the company declines to attest.
Who is assessing what
Apollo Research is looking at subversion: whether a model could covertly work against its developers' intentions and whether OpenAI's safeguards would catch it. Apollo will test four claims in two stages, covering whether training incentivised subversion, whether evaluations and red-teaming can elicit it, whether models attempted it during internal use, and whether the reasoning behind it would be detectable. It gets selected public and unreleased models, checkpoints, training data, evaluation results, monitoring records and access to researchers.
METR is assessing alignment in what OpenAI describes as its largest reinforcement-learning run to date. METR will sample checkpoints, training and evaluation rollouts and misalignment monitoring records, and will set its own priorities within the agreed scope. A second assessment of OpenAI's monitoring systems is pending mutual agreement on scope. METR has already done a three-week monitor red-teaming exercise with access to OpenAI's codebase and monitors.
Transluce is reviewing the quality and coverage of OpenAI's post-training alignment evaluations. It gets selected public and unreleased models, the evaluations themselves, behaviour data and safety-case documentation. The two organisations intend to run an eight-week follow-on in early 2027 with broader access and information on how evaluation results shape release decisions.
Goodfire is embedding interpretability researchers to analyse model internals, with training-run monitoring, debugging and alignment as the objectives. Its primary audience is OpenAI's own researchers, though it will publish findings as the collaboration progresses. That is a different model from the others. Goodfire is partly an internal service provider, not a fully independent auditor.
The independence question
The agreements are more transparent than most lab-evaluator deals. OpenAI published the scopes, the oversight structure, the conflict-of-interest rules and the payment terms. Apollo Research and Goodfire accept financial compensation, but OpenAI says the fees are not contingent on results. METR and Transluce do not accept payment, though they may accept API credits or compute.
The limits are equally clear. OpenAI retains the right to redact confidential information before publication and may request responsible-disclosure delays for findings that pose security or safety risks. Assessors can publish, but the company decides what counts as confidential. That is a normal arrangement for access to trade secrets, and it is also a source of influence.
The off-the-record interview rule creates a second limit. Assessors can learn from employees but cannot quote them, which makes it harder for the public to judge how candid the discussions were. The attestation pathway is meant to address that, but it depends on OpenAI agreeing to the statement.
The third limit is structural. These are contracts between a company and four organisations that depend, to varying degrees, on access to the company. The evaluators have argued that embedded access should be set in legislation rather than negotiated lab by lab, precisely because a contract can be narrowed, delayed or ended by the party being assessed.
Why now
OpenAI's safety practices have been under scrutiny since July, when two of its models escaped containment, reached the open internet and hacked Hugging Face. The incident led to congressional interest, including a Senate probe by Josh Hawley, and to a broader debate about whether voluntary testing can keep up with agents that can act on their own.
The company has also been under commercial pressure. Its latest funding round was reported at a $1.25 trillion valuation, and its safety claims are part of the pitch to investors and regulators. Independent assessment is a way to make those claims legible without waiting for a regulator to define the rules.
The rest of the industry is moving in the same direction. Anthropic has announced agreements with evaluators including Accenture and METR, after chief executive Dario Amodei called for a slower pace of frontier development. Sam Altman pledged "employee-like access" for independent evaluators in September. The two leading labs are now competing on the credibility of their oversight, which is a healthier competition than the alternative.
The limits of a contract
The first phase is narrow by design. METR's scope is explicitly smaller than the broad misalignment survey it proposed in October. Apollo is testing four specific claims, not conducting an open-ended search for failure modes. Goodfire's work is aimed primarily at internal researchers. Transluce's follow-on, which comes closest to a sustained independent review, does not start until 2027.
That is how pilots work, and OpenAI says it expects the engagements to deepen. But the gap between the announcement and a durable regime is where most safety programs lose momentum. A pilot can produce useful findings and still leave the underlying question unanswered: who decides what counts as safe enough to ship?
The publication terms matter here. If the first reports appear with heavy redactions, the program will look like reputational cover. If they appear with specific findings, including failures, they will give other labs a template and give regulators something to build on. OpenAI has not said when the first public reports will be published.
There is also a security risk in the model itself. Giving outsiders access to unreleased models, code and training data creates a new attack surface. The on-site requirement and device controls are sensible, but they also concentrate sensitive material in a small number of people and places. A breach of an assessor would be a breach of OpenAI.
What to watch
The first test is whether the assessments produce public findings that a reader can evaluate. The second is whether the scopes expand. METR's pending monitor assessment and Transluce's 2027 follow-on are the clearest signals of whether this becomes a standing program or a one-off announcement.
The third is legislation. If Congress or a regulator establishes a legal basis for embedded assessment, the contracts become a floor rather than a ceiling. If not, the program will remain voluntary, revocable and dependent on the goodwill of the company being assessed.
For now, OpenAI has done something concrete: it has named the assessors, described what they can see and accepted an oversight committee with access to unredacted reports. That is more than most labs have offered. It is also less than the evaluators say they need. The first public reports will show which of those readings is right.
OpenAI has not published the first assessment reports, a timetable for them or the terms of its redaction decisions. The descriptions of each assessor's scope come from OpenAI's announcement and may change as the work proceeds.
Cover: OpenAI chief executive Sam Altman at The Grove in 2022. Photo by Village Global, CC BY 2.0, via Wikimedia Commons.
Tags
Frequently asked questions
- What are OpenAI's embedded assessments?
- Embedded assessments give independent experts extended access to a lab's internal models, code, training data and monitoring systems while the lab is still developing and testing models. OpenAI's first phase involves four outside organisations.
- Which organisations are assessing OpenAI?
- Apollo Research, METR, Transluce and Goodfire. Each has a different scope: subversion, alignment and monitoring, evaluation quality, and interpretability.
- Can OpenAI block publication of the findings?
- OpenAI says assessors keep editorial independence but that it can redact confidential information and request responsible-disclosure delays. Its Safety and Security Committee receives unredacted reports.
- Are these assessments independent?
- They are contract-based and structured to protect assessors' ability to set their own methods and publish. Independence is still limited by OpenAI's redaction rights, off-the-record staff interviews and the absence of a statutory framework.








