1. What is in the White House AI audit accord?
On September 29, Nvidia, OpenAI, Anthropic, Google, Meta and xAI signed an accord with the White House covering advanced AI models. Each pledged to work with an independent outside auditor or evaluator, and to have independent board members review the resulting reports and make sure problems get fixed. President Donald Trump described the commitment as "morally binding."
Samuel Vance-Law, head of research at the Decentralization Research Center, argues in an opinion column that the pledge is worth supporting but incomplete. His concern is not the idea of outside scrutiny. It is that the text leaves the most consequential questions unanswered, and that the answers will determine whether the audits tell the public anything useful.
The accord, as he reads it, does not say who picks the auditors, what those auditors are allowed to examine, or whether their findings will ever be made public. It also carries no penalty for a company that fails to comply. In practice, that leaves each signatory free to choose its own reviewer and to define how far the review reaches.
2. Why does auditor independence matter?
Independence matters because an audit is only as credible as the distance between the examiner and the examined. When the party being checked selects the checker, sets the scope and decides what to disclose, the exercise can become a formality that offers reassurance without much evidence. Vance-Law makes the case that this is the central weakness of the accord as written.
He points to a historical precedent from the world of corporate accounting. In 2000, Enron paid its auditor, Arthur Andersen, $25 million for auditing work and a further $27 million for tax and consulting services. Both companies were gone within two years. The lesson he draws is that the collapse did not stem from a shortage of technical skill at the accounting firm.
Instead, he describes it as a failure of incentives. A client relationship worth $52 million in a single year gives an auditor a strong reason to avoid delivering unwelcome conclusions. The expertise to spot trouble may have been present, but the financial dependence discouraged acting on it.
3. What did Congress change after the Enron collapse?
Congress responded with the Sarbanes-Oxley Act, a 2002 law that rewired the relationship between companies and their auditors. According to the column, it took the choice of auditor out of management's hands, restricted the other services that an auditing firm can sell to a company it audits, and created a public board with the power to inspect the auditors themselves.
Those three features address the same problem from different angles. Taking selection away from management weakens the client's leverage. Limiting side business reduces the money that depends on keeping the client happy. A public inspector adds a layer of oversight so that the auditors are themselves examined rather than simply trusted.
Vance-Law notes that the White House accord has no counterpart to any of this. In particular, nothing resembles the public board that checks the checkers. He says Congress should decide how that gap is filled, since the voluntary pledge cannot create such a body on its own.
4. How good is the record of voluntary AI commitments?
The author is not optimistic about voluntary promises, citing the outcome of an earlier effort. In 2023, the White House secured a set of commitments from AI developers that likewise included outside testing. A later study by researchers at Stanford, Harvard and Brown scored 16 companies against those commitments, and the average mark was 53%.
The gaps were uneven. On the narrower question of helping outsiders report problems they find, eight of the 16 companies received a score of zero. For Vance-Law, the result suggests that pledges without enforcement or public detail tend to be honored selectively, with the easiest steps taken first and the more demanding ones left undone.
The pattern also bears on the new accord. If the earlier round of commitments produced an average of just over half marks, there is little reason to expect that a document with no penalties and no disclosure requirements will perform better unless its design is tightened.
5. How much do developers control the testing terms?
Even when a company welcomes outside testers, it usually sets the conditions. The column offers an example from METR, a nonprofit that evaluates AI systems. METR said its pre-release testing of OpenAI's o3 model lasted three weeks, that it received limited information, and that it was not able to see the model's step-by-step reasoning.
Vance-Law is careful to be fair on the point. OpenAI did permit the testing, and the access it granted was real. The example is useful, in his view, precisely because it shows how much of an audit's value depends on what a company chooses to share, from the time window to the level of technical detail.
A reviewer working under a short deadline with partial information can still find problems, but cannot be expected to give assurance about what it was never shown. Without a fixed standard for access, two audits that carry the same label can differ enormously in what they actually establish.
6. Why is sharing a model with testers so hard?
There is a genuine security obstacle behind the limited access. A trained model is essentially a very large file of numbers, and producing one can cost more than $100 million. Handing copies to outside groups creates a risk that the file leaks, which is a serious concern for a company whose competitive position rests on it.
That produces an awkward choice for testers, one that Google DeepMind has described publicly. Either the testers give the company their questions in advance, which lets the company prepare for them and weakens the test, or the company gives the testers the model, which exposes its most valuable asset. Neither option satisfies both sides.
The practical effect is that auditors have often ended up working on the developer's terms. The developer argues it cannot hand over the model, and the testers accept a restricted setup because the alternative is no test at all. The column suggests this trade-off is no longer unavoidable.
7. Can AI models now be audited without being handed over?
In August, Google DeepMind worked with Singapore's AI Safety Institute and three other outside groups on what it called the first double-blind test of an AI model. The setup relied on sealed hardware, so that the testers never saw the model and Google never saw the testers' questions. Google says each side can verify mathematically that its own material stayed private.
The design removes the dilemma described above. The company keeps control of its model, and the evaluators keep control of their test, yet an evaluation still takes place. For the author, this changes what can reasonably be demanded of an audit regime, because the old argument that outsiders cannot be allowed near the model loses much of its force.
He does not present the approach as flawless. It moves the burden of trust onto the hardware, and researchers broke into similar sealed hardware last year using less than $1,000 of equipment. That attack, however, required physical access to the servers. Vance-Law regards the risk as real but smaller than the claim that nobody can be allowed near the model, and he says it can be managed by controlling who has access to the machines.
8. What does the new research report recommend?
Vance-Law and co-author Kyle Bligen published a report this week titled "Alternatives to Concentrated AI." They interviewed 36 people and mapped nearly 30 alternatives to the concentration of control over AI. The main finding is that outsiders can now scrutinize advanced AI systems without possessing the model or its training data.
From that finding, the column draws three recommendations for the accord. First, auditors should be able to choose what they test, which is now feasible without taking copies of the model. Second, companies should name their auditors and describe the scope of each audit, even if the detailed findings remain private.
Third, the board committees that are supposed to review audit reports should answer to someone outside the company. The author observes that publicly traded companies have disclosed their auditors for decades without revealing trade secrets, so asking AI developers to do the same would not be a new burden.
Taken together, the proposals aim to give the accord the features that the post-Enron reforms gave to financial auditing: separation between the examiner and the examined, transparency about who is doing the examining, and a mechanism that holds the examiners to account. Whether lawmakers or the companies themselves adopt any of them is not yet known.
9. What happens next for the AI audit accord?
The accord is in force only as a voluntary commitment, so its practical shape will depend on how the six signatories choose to carry it out. The first visible signals will be which auditors they name, how much of the audit scope they describe, and whether any findings are made available beyond the companies themselves.
The author's view is that the answer should not be left to each company's discretion. He points to Congress as the body that would have to decide how to create an equivalent of the inspector that Sarbanes-Oxley established for financial auditors, since the accord has no such mechanism and no penalty to back it.
For readers following AI policy, the column's value lies in a concrete test: when the companies report on their audits, check whether auditors chose their own tests, whether the auditor and scope are disclosed, and whether anyone outside the company oversees the review. The answers will show whether the pledge amounts to independent scrutiny or to self-assessment with an outside logo attached.