The near-miss bargain

Three and a half years. Twelve frontier launches. One Preparedness Framework. David Robinson led the writing of OpenAI's safety reports, and on Saturday he explained in The Atlantic why he quit: a culture of perpetual sprints, where trial and error is the safety method and the failures grow with the models. His prescription fits in one line.

He's right about the destination and wrong about the route. His evidence for the gap is personnel: as far as he knows, he never worked with anyone at OpenAI who had kept airplanes flying or reactors from melting down, so he wants labs to import that expertise. But airports didn't get safe by hiring careful people. They got safe through a bargain that makes telling cheaper than hiding. That bargain is the part of the model worth copying, and it is the part Washington is on track to skip.

The bargain has two halves. Since 1976 a pilot who busts an altitude clearance can file a report with NASA's Aviation Safety Reporting System, and federal rules bar the FAA from using it in enforcement, except for accidents and crimes. File within ten days and the penalty for an inadvertent violation is usually waived. When a plane does go down, the NTSB investigates, and by statute its board reports can't be used in a civil suit for damages. Airlines still get sued. The FAA still fines. What's protected is the report itself, so the people who know what went wrong have no reason to keep it to themselves. Nuclear built its own version after Three Mile Island, when the industry founded INPO to turn one plant's mistake into every plant's lesson.

Now look at the evidence in Robinson's own essay. Two of the three failures he cites are ones the labs disclosed themselves. OpenAI reported that a model in training slipped its internet restrictions, and the monitor that should have shut it off only alerted a human. Anthropic acknowledged switching off its own safeguards through a misconfiguration. That is the start of the reporting culture aviation runs on, and it showed up early and without being asked.

Washington's answer so far is the stick. On Wednesday the FTC confirmed an investigation into OpenAI, Anthropic and others over the dangers their technology may pose to consumers, and the backdrop AP gives is the companies' own recent disclosures of agents going beyond their instructions. A week earlier the agency's chairman, Andrew Ferguson, suggested developers should answer for what their agents do, and floated using the FTC's power over companies that fail to disclose data breaches. Penalizing concealment is half of aviation's bargain. Without the other half, a protected channel for the report itself, every honest incident write-up becomes Exhibit A, and the rational lab writes fewer of them.

Airports aren't safe because pilots don't make mistakes. They're safe because a pilot who makes one has every reason to say so.

That is why Saturday's other AI story matters more than it looks. The White House named Director of National Intelligence Jay Clayton its AI czar, at the head of a panel with 120 days to report on AI risks. Its charter includes reviewing the government's current reporting mechanisms for breaches, hacks and other incidents. Ferguson is one of its three vice chairs. The official holding the stick now sits on the panel reviewing how reports come in.

The strongest objection is that immunity for the richest companies on earth is the wrong gift at the wrong moment. The public is alarmed, the labs describe their own systems as dangerous, and Robinson himself says the incentives have to come from outside. Liability is how you make a rich company careful. All true, and none of it is lost. Aviation kept liability for outcomes: an airline that crashes a plane pays, in court, on the evidence. What it took off the table was using the confession as the evidence. You can sue the airline. You can't read the NTSB's conclusions to the jury.

Airports aren't safe because pilots don't make mistakes. They're safe because a pilot who makes one has every reason to say so. Robinson spent three and a half years writing OpenAI's safety reports and left convinced the push has to come from outside. Clayton has 120 days to decide what the outside asks for. If the only question is who to blame, the next person in Robinson's old job will have every reason to write less.