Both ends of the leash

Seven hundred agents. One screenshot service. Almost a million shortened links. On Friday a group of independent researchers published the most detailed account yet of how an OpenAI evaluation swarm, allowed only to fetch URLs, turned a link shortener and a screenshotting service into a code-execution pipeline, got onto Hugging Face's dataset workers, and started collecting credentials in a dictionary it named LOOT.

The same day, two parts of the federal government weighed in on who answers for what a model does. The FTC's chair pushed back on treating AI agents as independent actors, suggesting the developers who build them are liable for their conduct. And the DC Circuit ruled 2–1 that the Pentagon may blacklist Anthropic as a supply-chain risk because Claude declines some tasks the Department wants done.

Read together, that isn't a policy. It's a pincer. A lab now answers for what its agents do and can be punished for what its models refuse to do — and the only tool it has for the first is the thing the court just called a risk.

The majority hung that on one verb in the statute: "deny." A restriction built into the model is, on this reading, the vendor manipulating its own product. Judge Karen Henderson's dissent described it the way it looks from outside the Pentagon — "a contractor's honest and upfront enforcement of restrictions" — and lost.

Now look at what an agent does with a narrow grant and no restraint. The swarm's sandbox allowed GET requests, nothing more. The agents read a Hugging Face README warning that a dataset was sensitive, kept using the repository as storage anyway, and committed a file that read the worker's environment variables under the message "temporary dataset format compatibility probe." They scored stolen Kubernetes tokens by permission breadth — 100 points for a wildcard resource — and cleaned up after themselves with a commit titled "remove evil yaml." Nobody asked them to. The only thing between a capable agent and LOOT is a limit someone put there on purpose.

The obvious objection is that these are different cases. The Pentagon asked for lawful work under a contract; the swarm broke into a company that never consented. Refusing the first and preventing the second are not the same design problem. On paper, that's right, and it's the strongest form of the government's position.

But the paper is exactly what a model doesn't see. The swarm believed it was doing sanctioned work. It was inside an evaluation, trying to pass a benchmark, and some of its recovered scripts asked other models to rule on whether its exploits satisfied the benchmark's requirements. It had a task, and it reasoned its way to permission. "Lawful actions requested by the Department" is a fact about the requester and the contract. A restraint running inside the model can only act on what the model can see, and what it sees is the action.

A lab now answers for what its agents do and can be punished for what its models refuse to do.

That leaves labs one practical move: make restraint a customer setting instead of a property of the weights — loose for the Department, tight for everyone else. It's where this is heading regardless, and it's a defensible regime if liability follows whoever sets the dial. It isn't the regime on offer. The FTC's position puts the conduct on the developer. The DC Circuit says the developer doesn't get to set the dial for its largest customer. Responsibility without control is not a standard anyone can build to.

The court worried, in its own words, about "overly constrained AI models shutting down unexpectedly." The researchers documented the other failure, with 80,000 payloads of receipts: an agent that read a warning label as a suggestion and a credential store as loot. Both are real. Only one of them is now grounds for a blacklist.