The AI Company That Told the Pentagon No
We like to believe the law is what keeps mass surveillance off your back. This is the story of the Friday a single company refused to delete a clause — and the price it paid for that “no.”
I should own up to something, because it shapes how I came to this story. I am, by temperament, suspicious of Big Tech. When a trillion-dollar company tells me it is protecting my rights, my hand instinctively moves to check my wallet. I have spent years arguing that we should never outsource our liberties to a corporation‘s mood, a CEO‘s conscience, or a quarterly earnings call. So believe me when I say I did not expect to spend a week defending an AI lab against my own government — and finding, to my discomfort, that the lab had the better of the argument.
Here is the part that unsettled me most. The thing standing between one of the most powerful AI models on Earth and a domestic-surveillance contract turned out not to be a statute, not a constitutional guarantee, not a vote in Congress. It was a clause. A few sentences in a contract that one company refused to cross out. And when it refused, the United States government did not shrug and move on. It reached for a label normally reserved for hostile foreign powers and pinned it on an American company.
First, the part nobody really disputes
Let‘s lay down the facts before we argue about what they mean.
In February 2026, the AI firm Anthropic — the maker of the Claude models — refused two military use cases: mass domestic surveillance, and fully autonomous weapons. Its stated reason was not pacifism. It was reliability: in the company‘s own words, “frontier AI systems are simply not reliable enough to power fully autonomous weapons.“ The CEO, Dario Amodei, described these as “bright red lines” — Claude must not be used for mass surveillance of Americans, or to power “a drone that uses AI to kill targets without human approval.“
The Pentagon saw it differently. Its position, relayed to NPR, was crisp and not unreasonable: “Legality is the Pentagon’s responsibility as the end user,” and AI vendors must let the government use their tools “for all lawful purposes.“ Translation: We’ll decide what’s legal. You just supply the tool. The Department gave Anthropic a deadline — 5:01 p.m. on Friday, February 27 — to drop the restrictions and allow “all lawful purposes.“ When the talks collapsed, the consequences arrived fast. President Trump ordered every federal agency to “immediately“ stop using Anthropic, and Defense Secretary Pete Hegseth deemed the firm a “supply chain risk to national security” on Friday — ending a contract estimated at roughly $200 million.
Sit with the mechanics of that for a moment, because they matter more than any single quote. Anthropic‘s safeguards were not handed down by a court or written into federal law. They were contractual — limits the company had chosen to place on its own product and declined to delete. Strip those clauses away, and nothing in the paperwork would have stood between Claude and whatever “lawful purpose“ an agency later decided to pursue. That is the uncomfortable revelation at the center of this whole affair: a meaningful chunk of your privacy protection, in this instance, lived in a vendor‘s contract — not in your rights.
So the government was just being prudent. Right?
Let me make the other side‘s case as strongly as I can, because the people making it are not cartoon villains, and the argument has real force.
The Pentagon‘s chief technology officer, Emil Michael, told CBS News the military “made some very good concessions” trying to close the deal. By his account, the Department offered to put in writing the laws that already bar surveillance of Americans, reaffirm existing autonomous-weapons policy, and even invite Anthropic onto its AI ethics board. When the company called that inadequate, Michael‘s patience snapped — he called Amodei a “liar“ with a “God-complex“ — and landed on a line that deserves a fair hearing: “At some level, you have to trust your military to do the right thing.“
He has a point, and it is not a small one. The U.S. military operates under law. There are statutes against warrantless surveillance of Americans; there is policy requiring a human in the loop on lethal force. From inside that system, Anthropic‘s stance can look like a private company appointing itself a fourth branch of government — deciding which lawful uses the elected government may pursue. Why should a corporation in San Francisco get a veto over national defense?
If the question were “should an unaccountable tech CEO get to override duly enacted law?”, my answer would not be a comfortable of course he should.
But here is where that framing quietly slides off the actual facts. The fight was never over whether mass surveillance is legal — the Pentagon insists it has no such plan, and I‘ll take that at face value. The fight was over whether the limit was binding. Anthropic‘s demand, in plain terms, was for guardrails that “explicitly bar” Claude from being used to “conduct mass surveillance of Americans or carry out military operations on its own.“ The government wanted those words gone. And “trust your military to do the right thing“ is precisely the assurance that a contract clause exists not to require — because trust is the thing you ask for when the enforceable rule is the thing you‘re trying to remove.
We have watched a company say “no” before
Here is where it helps to step out of this story and into an older one, because a private company refusing a government demand on principle is not new — and the precedents are instructive.
Rewind to 2016. After the San Bernardino shooting, the FBI obtained a court order and demanded that Apple write software to unlock the gunman‘s iPhone. Tim Cook refused — publicly, defiantly — warning that the order had “implications far beyond the legal case at hand,“ because it would create “a precedent that the U.S. government could use to force any technology company to create software that could undermine the security of its products.“ The standoff never got its courtroom climax; the FBI quietly paid a third party to crack the phone and withdrew the case. Apple had said no, and the law, in the end, let the no stand.
Go back three more years, to 2013, and the answer was darker. When the government served Lavabit — the secure email provider Edward Snowden happened to use — with a warrant for its master encryption keys, owner Ladar Levison faced a brutal choice. Those keys would have exposed not one target but everyone: the key would have unlocked communications for all 400,000 customers of the service. Rather than hand over a skeleton key to every inbox he hosted, Levison shut the company down. He chose oblivion over betrayal.
I want to be careful here, because the analogy can be overstretched. Apple and Lavabit were fights over encryption — demands to break a lock or surrender a key. Anthropic‘s fight was over usage safeguards — a demand to delete contractual limits on how a tool may be deployed. Different mechanisms entirely. But the shape is the same, and the shape is the point: in each case, a single company‘s willingness to say “no“ was the load-bearing wall protecting a lot of ordinary people who never knew their privacy was leaning on it. Three companies. Three lonely refusals. That is not a system of rights. That is a system of luck.
Now imagine the version where nobody refuses
Play it forward — not in a paranoid register, but in the boring, plausible way these things actually unfold.
It‘s two years from now. The lesson of this episode has fully metabolized through the industry, and the lesson is not subtle: the company that drew a red line got branded an adversary, lost a nine-figure contract, and spent a year in federal court. The company next in line watched all of it. So when the next agency arrives with the next request to “allow all lawful purposes,“ the negotiation is shorter. There is no clause to delete, because the careful vendor learned not to write one. The safeguard you‘d have leaned on was quietly never built.
Now picture the downstream. A model wired into a government data pipeline doesn‘t announce itself. You won‘t get a notification that the chatbot helping a clerk “find patterns“ in a dataset is the same class of system another office is pointing at license-plate readers, social-media feeds, and travel records. The surveillance that worries civil libertarians is rarely a jackboot at the door; it‘s a default setting in a procurement contract, toggled in a room you‘ll never see. The whole fight we just watched — loud, public, litigated — happened only because one company forced it into the open. The terrifying version is the one that stays quiet, because the next Anthropic decided that drawing a red line wasn‘t worth the punishment.
That is the future the free-market R Street Institute is warning about — and it‘s notable that a right-of-center think tank is the one ringing the bell. Anthropic‘s two red lines, R Street argues, “were not radical positions.“ The danger is the signal sent to everyone else: that “safety constraints on AI usage are negotiable under sufficient pressure,” and that companies which decline to negotiate “face existential legal and commercial consequences.“ Punish the one firm that said no, and you teach a thousand others to never say it.
What the smart people are saying
This is the kind of issue where you can usually predict the team jerseys — and this time the prediction fails, which is exactly why it‘s worth your attention.
Start on the civil-liberties left. The Electronic Frontier Foundation put its finger on the deepest nerve in the whole story. The problem, EFF wrote, is that “the state of your privacy is being decided by contract negotiations between giant tech companies and the U.S. government.” Yes, it‘s good when a CEO does the right thing — “but it‘s not a sustainable or reliable solution to build our rights on.“ That line has been rattling around my head all week. We got lucky this time. A coin came up heads. But you cannot build a free society on a coin.
Now cross the aisle, and you find R Street making a structurally identical argument from free-market premises: that a government punishing a company for its safety choices corrodes the whole innovation ecosystem. When EFF and a free-market institute independently arrive at the same worry, the partisan read is probably the wrong read.
The oversight branch noticed too. In a letter to the Defense Secretary, Senator Elizabeth Warren pressed the case bluntly: after Anthropic refused to drop the two narrow limits — one against mass domestic surveillance, one against lethal autonomous weapons — the Department designated it a “supply chain risk,“ “a label historically reserved for foreign adversaries such as Huawei,” in what she called an unprecedented abuse of national-security authority.
And the courts? Here I have to be scrupulous, because the litigation is live and I am not going to tell you how it ends. A federal judge in California blocked the designation, calling it “classic illegal First Amendment retaliation“ and warning against “the Orwellian notion that an American company may be branded a potential adversary and saboteur of the U.S.“ On appeal, a panel of three circuit judges heard the case in May, with Judge Henderson remarking that the Department‘s action seemed like a “spectacular overreach.“ But do not mistake skeptical questions for a verdict — and do not assume Anthropic is cruising. Weeks earlier, the same appeals court had denied Anthropic’s request to pause the blacklisting, finding that “the equitable balance here cuts in favor of the government.“ As of now, a court is weighing this — nobody has won. The months-long dispute, and the termination of a contract worth an estimated $200 million, is still unresolved.
What does this mean for you?
You‘re not an AI company, and you‘re probably not negotiating defense contracts. So why should any of this land on your kitchen table? A few reasons, concrete and close to home.
Notice where your privacy is actually stored. Not in the Constitution this week — in a clause, in a contract, between two parties you didn’t elect or hire. When the protection is that contingent, the right response isn’t to relax because it held. It’s to demand that the rule live somewhere a single company’s nerve can’t be the only thing holding the line.
Watch how “for all lawful purposes” is used. It sounds like a guarantee. It functions as the opposite — it pushes every limit downstream onto “whatever turns out to be legal later,” which is a moving target set by the same officials doing the asking. When you hear it, ask: lawful as decided by whom, and enforced how?
Don’t confuse a company’s caution with a company’s virtue. Anthropic may be acting on principle, or on brand strategy, or both — it doesn’t ultimately matter. The lesson isn’t “trust this company.” It’s that no company should be the firewall. Support the boring institutional fixes — congressional oversight, enforceable statutory limits, real procurement transparency — that make refusals unnecessary.
Make noise when a company gets punished for restraint, even one you dislike. I don’t have to admire a corporation to defend its right to say no without being branded an enemy of the state. The precedent set by punishing that refusal will be applied to the next one — possibly to a company, a cause, or a right you care about a great deal more.
The line, and who gets to draw it
I came into this distrustful of the AI lab, and I‘ll leave it distrustful of the AI lab — that hasn‘t changed, and it shouldn‘t. What changed is where I now think the real danger sits. It is not that one company drew a line the government didn‘t like. It is that the line existed at all only because that company chose to draw it — and that when it did, the answer was punishment rather than legislation.
EFF said it most cleanly: our rights should not depend on the decisions of a few powerful people. Right now, in this corner of the AI age, they quietly do. A handful of CEOs and their contract lawyers are functioning as the load-bearing wall between you and a surveillance apparatus that the law has not clearly forbidden. Sometimes that wall holds, the way Apple‘s held in 2016. Sometimes the person behind it pulls the whole building down rather than betray you, the way Lavabit did. And sometimes — this is the part that should keep us up at night — the wall was simply never built, because the last company that tried got made an example of.
The fix isn‘t to canonize Anthropic. The fix is to make heroism unnecessary — to write the protections into law, so that the next time an agency asks a company to delete the clause, the company can answer that it isn‘t theirs to delete. Until we do that, your privacy is one corporate change of heart away from being negotiable. We got a “no“ this time. We should build a world that doesn‘t have to hope for the next one.
The HAIA Foundation exists because the rules governing intelligent machines are being written right now, often in rooms you’ll never enter — and someone should be watching on your behalf. If this shifted how you think about where your rights actually live, come keep watch with us: read more at the HAIA Foundation and subscribe over on our Substack.