You have done this, probably more than once: called a government office or typed a question into a government website, taken the answer at its word, and planned around it. Who would know better than the government?
Social Security's own rulebook has a worked example of how that can go wrong. You are 60 and widowed, and you ask your Social Security office about widow's benefits. The employee you reach says you must be 62. That is wrong, and because of it you don't apply.
What happens when you find out? Here the law has an answer: because the bad information came from a Social Security employee, Congress lets the agency treat you as having applied when you were misinformed (if you can prove it to the agency's satisfaction, which is harder than it sounds).
Now ask somewhere new. On September 29, 2026, President Trump signed an executive order creating America.gov, a federal website built around an AI chatbot, which went live that day. The order says it will be a place where you can "receive accurate answers."
What happens if one of those answers is wrong? As I read the law, the answer is the one the Supreme Court gave a former Navy welder in 1990.
A front door, a promise and a sentence most orders carry
So what does the order actually do?
Executive Order 14432, published in the Federal Register on October 2, 2026, names America.gov the "unified digital front door" to the federal government. The General Services Administration must "establish and operate America.gov," working with the White House's National Design Studio, and GSA says the site draws on "more than 29,000 government websites." Among its stated policies: "ensure that super intelligence used in connection with America.gov is accurate, reliable, and transparent." (Super intelligence is now the executive branch's official term for AI, under a second order signed the same day.)
At launch, the site answered questions. The White House says you can already "get an up-to-date answer," with passport renewal and Medicare enrollment to follow later in 2026. The next phase, which GovExec reports is "set for early 2027," would let you apply for benefits such as Social Security inside the chat. The order gives the Office of Management and Budget 90 days, until December 28, 2026, to tell agencies how to plug in; as of October 3, 2026, OMB's list of memoranda showed nothing on America.gov.
"So there's no chance of getting outdated or inaccurate information like you might on third-party websites," Joe Gebbia, the U.S. Chief Design Officer, told CNBC.
Its closing section holds a sentence that can sound like a trapdoor: the order does not "create any right or benefit, substantive or procedural, enforceable at law or in equity by any party against the United States." The renaming order signed beside it carries the same sentence, word for word, and a Yale Law Journal note describes it as the disclaimer "the typical modern executive order" makes. It is boilerplate. Delete it and, as I read the law, your position would be the same, because the rule that decides what happens when a government answer is wrong comes from the Supreme Court.
The welder, the out-of-date form and the rule that still decides
Why should a 1990 case about a welder decide anything about a chatbot? Because it answered the same question, about a government form.
Richmond had been a welder at the Navy Public Works Center in San Diego. Relying on a government form (and on oral advice a dissenting justice called "similarly erroneous"), he earned more than the statute permitted and "lost six months of benefits." The form, the Court said, "was correct when written in 1981; but when given to respondent, the form was out of date and therefore inaccurate." Set that beside Gebbia's "no chance of getting outdated or inaccurate information": thirty-six years apart, the same risk.
The Court ruled against him anyway. "We hold," Justice Kennedy wrote for the majority in Office of Personnel Management v. Richmond, "that payments of money from the Federal Treasury are limited to those authorized by statute," because "Money may be paid out only through an appropriation made by law." The Court left open "whether an estoppel claim could ever succeed against the Government" (estoppel means holding a party to what it told you, once you relied on it), but for money the answer was no: a wrong answer cannot make the Treasury pay a benefit the law doesn't allow.
The opinion also says where the fix belongs: "Congress may always exercise its power to expand recoveries for those who rely on mistaken advice." For Social Security, Congress has, narrowly. If the agency is satisfied that you "failed as of any date to apply" because of misinformation from "any officer or employee of the Social Security Administration," you are treated as having applied when you were misinformed (or when you first qualified, if that came later). That is the whole remedy: an earlier filing date, for people who didn't apply.
The agency's regulation narrows it further. The misinformation must come from "one of our employees while he or she was acting in his or her official capacity as our employee," and the agency "will not find that we gave you misinformation, however, based solely on your statements." Among the things it says do not count as proof: "Information provided by other governmental agencies."
America.gov is operated by GSA, not Social Security. So, as I read these rules, its chatbot's answers fall outside them: they come from no Social Security officer or employee, and they are information from another agency. That is my reading, not a holding; I know of no court that has ruled on a chatbot's answer under any of this. I would not want to be the widow who tests it.
Isn't some advice better than none?
The Supreme Court thought so, and the argument deserves a fair hearing. "A rule of estoppel might create not more reliable advice, but less advice," Justice Kennedy wrote, and the loss would fall hardest "on those of limited means, who can least afford the alternative of private advice." It still bites — a chatbot whose every sentence could bind the Treasury is a chatbot no agency lawyer would let anyone switch on.
The administration's case is strong, too. "Americans spend around 10 billion hours each year dealing with federal paperwork," says the White House's fact sheet, and the president promised "one front door for every single question" without bricking up the old ones — the order keeps phone, mail and in-person service. The site says it "never includes ads, and keeps your privacy protected," and its privacy page, GovExec reported, says it doesn't save conversations. And according to the New York Post, it warns users that "AI can make mistakes" and tells them to "check the sources included with each answer before making an important decision."
So far so good. Then what's wrong with the bargain?
When CBS News's medical correspondent, Dr. Céline Gounder, tested it at launch (the answers may have changed since), it handled emergencies well: chest pain and a numb left arm drew "Call 9-1-1 now." On insulin, it said "most Part D plans cap insulin at $35 for a month's supply (participating plans; check your formulary)." "Participating plans" belongs to a voluntary 2021 pilot; a 2022 law extended the cap to every Part D plan, and Medicare's own page says "the $35 limit applies to everyone who takes insulin." The chatbot, she wrote, "described the pilot and left out the law that replaced it." It was Richmond's out-of-date form — reissued by a machine.
And the answers move. While the president was still speaking at the launch, the Associated Press reported, "the site began modifying its answers"; a White House official's response to CBS was that America.gov "will continue to be updated." An answer that can change within hours is one you will struggle to prove you were given.
Two justices dissented in Richmond. "In this case," Justice Marshall wrote, "we must decide who should bear the burden of the Government's error." The majority's answer was the claimant, unless Congress provides otherwise.
What would a slower launch have looked like?
Britain has run one. Its Government Digital Service tested a first version of GOV.UK Chat, an AI assistant for the government's main website, in 2023, and said publicly the next year that its answers "did not reach the highest level of accuracy demanded for a site like GOV.UK."
Then came two public pilots over 18 months, more than 10,000 users and a published scorecard: accuracy up "from 76%, our earliest benchmark, to our latest figure of 90% accuracy across all topics." To be clear, that is GDS's own score on its own benchmark, not an independent audit.
Chat appeared in the government's app on March 26, 2026, "without any proactive communications," and only on May 14 did GDS start "officially launching GOV.UK Chat," as an opt-in feature for app users signed in with GOV.UK One Login, after what PublicTechnology called "two and a half years of development and trials."
London promised "instant, clear and reliable answers" too, and kept its helplines open. What came with the promise was the fine print. The app's terms call Chat "experimental" and "voluntary," and say it outright: "You are responsible for checking the accuracy of GOV.UK Chat's answers. You must verify any information provided in GOV.UK Chat responses before taking any action or otherwise relying on the information." Its designers set out to "be upfront about the small, unavoidable risk of inaccuracy with AI tools." Compare that with "no chance."
It still made mistakes. Ian Murray, then the minister for digital government, told The Times the bot had been "tested to essentially destruction," LBC reported on May 15, 2026. Yet in questions The Times published, the tax expert Dan Neidle of Tax Policy Associates had asked whether a £1,000 raise from £99,000 would affect tax-free childcare, and was told "the scheme does not have an upper income limit," missing the well-known £100,000 cliff edge. The Chartered Institute of Taxation tells its members to treat Chat as "a signposting and research tool."
So the difference isn't the fine print. Britain doesn't let you rely on its chatbot either; its terms say so, and America.gov's own warning tells you to check the sources. The difference is process: years of testing, a published score and an opt-in launch inside an app, against a site launched the day its order was signed with a promise of "accurate answers." As of October 3, 2026, I could not find a published accuracy figure or test results for America.gov in the order, the fact sheet or GSA's release. GSA says it built the platform "in record time"; the president says the project began "almost on day one" of his administration. What's missing is the published evidence.
When the chatbot starts filling in your forms
"Very soon, all she'll have to do is ask America.gov and it'll fill out the forms, track the progress," the president said of a newlywed changing her name, according to Fox News.
So imagine it is 2028 (this part is my extrapolation, not anyone's plan). You are 60 and widowed, and you ask America.gov whether you can claim. It tells you to wait until 62 and offers to start the paperwork then. Why wouldn't you believe it? The order behind it promised accurate answers. Two years later, someone at a Social Security office tells you otherwise.
Whose mistake is that? The order keeps each agency's "adjudicatory authority," so Social Security decides your claim — but its misinformation rule covers its own officers and employees, and the chatbot is GSA's. You would need proof, and the privacy design that keeps your conversation from being saved (a real virtue) also means there may be no record unless you kept one. Once the chatbot is filling in Social Security's own forms, would a court treat its answers as Social Security's? I don't think anyone can know yet. The order says nothing about relying on an answer at all.
What the builders, the skeptics and the scholars say
The people running America.gov call it a beginning. "People need to start with something simple to gain confidence and familiarity with it," GSA's administrator, Ed Forst, said, according to the New York Post. "Quite frankly, you don't jump to the end game past that." On the free-market right, R Street's Spence Purnell and Adam Thierer, opposing the FTC's proposed accuracy policy for private AI companies in July 2026, noted that most large language models "already have disclosure statements." By that logic, America.gov's warning does its job.
On the center-left, Brookings' Nicol Turner Lee told Newsweek that using AI "in theory, could help alleviate some of that burden," but that "users will need accountability when something goes wrong." Code for America's chief executive, Amanda Renteria, backed the goal but told FedScoop that "the measure of success isn't whether government has adopted a new technology."
The scholars got here first. In 2022 the Administrative Conference of the United States (ACUS), a federal agency that advises other agencies, noted that "Agencies generally take the position that users cannot rely on automated legal guidance." In a report for ACUS, the law professors Joshua Blank and Leigh Osofsky found that officials "all agreed that, as a matter of law, users could not, in fact, rely upon such tools," and one agency stressed that its tool could not give a wrong answer "because all the answers had been internally vetted." (Sound familiar?)
What does this mean for you?
Use America.gov to find the right door. Just don't mistake it for the person behind the door.
Treat every answer as a pointer, not a ruling. The site itself says "AI can make mistakes," and the agency that runs your program, not the chatbot, decides your claim.
Click through before you act. The site tells you to "check the sources included with each answer before making an important decision." Gounder's advice: "Use it to find forms and deadlines."
Screenshot the question and the answer, with the date and time showing. The site says it doesn't save conversations, and answers change. A screenshot guarantees nothing (Social Security won't find misinformation "based solely on your statements," and, as I read its rules, a chatbot's answer may not count at all), but it may be the only record you have.
For anything touching money or eligibility, ask the agency. The order keeps in-person, phone and mail service, so that America.gov "does not become the only option." For Social Security, that means the toll-free line, 1-800-772-1213, or your local office, in person or by mail. As I reported on September 30, the average wait on Social Security's national 800 number has fallen from 34 minutes to 0.6 minutes, though Representative Judy Chu, pointing to how phone waits are counted, called some of the agency's statistics "extremely misleading."
For Social Security, get it in writing. The "preferred evidence" in its misinformation rule includes "A notice, letter or other document which was issued by us and addressed to you" and "Our record of your telephone call, letter or in-person contact." When you call, note the date, the time and who you spoke with. It is the habit I urged in August for anyone denied credit: get it in writing, and note the date.
If anyone says you don't qualify, protect your date anyway. Social Security will use the filing date of "a written statement, such as a letter," showing you intend to claim, if you meet the rule's conditions, including filing the application within six months of the agency's notice. By mail, the postmark can count when the date of receipt would mean losing or lessening your rights. And filing an application is what gets you "a written notice of our initial determination," with the reasons and your right to reconsideration.
The lesson, as I see it
The order's promise of "accurate answers" is a fine one. But a promise with no remedy behind it works like a slogan — and the remedy isn't the order's to give. It lives in a 1990 Supreme Court rule and a Social Security statute written for the agency's own officers and employees, and as I read both, a wrong answer from the government's chatbot falls between them.
In July I wrote that "a machine that cons you is a machine that should answer for it." America.gov isn't conning anyone; a stale insulin answer is an honest mistake. But the question is still Justice Marshall's: who should bear the burden of the government's error? As the law stands, as I read it, you do.
The fixes are within reach. OMB, whose implementation memo the order requires by December 28, 2026, could adopt what ACUS recommended in 2022: let users "obtain a written record of their communication with automated legal guidance tools," with "date and time stamps," and keep a public archive of every update. The administration could publish an accuracy benchmark, as Britain did. And Congress, which the Court said "may always" expand recoveries for people who rely on mistaken advice, could extend Social Security's misinformation rule to the government's own chatbot. (It would not take a long bill.)
My vote? Use the front door. Keep the receipt. And until the law catches up with the promise, check the answer with the agency that decides.
If you handle a parent's Social Security, Medicare or VA paperwork, or know who does, send this along before the chatbot becomes the first call.