If you have been a patient in an American hospital lately, there is a question you probably didn't think to ask: who decided you were fine?
The honest answer may be a person and a piece of software, taking turns. In 2024, 71 percent of hospitals reported using predictive AI built into the electronic health record (the federal count covers non-federal acute care hospitals), most commonly to predict "health trajectories or risks for inpatients," such as early detection of a condition "like sepsis." So while you slept, a model may have been scoring how worried everyone should be about you. When it flagged something, a nurse or a doctor looked at the alert, looked at you, and decided whether the machine was right.
On September 30, Governor Gavin Newsom signed AB 1979, which tells hospitals to take reasonable steps to protect that clinician's judgment whenever software informs your care. The same day, he vetoed AB 2575, which would have barred punishing a bedside worker solely for overruling the software (or for trusting it). The institution got a duty; the person at your bedside got no protection of their own. Why should you care, lying in that bed? Because whoever decides whether to believe the screen may also be weighing what disagreeing will cost them.
So what does the new law make your hospital do?
AB 1979, by Assemblymember Mia Bonta, turns on one sentence: a health facility, clinic, physician's office or group practice "shall take reasonable steps to ensure" that a licensed provider, acting within their scope of practice, "retains the ability to exercise independent professional judgment" whenever your care "is informed by the output of a clinical decision support system." That means any AI producing "a prediction, classification, recommendation, evaluation, or analysis" that helps decide the timing of your care, your diagnosis or your treatment.
The governor's office billed it as "Empowering doctors to make the final call." (Doctors, yes, but the law covers every licensed clinician, nurses included, and the California Nurses Association sponsored it.)
It also bars those institutions from using AI to "independently perform" clinical work the law reserves for licensed professionals, and exempts tools for "documentation and communication that does not involve the application of professional judgment." (Whether the AI scribes I wrote about in August qualify, the law doesn't say.) It takes effect on January 1, 2027.
So far so good. But the Senate softened it. The bill lost a requirement that "no clinical decision" rest solely on the software's output, traded "ensure" for "take reasonable steps" to ensure, and struck the enforcement provisions that ran through the unfair competition law and the Department of Public Health. What remains sends "a violation of this section by a physician" to the physicians' licensing boards, lets licensing boards seek injunctions against unlicensed practice, and preserves any remedy "otherwise authorized under the law."
I went looking for a penalty aimed at the hospital itself. The law names none; whether the state health department could still act against one on its own authority, the text doesn't say. Nor does it give the clinician a remedy of their own. After the amendments, the California Hospital Association listed the law as "Neutral as Amended."
What would the vetoed bill have added?
AB 2575, by Assemblymember Liz Ortega, was about the person rather than the building. When the AI gets it wrong, Ortega said, bedside workers face a double bind: "Follow the machine, and then they get blamed, or override the machine and then they risk retaliation."
Her answer was a shield. An employer "shall not retaliate or discriminate" against a worker providing direct patient care, using professional judgment within their scope of practice, "based solely on the worker's override of, or reliance on," the output of a clinical decision support system. Three details matter:
It covered trusting the software as well as overruling it, for any direct-care worker.
"Solely" left the employer every other reason. The duties to meet the standard of care and follow written policies stayed, and the Labor Commissioner, who would have taken workers' complaints, could weigh an alleged failure to meet them.
Deference would not have counted as a duty. A rule requiring staff to "accept, defer to, or refrain from overriding" the output was expressly excluded from those duties.
It also reached the courtroom. In a suit over harm from one of these systems, anyone who "developed, modified, selected, or deployed" it — the vendor, but also the hospital — could not have argued that a health worker's failure to override was "a superseding cause severing the defendant's liability" (lawyer-speak for an intervening act that breaks the chain of blame). Other defenses remained, and malpractice suits against licensed clinicians were carved out. A law signed last October already bars a defendant who developed, modified or used AI from arguing "that the artificial intelligence autonomously caused the harm." Blaming the machine is out. Pointing at the nurse who didn't overrule it is still on the table.
So why the veto?
Newsom's veto message makes two arguments. The first is that by barring adverse actions based solely on an override or reliance, the bill "ties the Labor Commissioner's hands in establishing a violation — a higher bar than other retaliation protections, which simply prohibit action because of a protected activity."
Read that twice. The hospitals argued the shield went too far; the governor's first objection is that it was too weak: "This bill's anti-retaliation provisions do not protect workers as intended."
The second is about expertise. Tying the protection to scope of practice, he writes, "puts the Labor Commissioner in the position of adjudicating the standard of care for patients — something the Labor Commissioner does not have the medical expertise or skill to do."
That objection has footing: the bill let the Commissioner weigh "evidence supporting alternate reasons" for the employer's action, including an alleged failure to meet the standard of care. But it also said it "does not require the Labor Commissioner to adjudicate or make a final determination" on that question. Can a labor official weigh medical evidence without ruling on medicine? Reasonable people can read it either way.
The nurses' union calls the result a gap: "Today, a nurse who sees that the algorithm is wrong and acts to save her patient can still be disciplined or fired by her employer," said Sandy Reding, the California Nurses Association's president. One qualification: California already bars a health facility from retaliating against a worker for presenting "a grievance, complaint, or report" to the facility, its medical staff, an accreditor or a government agency. The report is protected. The bedside decision to overrule the screen isn't, specifically.
Isn't the nurse sometimes the one who's wrong?
Yes, and the hospitals' case deserves its full weight: clinicians miss sepsis, and alerts exist to catch it. The software misses too. A University of Michigan team that tested a widely used proprietary sepsis model, then implemented at hundreds of U.S. hospitals, reported in 2021 that it alerted on "nearly 1 in 5 of all patients, with most of those patients not actually having sepsis." (Karandeep Singh, one of the researchers, added that the vendor's own analysis wasn't wrong; the model had been built to predict sepsis as clinicians recognized it, and "we know that clinicians miss sepsis.") An updated version, tested across four health systems in a study published this year, ranks risk better but still comes with "high alert burden."
The case for trusting the alert is real, too. At five hospitals using a sepsis alert developed at Johns Hopkins, sepsis patients flagged before antibiotics whose alert a provider confirmed within three hours had a reduced in-hospital mortality rate, 3.3 percentage points lower after adjustment than those whose alert wasn't confirmed in that window. That is an association from an observational study, not a randomized trial, and the alert's developers led it (its senior author is a founder of, and holds equity in, the company that sells it). A companion study found 89 percent of its alerts were evaluated by a physician or advanced practice provider, and fewer than four in ten of those were confirmed. Overriding is routine. Sometimes it is also the mistake.
That is the opposition's strongest point. Hospital systems and physician, health-plan, device and technology trade groups warned that AB 2575 gave a worker "unrestrained authority to override" the software "without review or supervision, which could lead to patient harm," while insisting that providers "do not deploy AI or related technologies to make care decisions." Technology and tort-reform groups called the liability clause "a one-directional liability exposure."
The bill's text answers part of that, since an employer could still act on a failure to meet the standard of care. The rest is a genuine choice: the alert is sometimes right and the nurse wrong, and sometimes the reverse, which is why it matters who pays for each mistake.
Brussels builds the override into the product, from 2028 (for now)
What would it look like to protect the override itself? Europe's AI Act offers one design, with an asterisk.
Start with the product. High-risk AI must be built so it can be effectively overseen by natural persons, who must be enabled, "as appropriate and proportionate," to set it aside in any particular situation "or to otherwise disregard, override or reverse the output," and to halt it with "a 'stop' button or a similar procedure."
Then the hospital. A deployer (for clinical AI, the hospital) "shall assign human oversight to natural persons who have the necessary competence, training and authority, as well as the necessary support."
Clinical software gets there through product law: AI that is, or is a safety component of, a product covered by Europe's regulation on medical devices is high-risk when that product needs a third-party conformity assessment, and the EU's medical-device and AI boards say "Most MDAI are classified as Class IIa (MDR), B (IVDR), or above," so most medical-device AI needs one. (Their guidance isn't legally binding, and software that isn't a medical device may never take this route.)
Now the asterisk — for AI in medical devices, these duties start on August 2, 2028, a year later than first planned; Europe set those fixed deadlines this summer.
And 2028 may not be the last word. The same amendments let the Commission, by August 2, 2027, limit some of the Act's product requirements, the override requirement among them, where device law already protects at least as well. A separate Commission proposal, still before Parliament with a first-reading vote penciled in for December 14, would take medical devices out of these duties, leaving AI rules to device law.
Europe also protects people who speak up. Article 87 extends the EU's Whistleblower Directive to "the protection of persons reporting such infringements" of the Act, and that directive requires member states to prohibit retaliation, including "any disciplinary measure, reprimand or other penalty." By my reading, Article 87 has applied since August 2, 2026, though the hospital duties a nurse might report won't bind medical-device AI before 2028, and might never if that proposal passes.
To be clear, Europe hasn't written the shield California just vetoed, either. The AI Act has no clause saying a nurse can't be disciplined for overriding; its "authority" duty runs to the hospital, which keeps its freedom to organize its own resources. My reading is that a hospital that punished someone for using the override would struggle to claim it had given them "authority." On paper, Europe builds the override into the product and the authority into the hospital's duties. California made protecting the clinician's judgment a hospital duty and vetoed the protection for acting on it.
Imagine the override becomes a metric
What follows is my extrapolation, not a forecast.
Imagine a unit dashboard in 2028 that charts each nurse's override rate on the sepsis score. AB 1979 says nothing about counting overrides, and the vetoed bill would have kept a rule demanding deference from counting as a duty. In a 2024 National Nurses United survey of its own members, 40 percent of nurses in facilities that use risk or outcome scores said they are unable to modify scores to reflect their clinical judgment, and Ortega says she heard that "the expectation from their bosses was to follow the output."
Imagine, next, the first trial over a missed sepsis case in which the defense argues that the night nurse's failure to override was a superseding cause. The person in that spot occupies what the researcher Madeleine Clare Elish calls a moral crumple zone: responsibility "misattributed to a human actor who had limited control" over an automated system, in an arrangement that "protects the integrity of the technological system, at the expense of the nearest human operator." (Her cases come from aviation, nuclear power and driverless cars; the bedside is my application, not hers.) Courts hadn't settled it in 2024, when two Stanford scholars found the case-law signals on AI liability "remain somewhat faint."
Labor, the hospitals and the right agree on more than you might think
The union side: "The governor knows A.I. should never replace a nurse's clinical judgment, but he vetoed the bill that would have protected us for using our judgment," Reding said, and Lorena Gonzalez, president of the co-sponsoring California Federation of Labor Unions, added, "We are committed to bringing this bill back next year."
The hospitals: the hospital association's David Simon warned of "a very chilling environment where a single error or a single bias" would outweigh a useful tool's benefits.
The doctors: the American Medical Association's AI principles say, "Where a mandated use of AI systems prevents mitigation of risk and harm, the individual or entity issuing the mandate must be assigned all applicable liability."
The nurses' professional body: the American Nurses Association, quoting the profession's ethics code, says nurses "are accountable for their practice even in instances of system or technology failure."
The center-right: "Because the clinician remains involved in the decision-making and can verify the system's outputs, these devices generally avoid the need for premarket FDA clearance," writes Scott Gottlieb, a senior fellow at the American Enterprise Institute (Health Dive lists his venture and board roles).
The libertarian right: Adam Thierer and Logan Kolas, writing for R Street and the American Consumer Institute before AB 1979 passed (and not about it), urge lawmakers to start from "the many existing laws, regulations, and court-based remedies" that already apply.
The union, the hospitals, the governor and even the case for lighter federal rules share one premise: the clinician keeps the decision. They split on who pays when the clinician and the software disagree. Elish's term names what happens when the question goes unanswered: the nearest human pays.
What does this mean for you?
If a score or alert is shaping your care, ask what it is and which clinician is responsible for the decision. From January 1, 2027, the hospital must take reasonable steps so that clinician can use independent judgment. It is a fair question, and in a California acute care hospital you already have a right to know who is primarily responsible for coordinating your care.
Read the fine print on messages from your care team. Since 2025, an AI-written message about your clinical information from a California practice or facility must carry a disclaimer that indicates to the patient that AI generated it, plus "clear instructions" for reaching a human, unless a licensed provider read and reviewed it first.
Treat a health chatbot like a provider, because from 2027 California will. A business offering a "health care chatbot" to manage your information or a medical condition will be "deemed to be a provider of health care" under the state's medical-confidentiality law.
If you work in a hospital and an algorithm's output worries you, put it in writing to the facility, the medical staff, an accreditor or a state agency. The law protects that report; no statute specifically protects the override itself. (This is not legal advice.)
Know where complaints go. About a physician: the Medical Board of California, where complaints generally must be filed in writing and can be filed online (osteopathic physicians answer to the Osteopathic Medical Board). About a hospital or clinic: they are licensed and regulated by the California Department of Public Health, which takes complaints.
Tell your legislators what you think of a second try. The state's lookup will find your State Senate and State Assembly members.
The lesson, as I see it
California has written down half of a sensible idea. From January, the hospital must take reasonable steps to preserve the clinician's judgment, and software may not "independently perform" the work a license exists for. That matters.
The other half, what happens to the person who uses that judgment, is back where it started. Overriding a veto takes two-thirds of the membership of each house, 27 senators and 54 Assembly members, more than the 21 and 51 votes the bill drew, and the session ends November 30. The next version goes to a new governor — Newsom terms out in January.
That version has to answer the veto's two objections: a "solely" test the governor called too high a bar, and a labor official asked to weigh the standard of care. Those read to me like drafting problems, not reasons to drop the idea.
Both sides of this fight reach for the same phrase. The governor's office says the signed law empowers doctors "to make the final call"; Ortega says "Californians want a nurse, not a robot, making the final call." My vote? Protect the call and the person making it in the same law, and write the second half so a labor official can enforce it without playing doctor.
If a nurse ever got someone you love through a long night, forward this to that nurse, with a thank-you on top.