You may have seen the news on October 29. The 74, a nonprofit news site covering education, reported that in a randomized trial of 1,179 students in 10 U.K. schools, an AI math tutor matched expert human tutors at helping students fix mistakes in the moment — and that after 10 weeks, neither kind of tutoring moved scores on a broad standardized math test.
Be honest: which half stuck? If you're a parent, a teacher or a student, it's probably the first, because it's the one you wanted. Human tutors are expensive, and an AI that keeps up with them sounds like tutoring for every kid who needs it. Last December, The 74 reported on a smaller study suggesting that the same company's AI tutor, paired with a human helper, can outperform a human alone.
Bibi Groot, chief impact officer at Eedi, the company that ran the trial, didn't skip the second half. "We don't see that the interactive tutoring improves Star Maths scores, over and above the standard remediation," she told The 74.
So which is it, a tutor as good as an expert or one that didn't move the test? Both. And the trial's own documents hold a third fact that matters more for what comes next — who read the AI's messages before a student did.
Four kinds of help after every wrong answer
So what did the trial test? Eedi, a London-based math platform, led the design and the work in schools and built the constrained AI Tutor, then analyzed and published the findings jointly with Google DeepMind. (The tutor is Eedi's, with Google's model inside: it uses Gemini, which Google DeepMind fine-tunes for teaching, "inside a strict Eedi architecture.")
The companies' technical report describes a pre-registered trial in 10 state secondary schools, across England, Northern Ireland, Wales and Scotland, all already using Eedi. From April 13 to July 7, 2026, classes of 12- to 15-year-olds used the platform for "roughly 30 to 50 minutes per week." Students were randomized one by one within each classroom, so classmates in the same lesson got different help after a wrong answer: the app's standard help (a tailored hint, instructional videos and fluency practice); a live text chat with an expert human tutor writing every message from scratch; or a chat with Gemini, given either the question's context or that plus the student's learning history.
In both AI versions, an expert stood between Gemini and the student. "The platform required explicit approval from the tutor before sending any message to the student," the report says. Tutors reviewed 94,385 of Gemini's drafts (others were superseded before review and never sent) and approved 94.7% of them without any changes, up from 74.4% in last year's smaller trial.
So far so good. After help, students answered the next question on the same topic correctly 57.7% of the time with an expert, 56.9% and 56.5% with the two AI versions, and 43.4% with the standard help. On the first question of the next unit, the AI versions were level with the experts, and all three kinds of tutoring beat the hints. "The AI and human tutor are basically on par," Groot told The 74.
The test the trial named first
Why fuss over one test when the lessons went well? Because the trial's registration — filed before it launched — named Star Maths as its primary outcome; the in-lesson measures were secondary. Star Maths is a standardized test from Renaissance, which gave and scored it independently, before and after.
On that test, the report found no overall advantage for tutoring. The AI version with question context came out "almost identical" to the standard help, at −0.01 standard deviations (range −0.15 to +0.14), and the version with the student's history at +0.02 (−0.12 to +0.16). (Standard deviations are the usual yardstick for test-score effects; both ranges run from a modest loss to a modest gain, centered near zero.) "More surprisingly, expert tutoring lagged behind all three other conditions," at −0.09 (−0.25 to +0.07) — a gap Eedi says "could be due to chance." All of it rests on the 916 students who sat both tests. In short, AI tutoring tied the hints.
As of October 30, 2026, none of this is peer reviewed. The companies published the report themselves ("© 2026 Google," led by Google's Kevin R. McKee and Eedi's Groot). It says the protocol went through independent ethical review, with a favorable opinion from an ethics committee at Google DeepMind.
That complicates a line of mine. In September, writing about seven English schools that put a chatbot against a notepad (the notepad won), I cited a smaller exploratory trial, 165 students in five schools, as evidence that "points the other way." This trial — seven times larger — kept the preprint's "at least as well as" and lost its edge. Last year the supervised AI came out 5.5 percentage points ahead of human tutors on new problems from later topics (with an uncertainty range, −1.4 to +12.4, that includes zero); this year it was level on that kind of measure.
The case for the tutor (it's stronger than it looks)
Before you file this away as a flop, hear the other side. The trial's own documents make it well.
First, broad tests move slowly. When Eedi's platform itself was tested over two years, the report says, it showed "only small, uncertain effects" at six months, with clear gains emerging "around one to one-and-a-half years into the trial." In 18 Tennessee middle schools, assignment to Khan Academy's Khanmigo raised math achievement by about 0.06 to 0.08 standard deviations a school year — gains its NBER working paper says "resemble those from Khan Academy practice without AI assistance." And the registration built its power analysis around comparing the two AI versions with each other (a 58.6% chance of detecting an effect of 0.1 standard deviations), so a small gain could hide inside ten weeks.
Second, the hints were no straw man: the authors chose "an active, evidence-based control condition," holding AI tutoring to "matching or improving upon proven interventions." Even expert humans didn't clear that bar.
Third, conversation costs time. Each chat took "roughly four to seven minutes of lesson time," and over the trial, students in the AI arms completed a median of 99 study units, against 132 with the standard help; Google's own post says AI-tutored students solved about 14 problems a session against 19 with hints. The authors call their explanations "tentative" and suggest the slowdown "may thus reflect an early cost—an investment in the future—rather than a lasting headwind."
Fourth, the average hides a split — if a fragile one. In exploratory analyses, boys outpaced girls on Star Maths by 0.31 standard deviations under the standard help, and under tutoring that gap shrank to between +0.09 and −0.05, every range crossing zero. Two-thirds of the students were boys, because the largest school was all-boys; Eedi calls the results "tentative," to be checked in "a larger trial." (As of October 30, the girls-only estimates still await an appendix that needs "a bit more time to compile.")
All of that is fair. None of it changes what this trial can claim today: a match with human experts on secondary measures, with an expert approving every message; no gain over the hints on the outcome it named first; and a next step that removes the approval under which the safety record was measured.
Who reads the messages now?
How safe was it? By the companies' count, very. Eedi says its checks found 11 AI messages containing a false math statement: tutors stopped 4, and 7 reached students (0.01%). The same checks found 7 genuine math errors in human tutors' messages (0.02%), "along with many more typing slips." The audits leaned on Google's own models, too: Gemini flagged potential errors, and team members then reviewed and adjudicated every flag.
The safety had a cost the tutors felt. The 58 experts, most from Eedi's own tutoring team, handled up to six conversations at a time, and Eedi says most found careful review harder when supervising several students at once. Groot told The 74 that keeping expert tutors in the loop to monitor the AI is also expensive and "not a scalable solution."
For its next trial, Eedi says, it has moved "from a human-in-the-loop approach, where a tutor approves every message, to human-on-the-loop, where tutors oversee conversations and step in where they make the most difference." (In plain words: a person still watches but no longer signs off on each message.) A Harvard-led registry entry that matches it describes a year-long trial with 15,000 planned students, with tutoring from September 1, 2026, to June 30, 2027, in which the AI opens a conversation after a wrong answer and "a live human tutor joins" when a student asks. Its primary outcome is Star Maths again, against "business as usual" without Eedi, a different question from this one.
As I wrote in October, the human in the loop was meant to be the check on the machine. Here it largely held — and it's the first thing being redesigned.
In Ghana, a WhatsApp tutor and a different experiment
What does an AI tutor trial look like when it asks a different question? Ghana has one. In 2023, students in five schools run by Rising Academies, an education network based there, got an hour a week with an AI math tutor on WhatsApp, on top of regular lessons, while six other Rising schools carried on as usual. The tutor, Rori, is Rising's own and free; its questions are written by math experts, African Business reports, with AI interpreting students' answers. Teachers were present "to help resolve potential technical issues," and the paper describes no one reviewing the tutor's messages.
On a 35-question test of grade 3 to 5 numeracy and algebra skills, among the 477 students in grades 3 to 8 who sat it twice, those with Rori gained 5.13 points against 2.12 for those without, an effect size of 0.36 that the authors call "approximately equivalent to an extra year of learning." The authors, some of them from Rising, add that the results "should be interpreted judiciously, as they only report on year 1," and that "it is also possible that Rori is only effective in helping students with relatively easier math topics." The paper appeared in an AI-in-education conference's proceedings.
Line the two trials up and almost every choice differs. Ghana added time; Eedi swapped one kind of help for another inside the same lesson. Ghana's comparison group got no extra practice; Eedi's got help that already works. Ghana tested basic skills from earlier grades; Star Maths is broad. Ghana randomized 11 schools; Eedi randomized 1,179 students. So the two results answer different questions — and neither grades the other.
What they share is the next question, and Rori's researchers wrote theirs down. The first evaluation ran "with rigorous oversight in controlled environments," their follow-up registration says, and "it is crucial to determine if other schools can adopt Rori without that level of oversight." (There, oversight meant frequent monitoring visits; here, an expert reading every message. Same problem, different machinery.) That 2025 pilot had posted no results as of October 30. Rising says a larger trial in more than 100 public schools was to start in September 2026, and African Business reported on October 16 that a five-district trial was under way. None has published a result yet.
The first number arrives in weeks. The second takes a year.
Let me sketch a plausible 2027 (a scenario, not a forecast). The U.K. government wants AI tutoring tools for up to 450,000 disadvantaged pupils "available to schools by the end of 2027," and Eedi holds one of eight government research contracts to build prototypes through March 31, 2027. In the U.S., Eedi has announced a year-long trial in two U.S. school districts.
Now picture the dashboard. Instead of an expert approving each message across six chats, one person watches a screen of conversations and steps in when a student asks or a flag goes up. Six chats per tutor becomes thirty, then sixty — that's where the savings are. (No document says sixty. That's what makes it a scenario.)
The in-lesson numbers arrive within weeks and look good, as they did this time. The Star Maths answer can't come before the tutoring year ends on June 30, 2027. In between, budgets get set and summaries get written, some of them by machines. Google's post on this trial already carries one written by Google AI ("Generative AI is experimental," its label says): "In the U.K., students using AI math tutors performed as well as those with human tutors, learning to solve problems step-by-step." True of the lesson; silent on the test.
Meanwhile the tutor opens after every wrong answer, for every child, all year (Eedi itself has floated, tentatively, that some students might feel being pulled into a conversation after every mistake is "a signal of failure"). Seven AI messages with a math mistake reached students here even with an expert approving each one; what that number becomes with one person watching dozens of chats is the figure the next trial most needs to publish.
Hopeful or wary, they mostly ask for the same thing: evidence first
So where do the people who study this land? Frederick Hess of the American Enterprise Institute, a right-of-center think tank, wrote in March, before these results, that on AI instruction the jury is out. He makes the hopeful case as well as anyone ("the costs and practical challenges of procuring enough trained, reliable tutors are brutal. AI offers a path out of that conundrum"), then adds: "The potential is real, but much caution is in order."
The National Education Union is wary: "The profession is far from convinced that AI tutors are a magic bullet for closing opportunity gaps for disadvantaged students," general secretary Daniel Kebede said in April; in the union's survey of 9,408 teacher members in English state schools, just 14% backed the government's planned AI tutors.
Stanford's SCALE Initiative said in an August brief that two recent studies, one of them last year's smaller trial, suggest AI-led tutoring with a human overseeing and intervening as needed is workable in design but unproven in effect, and its review of the research on AI in K-12 schools found that only 20 papers produced strong causal evidence. Its summary of the pattern could caption this trial: "Short-term boost, uncertain transfer."
Even Eedi wants patience: it says the field needs trials that run "for at least a full academic year" before findings like these guide AI tutoring at scale.
What does this mean for you?
Ask which number a claim is about. Here, The 74's "matched expert human tutors" meant in-lesson accuracy, a secondary outcome. Ask whether the study was registered in advance and what it said it would measure first.
Ask what the tutor replaces, in minutes. Here a chat replaced hints inside the same 30 to 50 minutes a week, and tutored students covered less ground. Stanford's SCALE brief says human-tutoring requirements, "such as a 90-minute weekly dosage threshold," apply to AI-led tutoring too.
Ask who reads the AI's messages, and whether that's changing. In this trial an expert approved every one; in Eedi's next, tutors oversee and step in. Get the oversight plan in writing: who watches, how many chats at once, and what brings a human in.
Ask whether your child is in a study, and whether the chat says it's AI. This trial used opt-out, told families some tutoring would involve AI support and marked AI chats with "a clear indicator"; last year, The 74 reported, students didn't know whether they were talking to a human or a chatbot. Defend Digital Me, a children's data-rights group, told Parliament's AI-in-education inquiry that "Schools operate as a compulsory, non-consensual environment," so saying no should cost nothing.
Ask for the error rates. Eedi is calling on every company building classroom AI to publish its safety benchmarks and error rates; hold your vendor to that. England's own standards say suppliers should not exaggerate the impact of their tools.
Ask what happens when a student just wants the answer. In a PNAS study of nearly a thousand high school math students, those given unrestricted GPT-4 did worse once it was taken away than students who never had it; a guarded GPT-4 tutor "largely mitigated" that harm. Eedi's tutor held back direct answers until the student had tried in 98.8% of conversations.
In the U.S., the Education Department's AI grant priority, effective since May 13, 2026, lists high-impact tutoring among the uses it can fund. Ask your district which priority any AI-tutor grant uses and which outcome it will report.
The lesson, as I see it
An AI tutor that keeps pace with an expert while an expert reads every word is a real achievement (last year's version needed far more correcting), but a conditional one — and the condition is what's about to change.
The next trial drops that condition and runs for a full school year, on the same test. The risk is the calendar: the in-lesson numbers will keep arriving first, and the Star Maths answer won't come before mid-2027.
My vote? Treat human-on-the-loop as the experiment it is until the year-long results are in. And anyone selling a school an AI tutor before then should hand over the two numbers this trial taught us to ask for: the score on a test named in advance, and the count of mistakes that reached a child.
In the lesson, the AI kept up with the experts. On the test, the hints kept up with both.