Earlier this year, the Divisional Court had to deal with two cases in a row where lawyers had relied on AI-generated legal research that turned out to be wrong, fabricated case law presented as if it were real. Nobody involved was trying to cut corners maliciously. They trusted an output that looked complete and confident, and didn’t check it closely enough before it went in front of a judge.
That’s not an isolated story. A database tracking AI hallucinations in legal proceedings had logged over 1,150 documented cases of fabricated citations or invented case law by January this year, and the rate has been accelerating, from roughly two a week to two or three a day. These aren’t cases of AI being useless. They’re cases of AI being trusted one step further than it should have been.
We build two products that couldn’t look more different on the surface. One turns hundreds of pages of medical records into a clear chronology. The other turns a law firm’s Google reviews into a picture of what clients actually think and when things changed. But they were built around the same lesson those court cases keep teaching the whole profession: the risk was never that AI gets things wrong occasionally. It’s that AI gets things wrong confidently, in a form that looks exactly like it got them right.
What the research actually says about AI and document review
The honest picture on AI-assisted document review is more nuanced than either the hype or the horror stories suggest. Research comparing AI-assisted review against manual review found that human reviewers typically catch somewhere between 25% and 80% of genuinely relevant documents in a set, while AI-assisted technology-assisted-review systems catch somewhere between 67% and 86%. On raw recall, AI usually wins, and it isn’t close.
But recall isn’t the only thing that matters, and it’s not where AI quietly fails. One detailed breakdown of AI use in medical records review put it plainly: AI helps with extraction, and it hurts with judgement. A model can correctly pull out that a record says “cervical strain, possibly aggravated by prior collision.” What it can’t reliably do on its own is decide whether that entry belongs in the current-injury story or the pre-existing-condition story, a distinction that can change what a case is actually worth. Get that distinction wrong once, and, as the same research notes, opposing counsel doesn’t just question that one entry. They use it to question every other entry in the chronology.
That’s the exact failure mode UK legal commentary has been flagging too. Coverage of AI use in clinical negligence and personal injury work has been consistent on one point: AI can organise information faster than any paralegal, but it does not remove the need for careful assessment and professional judgement. The tools that get this right share one design feature in particular, every AI-extracted fact needs to link back to a specific page in the source record. Without that link, checking the AI’s work takes almost as long as doing the review manually, which quietly defeats the entire point of using it.
Why the reviews side works on the same principle
The Google reviews side of what we do looks like a completely different problem, but it fails in the same place if you’re not careful. It’s easy to build a tool that gives a firm a generic sentiment score, some percentage that goes up or down. It’s much harder, and much more useful, to build one that ties every finding to something real: an actual month where something changed, an actual named local competitor, the client’s actual words rather than a paraphrase of them.
A sentiment score a partner can’t trace back to a real review is exactly the same problem as a chronology entry a solicitor can’t trace back to a real page. Both are outputs that look finished and can’t actually be checked, which means both are only as trustworthy as the model that produced them, with nobody able to verify that from the outside.
The actual takeaway
None of this is an argument against using AI for either job. The recall numbers alone make the case for it. A tool that reads a thousand pages of records or years of reviews in minutes is doing something no person can do at the same speed, and there’s no good reason to give that up.
The argument is about what happens next. The firms and tools getting into trouble aren’t the ones using AI. They’re the ones treating a fast, confident-looking output as a finished one. That’s why every chronology we produce is checked by a person against the original pages before it reaches a solicitor, and why every review-based finding we hand back is tied to a real review, a real month, a real comparison, not a generic score. AI does the reading. A person makes sure what comes out the other end can actually stand up to the same question that’s now shown up in open court more than once this year: can you show me exactly where this came from?
If the answer to that question is yes, the tool was built properly. If it’s no, that’s worth finding out before it costs you a case, not during one.