--- title: "We let an AI write our client reports. It is not allowed to invent a number." description: "Coach uses AI to write the plain-English paragraphs of a client report. Code computes every figure, a checker rejects any number the AI was not given, and a person reads it before it goes." section: Inside Sneeze It author: David Steel published: 2026-10-10T20:15:31.133Z url: https://one.sneeze.it/blog/an-ai-that-cannot-invent-a-number tags: ["inside-sneeze-it", "coach", "ai", "reporting"] --- # We let an AI write our client reports. It is not allowed to invent a number. _Coach uses AI to write the plain-English paragraphs of a client report. Code computes every figure, a checker rejects any number the AI was not given, and a person reads it before it goes._ There is a sentence every agency should be afraid of: "Your cost per lead came in around $48 this month." It reads well. It sounds like someone looked. And if the real figure was $50.24, it is wrong, in a document the owner will forward to their partners. That is the specific way AI goes wrong in reporting. It rarely makes up a whole story. It rounds a number into a different number because the sentence flowed better, or it fills a gap with a figure that feels right. You cannot fix that by asking the model nicely. So when we built the client report inside Coach, we did not ask. We built it so the AI is unable to publish a number it was not handed. ## Who writes what A Coach client report has a standard structure, and most of it never touches AI. The comparison tables, the list of creative that ran this period against last, the recommendation and the measurement notes are all arithmetic over measured readings with a threshold. There is no judgment in them, so code draws them, the same way for every client every period. That sameness is a feature: an owner can compare two reports without relearning the document. AI writes only the narrative: one short paragraph per finding, in plain English, in two sections. The model we use is Claude, from Anthropic. Before it writes a word, Coach computes every figure in code from readings that actually answered, and hands the model that list, which we call the fact sheet, and nothing else. :::flow "How a paragraph gets into a report" Readings | Every source is read; a source that did not answer contributes nothing Fact sheet | Code computes each figure from the readings that answered AI writes | One short paragraph per finding, using only the fact sheet Checker | Any numeral not on the sheet, any banned word: rejected Coach | A person reads and edits; the check runs again at send ::: ## The checker After the model returns a paragraph, Coach reads it back and pulls out every numeral. Each one has to match a figure on the fact sheet, or the finding's own evidence, or it is rejected. The line between rounding and inventing is drawn in code, not left to a tolerance nobody can explain later. $50.24 may be written as "$50" or "about $50", because that is how a person talks. It may not be written as "$48", because that is a different claim. A share of 0.5436 may appear as 54% or 54.4%. A large figure may be rounded to the nearest ten, hundred or thousand. Anything else is invention. :::pullquote A gap is a visible hole. A hallucinated figure is not. ::: The same pass checks two other things. Alarm words are refused: concerning, crisis, unfortunately and similar. A report should say what is happening and what we are doing about it, plainly, not raise its voice. And each client can have a short brief with phrases we must never write for them, which the checker treats as a hard stop. Em dashes are stripped while we are at it. ## One retry, then a person When a paragraph fails, Coach gives the model exactly one more try, and tells it precisely why: these numbers are not in the figures you were given, use only the figures above or write the sentence without a number. That retry is not optimism. It is aimed at the most common failure, the sentence that rounded a number into a different number, and a model told exactly which figure it may not use usually writes the sentence correctly. If the second try also fails, Coach stops. The paragraph is dropped and the section shows up as "To be written" for the coach. We chose a visible gap over a plausible guess on purpose. A coach who sees a hole fills it. A coach who sees a smooth, wrong sentence may not notice it at all. :::compare "Asking the AI to be careful" "Making it unable to be careless" The prompt says "only use real numbers". The draft reads well. Nobody checks the $48 against the $50.24. --- The AI sees only the fact sheet. Every numeral is checked. Two failures leave a visible gap for a person to write. ::: Two more rules sit around the AI rather than inside it. **A miss stays a miss.** Only readings that answered become facts. If the call center sheet could not be read, there is nothing on the sheet for the model to write about, so the report says that number was not measured. It cannot become a zero by accident, because the zero was never offered. **A critical problem is never a paragraph.** Anything Coach rates critical is held out of the emailed report entirely. The page tells the coach the client needs a phone call first, and the team is alerted in Slack. Some news should come from a person's voice. :::figure wide ![An illustrated report page moving along a desk past three stations: a short pinned list of figures, a magnifying glass over the page, and a person with a pen making a final mark in magenta.](/blog/media/0730b9a88d3c2297857e4d07.jpg) The fact sheet, the checker and the coach. The AI writes in the middle, and both ends belong to code and to a person. ::: ## The check runs again when the coach presses send Once the AI is held to the facts, the most likely way a wrong number reaches a client is a person. A coach edits a paragraph and pastes in last month's figure. So the same gate runs again on the coach's edited text, before every send and every export. A numeral that no stored fact accounts for blocks the send. A banned phrase blocks the send. A coach can add a number on purpose. If they put it in their own note, it counts as allowed, and the note is shown beside the report on the page, so the figure has an owner who can stand behind it. After it goes, the report is frozen as sent, and the owner can rate it from 1 to 10. A score of 6 or lower alerts the team in Slack. And when a coach corrects something, the correction is fed back as a lesson the next time the model writes, so the same mistake is less likely to come back. :::stats 516 | automated tests across 31 files in Coach 23 | client reports delivered 1 | retry before a paragraph becomes a gap source: Coach codebase, counted October 10, 2026; delivered count from the live strip at [coach.sneeze.it](https://coach.sneeze.it) ::: The report pipeline has been live since September 8, 2026, and was reworked into its current form on September 16, when sending from the app went live. ## What this means for you More and more agency reports are written with AI. The useful question is not whether a model wrote the sentence, it is what the model was allowed to see and what checked it afterward. Ours sees a list of measured figures, writes in plain words, and loses the argument with a checker every time it reaches for a number it was not given. Then a person reads it. If you want to know why the report opens the way it does, read [Why our client reports open with the truth](/blog/truth-not-wins). For the full map of the systems behind it, start with [Inside One](/blog/inside-one-an-agency-that-runs-on-its-own-software). :::takeaways "What to ask any agency that uses AI in reporting" - Which numbers does the AI compute, and which does it only write about? The answer should be "none" and "all". - What happens when a number in the draft is not in the data? It should be rejected, not reviewed later. - When a paragraph cannot be checked, does the report show a gap or a guess? - Does the check run again after a person edits the text? - Who calls you when something is seriously wrong? It should be a person, before the email. ::: :::cta button="See Coach" href="https://coach.sneeze.it" The public page shows a sample report, the five lenses it reads, and a person's name on every send. :::