Does AI make up numbers? How do you prevent it?

Can AI really get numbers wrong?

Yes. You are right to ask; it is the first question to put to anyone proposing to install AI in a firm or a business. Generative AI produces plausible text, and plausible text can be false: an amount copied wrong, a total that looks right, a convincing explanation of an error. That is not a defect a vendor patches away in the next update; it is the nature of the tool, and a system has to be built with that reality in mind. The honest conclusion is not to keep AI out: it is to build the system accordingly, so that generative AI is never the source of record for a number. In a well-built system, the question “does AI make up numbers?” becomes “what holds the official numbers, and how is that verified?”. The architecture, not the promises, is what tells a buyer the work was done seriously.

What holds the official numbers, then?

The official numbers never live inside the AI. The accounting software remains the system of record for amounts, remittances and filings: it is what gets consulted, it is what counts, and it stays that way. Around it, the steps that must be exactly repeatable run through deterministic workflows: fixed-rule automations where the same input always produces the same output, with no AI judgment involved. Moving a file, sending a reminder on a schedule, applying a categorization rule, computing a remittance date: all of that belongs to fixed rules, not to generation. When an amount moves from one system to another, a rule carries it; no one retypes it, and no AI “remembers” it. The separation is structural: what must be exact is deterministic; what calls for language and nuance goes to the AI. That is also why an audit trail exists at all: fixed rules leave tracks by design, and tracks can be checked.

What is the AI for, if fixed rules hold the numbers?

The AI reads, summarizes, drafts and calculates with the financial data, because that is precisely what makes it useful. It reads a shared inbox and pulls out the supporting documents; it summarizes a client file before a call; it drafts the follow-up someone would have typed by hand; it explains a variance between two months. Pulling it off that work “to be safe” would sell a weaker system, not a safer one. The distinction that matters is not “does the AI touch the numbers”; it is “what counts as official”. The AI works on the data and prepares; the accounting software and the fixed-rule workflows record and count. A draft email, a file summary, a checking calculation can all come from the AI without danger, precisely because none of it becomes official until a person has passed on it. The useful question for any vendor is where they draw that line, and whether they can show it.

How do you know an error will not reach a client?

Through verification, on two levels. First, the system flags its own exceptions: the document that does not match, the amount out of the ordinary, the case the rule does not cover. Those cases are not hidden; they are queued for a person on the team to decide. Second, nothing critical leaves without the team’s approval: anything that touches the books or goes out to a client passes through someone. Instrumented workflows record their own activity (what ran, when, on what), which makes every step checkable after the fact instead of assumed correct. The principle that sums up everything else: trust comes from verification, not restriction. A vendor who answers this question by promising its AI “touches nothing” is selling a less useful tool, not a safer one; a system worth trusting shows its checks. Ask to see the exception queue and the approval step; a real system has both on screen.