Quality control
How to check AI output when you are not the expert
Checking a draft of your own marketing is easy — you know what good looks like. Checking a contract clause, a tax treatment or a technical spec is the opposite, and it is exactly where people accept the answer because it sounds authoritative.
This is a different problem from stopping AI inventing answers to customers, which is solved by supplying the facts up front. Here you genuinely do not have the facts, which is why you asked.
Confidence carries no information
The single most important thing to internalise: a model sounds exactly as certain when it is right as when it is wrong. There is no tremor in the voice, no hedging, no tell.
Human experts signal doubt — they say "I think", they qualify, they mention what they would check. A model asked a question it cannot answer will produce something fluent and complete-sounding. Fluency is not evidence.
Ask it to argue against itself
The cheapest check available, and it takes one line.
"What is the strongest case that this answer is wrong?" or "What would a specialist say I have missed here?" You will frequently get a genuinely useful list of caveats that the confident first answer omitted entirely.
This works because you have changed the job. The first request asked for an answer; the second asks for criticism, and criticism surfaces the uncertainty the first pass smoothed over.
Ask the same thing twice, separately
Open a fresh conversation and ask again, worded differently. Do not reference the first attempt.
Two independent answers that agree on the substance are more trustworthy than one. Two that differ have shown you precisely where the shaky ground is — and that is the part to take to someone who knows.
Better still if you can ask two different tools. Where they disagree is almost always where the genuine ambiguity lives, and it costs you two minutes to find out.
Make it show its working
"Explain how you reached that, step by step." You may not be able to judge the conclusion, but you can very often judge the reasoning — and errors that were invisible in a confident summary become obvious in a sequence of steps.
Watch specifically for the step where a specific figure or rule appears from nowhere. That is where the invention usually sits.
Check the citations exist
If it names a source, a standard, a regulation or a case, look it up before you rely on it. References that do not exist, or exist but say something different, remain one of the most common failure modes.
This takes thirty seconds per reference and catches the errors that would embarrass you most, because a fabricated citation is far harder to explain away than a wrong opinion.
Know which questions to stop asking
Some things should not be resolved this way at all, however good the checking.
- Anything with a legal deadline — filing dates, notification windows, limitation periods.
- Anything where being wrong is expensive and irreversible — contract terms, tax treatment, employment decisions.
- Anything medical or safety-critical.
- Anything specific to your jurisdiction, where rules differ by country and change without notice.
Use it to understand the shape of the problem and to arrive at your accountant or solicitor with better questions. That is genuinely valuable and it is not the same as using it instead of them.
The question to ask yourself
Before acting on something you could not have produced alone: if this turns out to be wrong, what happens, and could I explain how I decided it was right?
"It sounded confident" is not an answer that survives contact with a consequence. "I checked it two ways, the reasoning held, and I confirmed the one number that mattered" is.