Why I Think Computers Should Be Allowed to Say "I Don't Know"

A useful answer is not always an answer

We are surrounded by systems that appear determined to have the last word. Search engines finish our sentences. Satnavs issue instructions with the calm authority of a Victorian headmistress. Generative AI can produce an essay, a recipe, a legal-sounding letter, or a remarkably convincing explanation of something that never happened.

That confidence is often mistaken for competence.

I would rather a computer occasionally said, “I don’t know”, “I’m not confident enough”, or “This needs a human to check it”, than gave me a crisp, fluent, entirely wrong answer. That is not a demand for timid software. It is a demand for honest software.

Confidence is useful only when it is deserved. In many cases, software able to express uncertainty may be considerably more valuable than software designed to produce an answer at any cost. An answer is not a prize for filling in every blank on the form.

Computers already deal in uncertainty

Despite the popular image of computers as perfectly certain machines, much of modern software is built on estimates. A temperature-monitoring system may infer that a sensor is faulty because its value has frozen, jumped beyond plausible limits, or disagrees sharply with nearby readings. A machine-learning model may classify an image as a cat with a score of 0.92. A GPS receiver calculates a position with an uncertainty influenced by satellite geometry, signal quality and interference.

The important question is not whether uncertainty exists. It does. The question is whether the system exposes it honestly and uses it sensibly.

A sensor reading of 4.1°C is not automatically a fact carved into granite. It might be a good measurement. It might be a probe losing thermal contact, a broken cable intermittently reconnecting, or an analogue-to-digital converter being introduced to electrical noise at a most inconvenient moment. Good engineering does not simply display the number in a large cheerful font and hope nobody asks awkward questions.

It applies plausibility checks, detects stale data, tracks calibration status, and raises an alarm when the measurement cannot be trusted. Crucially, it distinguishes between “the temperature is out of range” and “we do not know the temperature reliably”. Those are different operational situations and should lead to different responses.

A probability is not a promise

AI systems make this problem particularly visible because they are very good at sounding sure of themselves. Language models predict plausible sequences of words. That can be genuinely useful, but it does not mean they possess reliable knowledge of every fact implied by those words.

Even a numerical confidence score needs handling carefully. A model that reports 90% confidence should be correct roughly nine times in ten across comparable cases. This is called calibration. Without it, the number is just a decorative badge, like putting “professional” on a PowerPoint slide in a heroic font.

A system can also be highly confident for the wrong reason. It may have learned a shortcut from its training data: a background pattern, a formatting quirk, an association that happened to work in the examples it saw. When presented with something outside that familiar territory, known as out-of-distribution input, it may continue confidently regardless. That is precisely when it ought to slow down.

The practical design response is not mystical. Set thresholds. Ask for more evidence. Compare independent sources. Detect missing or contradictory data. Present the confidence level and the reason for uncertainty where possible. Escalate borderline cases to a person who can make a judgement.

None of this is glamorous. It is also how we avoid building a digital version of the colleague who answers every question in a meeting before realising it was addressed to someone else.

“I don’t know” needs to be useful

There is a lazy version of uncertainty, of course. A system that responds “cannot determine” to everything is not cautious. It is a very expensive ornamental plant.

A useful admission of uncertainty should help the user decide what happens next. It might say:

• “I cannot verify this claim from the available sources.”
• “This reading is unreliable because the sensor has not updated for ten minutes.”
• “Two data sources conflict. Please review the original records.”
• “I can offer a draft, but this needs expert review before it is used.”

That last distinction matters. Software can still be helpful when it is uncertain. It can show its working, identify what information is missing, propose alternatives, or route a case to the right person. The goal is not to make the machine helpless. It is to stop it pretending that uncertainty has disappeared merely because uncertainty is inconvenient.

This is especially important in systems affecting health, money, safety, education, benefits, employment, or access to public services. If a system cannot explain its confidence, indicate its limits, or provide a route for challenge, people carry the consequences of its guesswork. Usually the people least able to absorb another administrative error. A cheery automated decision is still an automated decision, even if it uses reassuring pastel colours.

Trust is built by limits, not theatre

It can feel counterintuitive, but a system that occasionally admits its limits can become more trustworthy. We generally trust people more when they can say, “I’m not sure, let me check”, than when they confidently invent an answer and hope the conversation moves on before anyone notices.

Computers should earn that same trust. They should tell us when a reading is degraded, when a model is operating outside its validated conditions, and when evidence is too weak for a firm recommendation. Designers should make uncertainty visible without making interfaces frightening or unusable. Users should not need an engineering degree to discover that the result was based on a faulty sensor, incomplete data, or a model having a bit of a guess.

There will always be pressure to make software look decisive. Decisiveness sells demos. It makes dashboards feel reassuringly busy. But a polished answer without adequate evidence is not intelligence. It is theatre with an API.

The most capable systems will not be the ones that answer every question. They will be the ones that know the difference between a well-supported answer, a tentative suggestion, and the moment to say, plainly and helpfully, “I don’t know.”