Does “90% confident” mean it is right nine times out of ten?
Calibration
Probabilistic confidence is calibrated when predictions given a particular probability are correct at roughly that frequency across comparable cases.
An everyday example
If 100 comparable predictions are each assigned 90% confidence, around 90 should be correct for that confidence level to be well calibrated.
Original illustration, not an observed participant case.
Where the idea stops
A confident tone or a model typing “90%” is not a validated probability. One answer cannot establish calibration.
Something to try
Separate how certain an answer sounds from the evidence for it. For repeated decisions, compare stated confidence with checked outcomes.
Practise on a fictional case →Go deeper: the builder’s view
For builders: evaluate calibration on held-out, relevant tasks and recheck it after system changes. Human confidence and model probabilities need separate measures.
Source and interpretation
This is a plain-language interpretation with an applied example; it is not a quotation or an endorsement from the source.
Guo et al. — On Calibration of Modern Neural Networks (2017) ↗Catalog v0.1 · checked 2026-10-29