Published on [Permalink]

We still stink at probability and uncertainty. It’s time for Steel Man Narratives.:

Consumers of popular media still struggle with 538 or with NYTimes and their needles on election night. It’s not because this form of journalism is not valuable or capturing important information. It’s because thinking in models and probabilities is still deeply unnatural for most folks.

In the practice of data visualization, there’s been all manner of attempts to visualize uncertainty. We add error bars. We add shaded regions. We play with jittering, simulations, alpha channels, and drawing curves. Folks have built simulators to demonstrate drawing from distributions or the structure of joint-probabilities and such.

But as in the world of policy research, even the most sophisticated folks are drawn strongly to point estimates.

Unfortunately, the idea of “getting it right” based on the point estimate remains a strong measure of success in the eyes of many. And deviations are not meant with an academic curiosity, examination of the data, or consideration of our own failing heuristics. People are angry because they rely on their perceptions, even when their perceptions are filled with unearned certainty built on the back of common failed heuristics.

I think we can toss the whole Ian-hurricane-path kerfuffle into this bucket as well, and I still routinely run into people IRL who are continually confused by the fact that different weather apps have differing forecasts and that those forecasts aren’t “accurate.”