The Ambiguity of…Numbers?

Researchers have shown people who use “50%” or “fifty-fifty” often do not mean it literally. They mean “I’m not sure” or “it’s uncertain” – or more simply “maybe.”

Previously, I wrote about The Ambiguity of Language. Words are fuzzy, their meaning is fuzzy, everything’s fuzzy, alright?

But surely numbers – those precise friends who help us with all manner of things – are immune from ambiguity?

Well…once you throw people into the mix, things get a little fuzzy once again.

Like the previous post, this post draws upon examples from Superforecasting.

A Flip of an Unfair Coin

In this first example, the authors tell the story of President Obama being briefed by his intelligence staffers about the possibility/probability that Osama bin Laden was holed up in a compound in Pakistan. (I’ve condensed it a bit, but have preserved the meaning)

Sitting in the White House’s legendary Situation Room, Obama listened as an array of CIA officers expressed their opinions on the identity of the man in the mysterious Pakistani compound. The CIA’s team leader told the president he was almost sure it was bin Laden. “He put his confidence level at 95%,” Mark Bowden wrote in The Finish: The Killing of Osama bin Laden, Bowden’s account of the decision making behind one of the most famous commando raids in history. A second CIA figure agreed with the first. But others were less sanguine. “Four senior staffers at the Directorate of National Intelligence had reviewed the case and written out their own opinions,” Bowden recounted. “Most seemed to place their confidence level at about 80%. Some were as low as 40 or even 30%.” Another officer said he was 60% confident it was bin Laden in the compound.

“OK, this is a probability thing,” the president said in response, according to Bowden’s account.

…

After listening to the widely ranging opinions, Obama addressed the room. “‘This is fifty-fifty,’ he said. That silenced everyone. ‘Look guys, this is a flip of the coin. I can’t base this decision on the notion that we have any greater certainty than that.'”

Bowden clearly admires Obama’s conclusion. Should he?

The information Bowden provides is sketch but it appears that the median estimate of the CIA officers – the “wisdom of the crowd” – was around 70%. And yet Obama declares the reality to be “fifty-fifty”. What does he mean by that? We have to be careful here because there are actually several possibilities.

One is that Obama means it literally. He heard an array of views and settled on 50% as the probability closest to the mark. If so, he’s misguided. The collected judgment is higher than that and based on Bowden’s account he has no reasonable basis for thinking 50% is more accurate. It’s a number plucked out of the air.

But, as researchers have shown, people who use “50%” or “fifty-fifty” often do not mean it literally. They mean “I’m not sure” or “it’s uncertain” – or more simply “maybe.”

Whoa whoa whoa, hold on there: people use “fifty-fifty” to mean something other than the literal meaning of “fifty-fifty”? To quote Peter Griffin, from Family Guy: “This is…news to me.”

People use “fifty-fifty” to mean something other than the literal meaning of “fifty-fifty”? To quote Peter Griffin, from Family Guy: “This is…news to me.”

After reading that paragraph, I was gobsmacked. Here I’ve been going through life, tra-la-la, being very careful when I use numbers and probabilities, and apparently other folks aren’t behaving with the same courtesy.

WHAT THE HELL IS WRONG WITH PEOPLE?!?!

The whole point of providing a quantitative estimate is to disambiguate, and now folks are just goin’, “Huh, fifty-fifty” all willy-nilly to mean uncertainty in general rather than perfectly even odds?

It’s like a foundation of my belief structure has been shifted. I mean, the news isn’t shocking…ever since I developed a modicum of social awareness I’ve realized I’m not representative of most people, but still. Why bastardize the meaning of something as beautifully pure and precise as a number?

Why bastardize the meaning of something as beautifully pure and precise as a number?

The Forecast Calls for…Confusion

A few pages later, Superforecasting provided another example that threw me for a loop.

This sort of primal thinking goes a long way to explaining why so many people have a poor grasp of probability. Some of it can be chalked up to simple ignorance and misunderstanding – like people who think that “a 70% chance of rain in Los Angeles” means “it will rain 70% of the day but not the other 30%” or “it will rain in 70% of Los Angeles but not the other 30%” or “70% of forecasters think it will rain but 30% don’t”.

WTF!!!

But there is something much more fundamental underlying mistakes like these. To grasp the meaning of “a 70% chance of rain tomorrow” we have to understand that rain may or may not happen, and that over 100 days on which we forecast chances of rain, if our forecasts are good, it should rain on 70% of them and be dry on the rest. Nothing could be further removed from our natural inclination to think “It will rain” or “It won’t rain” – or, if you insist, “Maybe it will rain.”

Note: In Superforecasting, this example is followed by a footnote that references Risk Savvy: How to Make Good Decisions, by Gerd Gigerenzer…so maybe check that out.

So…there are people, in the general population, who don’t know how to interpret weather forecasts? This is a failure of our education system!

I’m genuinely at a loss…language is meant to provide common ground, common meaning, a way for us to exchange thoughts, ideas, and information by using terms with agreed-upon meaning. It’s like money: we all agree on the value. But things apparently go awry when numbers – specifically, when probabilities – are involved.

Update 2018-04-25: Here’s a wonderfully topical xkcd!

Language is meant to provide common ground, common meaning, a way for us to exchange thoughts, ideas, and information by using terms with agreed-upon meaning. …But things apparently go awry when numbers – specifically, when probabilities – are involved.

OK, so what?

Maybe you’re wondering why I’m all riled up; or maybe you’ve already concluded that I’m crazy.

In some respects, I’m a professional communicator: I’ve worked for years in marketing, so I have to communicate information outward in the form of brochures, datasheets, presentations, videos, whitepapers, position papers, etc; I also communicate information inward, in the form of quantified market intelligence, decision-making analysis, and so on.

Precision and accuracy are things I strive for (y’know, except when a little ambiguity is helpful). I realized upon reading the passages above that one of my fundamental assumptions – that we all use numbers in the same way, with the same meaning – is flawed.

So what does it mean for me? Well, it means that whenever possible and prudent, I need to take even greater care when using numbers. I can’t take for granted that my audience shares my meaning. Often, a little misinterpretation or fuzziness won’t matter, but I’m concerned about the times that it does, and I think it’s worth getting right.

About Lee Brooks

Lee Brooks is the founder of Cromulent Marketing, a boutique marketing agency specializing in crafting messaging, creating content, and managing public relations for B2B technology companies.