Incompleteness and problem solving

“Almost all real problems are incompletely and imperfectly specified.” – John Kay

Did you ever notice that to solve most real-world problems of any complexity, we have to find ways to simplify the situation? Well, economist John Kay noticed, and included a chapter on the subject in his great little book Obliquity.

This chapter, Incompleteness: how we rarely know enough about the nature of our problems, explains why the direct approach to problem solving often fails; put simply, the direct approach requires perfect (or nearly perfect) knowledge, and that is in very short supply.

As Kay says, “Almost all real problems are incompletely and imperfectly specified¹, and to tackle them we have to try to close them in some way. Closure means deciding what to bring in and what to leave out. Even when faced with what appear to be simple choices, we have to create our own description of the problems we try to solve.” (Obliquity – p98)

“Even when faced with what appear to be simple choices, we have to create our own description of the problems we try to solve.”

When scientists were teaching computers to play chess, they had to artificially close the problem by using advanced algorithms to decide which relatively few move paths to explore – it was simply too computationally intensive to fully explore the solution set. In a more prosaic manner, when we face a problem to solve in life or in work, we have to apply our own simplification algorithms, because there are simply too many unknowns.

Closing a problem isn’t limited to discarding information and simplifying variables, though: “Closing a problem means deciding what information should be discarded. It also means deciding what information should be added. In even the simplest problem, our analysis is based on interpretation of the context.” (Obliquity – p100)

Things are tricky enough when we’re dealing with problems of the present, but things are worse when we’re looking into the future: “Problems whose solution requires us to predict the future cannot, even in principle, be completely closed… Most of what will be important in the future is outside our knowledge; it exists only in the future. The direct approach demands a capacity for prediction that we can never possess.” (Obliquity – p103-104)

“Problems whose solution requires us to predict the future cannot, even in principle, be completely closed.”

C’mon, not even in principle? That’s unfortunate, but true. It is just impossible to know precisely how our environment will change over time, so the nature and the context of the problem with which we are faced will shift.

So why do we spend so much time convincing ourselves that we have perfect (or nearly perfect) knowledge, or that we have complete (or nearly complete) control over our problem domain? I think the answer has three elements: first, we find it very unsettling to admit that we’re at the whim of the world; second, we’re a supremely arrogant species; third, we’re wonderfully ignorant of our own ignorance.

However, simply acknowledging the truth puts us on a practical path to better problem solving. As Kay says, “…the key point is not that we mostly fail to anticipate the answers, rather that we mostly fail to anticipate the relevant questions.” (Obliquity – p103)

When we ask the right questions, we recognize the bounds of our own knowledge and influence, and we will be less likely to pursue an ill-suited direct approach to a realistically incompletely specific problem.

===

¹As the song goes, “I got 99 problems, and they’re all imperfectly defined.”

About Lee Brooks

Lee Brooks is the founder of Cromulent Marketing, a boutique marketing agency specializing in crafting messaging, creating content, and managing public relations for B2B technology companies.