May 29, 2012
Judgment Misfires and Unreliable Narratives
When good judgment processes produce bad outcomes and reliable-seeming stories mislead - the systematic ways our interpretive apparatus fails.
6 min read
Two Kinds of Error
Judgment misfires can occur in two distinct ways. The first is a process failure: the reasoning process itself is flawed. Bad logic, incomplete information, wrong analogies. These failures are, in principle, correctable by improving the process.
The second is an orientation failure: the reasoning process is working correctly but it is operating on a flawed model of the situation. The logic is valid, the information is processed correctly, but the inputs to the reasoning are wrong. The map is wrong, not the navigation.
Most advice about improving decision quality focuses on process failures. It tells you how to reason better, how to check for biases, how to gather more information. This is valuable, but it cannot help with orientation failures. You can reason perfectly from wrong premises and still produce bad conclusions.
The Narrative as Orientation
Narratives are the primary mechanism for orientation. When you encounter a situation, you do not process raw data - you interpret the situation through a narrative that tells you what kind of situation it is, who the actors are, what they want, and what is likely to happen next.
This interpretation is fast, largely automatic, and usually correct. Pattern matching to past experience is a powerful cognitive tool. The problem is that the speed and automaticity of narrative interpretation is indistinguishable from accuracy. A wrong narrative feels exactly like a right one.
Unreliable narratives - narratives that misidentify the situation - produce orientation failures of a specific type. The reasoner is working correctly. The narrative premise is wrong. And because the wrong premise feels like accurate perception, the error is not apparent until events diverge from the narrative's predictions.
The Most Reliable Misleaders
Some narrative types are more reliably misleading than others.
The agent narrative is one. When we interpret a situation through an agent narrative - there is an actor with intentions who is causing events - we impose a structure of will and purpose on situations that may be produced by impersonal processes. Market dynamics, organizational behavior, and social trends are particularly prone to agent misinterpretation. Finding the responsible individual is satisfying and wrong in proportion to how systemic the phenomenon actually is.
The precedent narrative is another. "This is like X" is a powerful interpretive move that activates everything you know about X and applies it to the current situation. The problem is that situations often differ from their apparent precedents in ways that are not visible at the point where the narrative is assigned. The financial crisis analogy from 2008 applied to a different financial stress may import the wrong response package.
The arc narrative - "we are in the X stage of a familiar progression" - can also mislead by suggesting that the current situation will follow a known developmental sequence when the underlying conditions have changed.
Reliability Signals
The unreliable narrative often announces itself, if you know what to listen for. The clearest signal is strong confidence in a situation that contains significant ambiguity. Confident clarity in ambiguous situations should trigger suspicion that a simplifying narrative has been applied that may not be warranted.
A second signal is the absence of alternative interpretations. If you have not considered what the situation looks like under a different narrative, you have not adequately tested your current one. The failure mode is not wrong reasoning but the refusal to entertain alternative premises.
A third signal is prediction failure - situations where the narrative predicts outcomes that do not materialize. The narrative might explain the failure ("unusual circumstances", "anomalous event") rather than updating. This defensive explanation is itself a signal that the narrative is being protected rather than tested.
The Discipline
The discipline for unreliable narratives is not to stop using narratives - you cannot. It is to treat your current interpretive narrative as a hypothesis rather than a perception.
A hypothesis you test. You generate predictions from it. You look for disconfirming evidence. You actively consider alternative hypotheses rather than dismissing them.
A perception you trust. You reason from it. You defend it when challenged.
The switch from perception mode to hypothesis mode for your interpretive frames is the cognitive shift that reduces orientation failures. It does not eliminate them - wrong hypotheses are held, tested, and eventually replaced, but the process is slower than everyone would prefer. It just makes the errors correctable rather than self-reinforcing.
The point at which a situation's events stop being explainable by your current narrative - when the defensive explanations start proliferating - is the point at which hypothesis replacement should begin. Waiting for that point requires a specific kind of intellectual honesty about when the narrative is working and when it is being protected.