Stumbling and Mumbling

Outcome bias & cargo cults

chris dillow
Publish date: Tue, 11 Oct 2011, 02:12 PM
chris dillow
0 2,773
An extremist, not a fanatic

Is it time for a Nobel prize for managers? asks Andrew Hill. The Platonic essence of a QTWTAIN, says John.
He's right. The reason for this is that our judgement of managers is clouded by the outcome bias.
When management theorists talk about bosses they very often don't take one early in his career at a firm, and say: 'This is his strategy. It's good because x, y, z. He's a good manager. His firm will succeed.' Instead, they use hindsight. They take a firm that's succeeded and infer from this that the CEO and the strategy was good. Or they see a firm struggling and infer the CEO is bad.
This is the outcome bias; we judge managers not by the quality of their work, but by their results. In this sense, management differs - to least some degree - from most of the disciplines which get Nobel prizes. We can assess the work of, say, Chris Sims, independently of anything it has led to. But we rarely assess managers independently of results. This is wrong because it ignores the role of luck. A poor boss could get lucky and succeed and a good one could get unlucky and fail.
Duncan Watts' Everything is Obvious - which I strongly recommend - gives an example of how tricky it is to slough off outcome bias. He compares the iPod and Betamax video tapes:

The iPod was exactly the kind of strategic play that the lesson of Betamax'should have taught them would fail. The iPod was large and expensive. It was based on closed architecture that Apple refused to license, ran on proprietary software, and was actively resisted by the major content providers. Nevertheless, it was a smashing success. So, in what sense was Apple's strategy better than Sony's? Yes, Apple had made a great product, but so had Sony. Yes, they looked ahead and did their best to see which way the technological winds were blowing, but so did Sony. And yes, once they made their choices, they stuck to them and executed them brilliantly; but that's exactly what Sony did as well. The only important difference'was that Sony's choices happened to be wrong while Apple's happened to be right.

This sort of thing is, of course common in sport. I've described how Arsene Wenger is the victim of it now. And a new paper shows how NFL coaches are prone to the bias. The authors show that coaches tend not to change their teams' offensive strategies after victories, even if these were narrow, but they often change them after defeats, even if the defeat was very narrow or if it were due to defensive errors rather than attacking ones. They say:

One can easily misinterpret a favorable outcome as justification for a given strategy, overriding more subtle evidence to the contrary.

All of this raises a question. Could it be that management is, to some extent, like a cargo cult? It performs certain peculiar rituals, after which nice things sometimes happen and sometimes not. However its members infer from this not that the rituals are (largely? partly?) irrelevant to the outcome, but instead that some of their performers have skill whilst others do not. And in this way, irrational practices persist, and their luckier practitioners get rich.

More articles on Stumbling and Mumbling
Discussions
Be the first to like this. Showing 0 of 0 comments

Post a Comment