Several tweeters, such as Anna and Hopi, have expressed dismay at the fact that Iain Duncan Smith has kept his job in the cabinet (re)shuffle, despite his gross inadequacies. His survival, however, highlights an important fact - that, sometimes, organizations and markets can actually favour incompetence. There are (at least) eight mechanisms through which this can happen.

1. The wet bed. If a man has pissed the bed, you don't ask someone else to sleep in it. Cameron might well have thought "Universal Credit is an immense clusterfuck which nobody can sort out in a few months. Why waste the reputation and morale of a good man by asking him to try?" I suspect similar thinking led Luiz Filipe Scolari to keep David Luiz on the pitch during Brazil's 7-1 mullering by Germany.

2. A disposition effect. Every bad employee is a bad hire. However, a combination of ego involvement ("this was my decision so it must be a good one") and the endowment effect ("we've got him so he must be good") stops hirers from immediately realizing their error. Just as stock market investors tend to hold onto bad stocks, because of their refusal to admit error, so employers hang onto bad staff.

3. Noise vs signal. In many contexts, feedback about performance is noisy. Ran Spiegler shows (pdf), this can create a market for quacks. For example, if you would have recovered from some ailment anyway but take a homeopathic treatment before the recovery, you can easily convince yourself that the treatment worked. And you'll tell others. A similar mechanism can favour the fund manager who rides a bull market, or the corporate boss who is lucky enough to get a job at a firm with good organizational capital.

4. The devil you know. In many jobs, a worker's ability can only be assessed after he has done it. As Marko Tervio has shown, this alone can generate adverse selection; the mediocrity who has a track record that is just adequate will be preferred to the unproven man of potentially greater ability. In itself, this is a story about markets favouring mediocrities rather than incompetents, but mechanisms 2 and 3 above suggest that the bar for mediocrity might be set so low as to allow idiots to thrive.

5. Survival of the unfittest. Bjorn-Christopher Witte describes how, sometimes, competition between fund managers can encourage reckless risk-taking with the result that lucky chancers rather than the genuinely skilled will thrive. For example, in the early 00s bankers who danced to the music and took risks got big bonuses whilst those who sat it out got sacked. And during the tech bubble money flowed to those managers who thought boo.com and Baltimire Technologies were good stocks, whilst sceptical fund managers such as Tony Dye were fired.

6. Desperation. If people are desperate for a very high pay-off, they'll be attracted to incompetents and fraudsters, as only these are stupid or criminal enough to offer such rewards. As Laurie has said, "sometimes when you're dying of thirst, you have to drink the Kool-Aid." This is why con-artists often prey upon the terminally ill or bereaved. But it also lies behind what I've called the Bonnie Tyler syndrome - the urge (often on the left) for a great hero.

7. Product differentiation. In a wonderful paper (pdf) on the persistence of the market for quack medicines in the 19th century, Werner Troesken points out that the manufacturers of such remedies spent fortunes on advertising and product differentiation. In this way, the failure of one medicine did not discredit the industry, but merely shifted demand to other quacks. I suspect the fund management industry operates on a similar principle. So might politics: insiders want us to think that Paterson, Fox, Hammond or whoever are distinguishable when in fact they are fungible.

8. Like hires like. Senior managers like to hire people like themselves not just because they want yes-men, but because (say) financial people find it easier to assess finance skills and engineers find it easier to assess other engineers. Knowing this, underlings hoping for promotion will cultivate the skills their bosses have, even if their comparative advantage lies elsewhere. The upshot of this, according to Eric Hughson and colleagues, is that organizations can eventually be run by second-rate MBAs whilst technical skills are completely weeded out. Joao Ricardo Faria shows (pdf) that this sort of mechanism can generate the Dilbert principle.

Now, I stress that these mechanisms are not universal, although they are sufficiently widespread to explain the obvious fact that some idiots and charlatans survive and thrive in many organizations. It is, of course, the case that competition does sometimes improve quality. But this Econ 101 view is only a partial truth. The notion that we live in a meritocracy - or even that a meritocracy is possible - is just...