One of the stronger defences of free markets is that they act as selection mechanisms. To the allegations that stock markets aren't efficient because investors are irrational or that firms can't maximize profits because bosses don't know what they're doing, the reply is that this might be so, but the market eliminates egregious irrationality and selects for those firms that have stumbled upon profitable strategies.

This, though poses the question: how does selection really work? A new paper by Pascal Seppecher and colleagues at the University of Paris gives one answer. They show that selection can lead to "wild fluctuations and deep downturns". For example in good times, the market selects for companies who borrow to expand because these take market share from their cautious rivals. However, high leverage exposes firms to the risk of bankruptcy when an adverse shock occurs, which leads to a sharp downturn as they try to deleverage. You needn't look far for an example of this; it describes banks behaviour in 00s.

The problem here is that, when faced with selection pressures, firms face a trade-off between what James March called (pdf) exploration versus exploitation. The firm that maximally exploits existing conditions will be maladapted when those conditions change. The firm that invests in exploring alternative strategies might survive change, but at the expense of not maximizing short-term profits.

To put this another way, the best chance of survival in a changing environment is to pursue mixed strategies. But these - by definition - do not maximize "shareholder value" in the short-run. And if all firms in an industry pursue the same strategy, we can have massive systemic instability, as the banking system showed us (pdf).

It's not just change in the environment over time that can lead to maladaption, corporate death and downturns. A similar problem occurs when businessmen move from one environment to another. For example, the market in sportswear selected in favour of Mike Ashley's ruthless cost-cutting. But that strategy when applied to Newcastle United proved less successful*. Or to take a more outlandish example, the market selects in favour of property developers who "restructure" their debts and dodge taxes. But it's not so clear that such strategies are a good way of handling the government's finances.

In fact, there's another problem here: markets might select not for sensible behaviour but for lucky chancers. As Armen Alchian wrote in a brilliant paper (pdf) way back in 1950:

The greater the uncertainties of the world, the greater is the possibility that profits would go to venturesome and lucky rather than to logical, careful fact-gathering individuals.

This is surely true in financial markets. Bjorn-Christopher Witte has shown that risk-taking fund managers can attract more inflows and higher salaries than more cautious but better managers. Economists at Cass Business School corroborate this. They show that just six months of good performance (far too short a period to demonstrate skill rather than luck) can attract inflows into funds - and those funds go on to under-perform.

In a similar vein, Brock Mendel and Andrei Shleifer show that investor can chase (pdf) noise - a process which can give big profits to the irrational noise investor who bought over-priced assets early enough.

Now, in saying all this I don't mean to reject the analogy between markets and natural selection. Quite the opposite. It's a good one - especially but not only in financial markets. We should push it further, and ask: what exactly is it that the market selects for? How fierce is the selection process; the existence of a "long tail" of badly managed firms (pdf) suggests it isn't always and swiftly brutal? And: how vulnerable are today's survivors to the danger of a change in market conditions?

These answers will of course vary from market to market and time to time. But that's the point. Economics is (or should be) an empirical discipline. Windy talk about "optimality" isn't good enough. Sometimes, the invisible hand gets the tremors.

* Yes, Ashley did invest in the club last season, but that change came too late.