Cognitive diversity is a good thing. We need it if we are to avoid the pitfalls of groupthink, and because - as Hayek said - adequate knowledge of a complex world cannot be known to a single mind. I therefore had sympathy with Dominic Cummings call for "weirdos and misfits" to join Number 10.

And yet one of the first such hires has ended in failure with Andrew Sabisky's departure following a row about his support for eugenics.

Which poses a question. Why is it so hard to achieve cognitive diversity?

In Cummings' case one problem is simply the poverty of thought on the right. Intelligent thinking does not come from unaided individuals, but from traditions. And these traditions have withered on the right: free market economics, for example, has pretty much died. Without such moorings, original thinking can be crankery.

A comparison with Thatcher will demonstrate this. Yes, some of her coterie flirted with eugenics: Keith Joseph fretted that "the balance of our human stock is threatened". But such ideas were sidelined because Thatcherites had other, fish to fry. They were trying to implement an agenda inspired by Friedman, Hayek and Popper. It is a long time since the Tories had influences of similar eminence.

Cummings, though, has another problem. Pretty much nobody of genuine ability would want to take an insecure job in a city that is too expensive to live in. And nobody in their right mind wants a boss who says things like "I'll see half of you next Friday" or "I'll bin you within weeks if you don't fit." What's remarkable, then, is not that Cummings should attract the overconfident and unemployable, but that he should have any chance at all of hiring anybody who isn't.

But, but, but. These are not the only barriers to cognitive diversity. Ours is an age of limited intellectual breadth. Many worthwhile perspectives are rarely heard: free market pessimism; small state Keynesianism, analytical Marxism and so on. There is poverty of thought on the left as well as right - especially among some of those with high profiles.

One reason for this lies with our media. The BBC's "grid" squeezes out too many voices and nuance, and reduces issues to simple tribal binaries. As an anonymous BBC employee has written:

Broadly speaking, there were only two types of guests for producers to choose from: Brexiteer evangelists or slick and power-hungry London Remainers. That the discussion surrounding Brexit should be conducted from these two extremes has become accepted as a sort of national truth, but it certainly doesn't reflect the real make-up of Parliament or the country.

The point applies beyond Brexit.

Academia must also take some blame here. A friend recently showed me a rejection letter from reviewer 2, who complained that he couldn't understand a section of the paper because it was "purely descriptive." What he meant is that there are rigid rules one must follow if one is to be published, and he was damned if he was going to understand anything outside those rules. Such performative stupidity is a great recipe for groupthink. And the time academics must spend obeying those rules is time they cannot spend in more productive thought. This is one reason (of several) why the UK has suffered the near-death of the public intellectual.

There is, I regret to say, more. Those rightists who whine about the mentality that has driven Sabisky out have a point. Too many of us are sanctimonious narcissists. One of my favourite theories here was suggested by Richard Sennett back in 1970:

The threat of being overwhelmed by difficult social interactions is dealt with by fixing a self-image in advance, by making oneself a fixed object rather than an open person liable to be touched by a social situation...The jarring elements in one's social life can be purified out...Threatening or painful dissonances are warded off to preserve intact a clear and articulated image of oneself. (The Uses of Disorder p6, 9, 11)

He was writing about the city, but his theory applies to social media too*. And it is rank hypocrisy to think this is found only on the left. The right has its own pitchfork mobs too.

Now, some of me is relaxed about Sabisky's departure. The problem with eugenic ideas is not that they are offensive but that they are tiresomely unoriginal, unscientific and - for all decent purposes irrelevant. But another part of me is disquieted, because it draws attention to a big problem - that we lack both the institutions and culture which are necessary for proper cognitive diversity.

* One manifestation of the desire to ward off dissonances is, ironically, eugenicists need to purge undesirable elements from society.