The inequality book that shaped a decade is coming apart

Wilkinson and Pickett's The Spirit Level is "meh"

More than 15 years ago, in 2009, researchers Richard Wilkinson and Kate Pickett published a best-seller that purported to rigorously document all the myriad negative effects inequality has on society. Everything from physical and mental health, trust and violence, education and drug abuse was said in The Spirit Level to be importantly negatively exacerbated by high and rising inequality, at least among rich nations. Inequality leads to societal deterioration virtually across the board, and the effects are quite large.

Many loved the book, arguing that it vindicates activism and policies targeted to lowering it. Inequality apparently isn’t just bad in itself, it also (and primarily) leads to bad social outcomes.

I’m not overstating its reception by saying “many people” loved it. The book has been cited over 6,000 times in peer-reviewed journals, policy briefs, and popular outlets. On Amazon, sales are still going surprisingly strong. It’s not doing bad on YouTube, either. Wilkinson and Pickett said last year that they’d been invited to give “well over 1,000 lectures and conferences.” (They also claim that inequality’s negative effect has only grown in the meantime: “Fifteen years later, things have only got worse,” they say.)

Immediately after publication, however, critiques started popping up as well. Quite a few critics took issue with the fact that The Spirit Level mostly presents simple bivariate scatterplots (see below), which are practically useless for any inferential purpose. There’s so much omitted-variable bias (and some potential reverse causality), that these are fun descriptive correlations, but there’s absolutely no causality at play.

Others excoriated the authors for their unnecessarily small sample size (only 23 developed countries internationally), they were unimpressed by how they handled outliers, took issue with the variables they used, or had other reasonable data complaints. (Note, though, that Saunders’ own counter-analysis is also quite problematic for similar reasons.)

In truth, however, the scatterplots – no matter how uninformative by themselves – were intended for illustrative purposes, as this was a general audience book. The Spirit Level cites a large number of studies (including meta-analyses) that use multivariate regression modelling or in certain cases even somewhat more sophisticated approaches. I don’t think that gets Wilkinson and Pickett all the way to actually establishing causality, and the book certainly can be picked apart in this regard, but I feel some of the early critiques were a bit overwrought.

To my mind, the fairest, most cool-headed assessment came from Karen Rownlingson in 2011. In her long report, she demonstrated the following (the numbering is mine):

(1) The literature shows general agreement about a correlation between income inequality and health/social problems.

(2) There is less agreement about whether income inequality causes health and social problems independently of other factors, but some rigorous studies have found evidence of this.

(3) The independent effect of income inequality on health/social problems shown in some studies looks small in statistical terms. But these studies cover whole populations, and hence a significant number of lives. …

(4) Not all research shows an independent effect of income inequality on health/social problems. Some highlights the role of individual income (poverty/material circumstances), culture/history, ethnicity and welfare state institutions/social policies.

So, the negative correlation is there in most (though not all) cases; we aren’t really sure whether it’s causal (although there’s some evidence of causality); the “effects” are statistically and in relative terms small but can be significant in terms of the absolute number of lives.

What have we learned in the meantime?

That was almost 15 years ago. In recent years, I’ve come across a number of reviews and meta-analyses that shed further light on these issues. Some topics covered by The Spirit Level haven’t received any particularly impressive, large-scale research over the past 10 years. Other themes have received it and some of that research has even corroborated the story as it was presented in the book (or at least in the more careful Rowlingson report). On yet other topics that have been investigated, the findings have been less kind to the book. In this post, I’ll cover the latter, but I’ll write about the book again in a future post, where I’ll go through some of the research that I won’t be mentioning today.

Self-rated health and all-cause mortality

A relatively big recent review focused on multilevel studies (primarily from the developed world) found there is a small association between inequality and both self-rated health and mortality. The overall effect sizes are very small, and a large chunk of included studies are at serious risk of bias, as diagnosed by the researchers. So I don’t think we can conclusively say that the (small) effect exists.

Though the paper is titled “Causal Assessment of Income Inequality on Self-Rated Health and All-Cause Mortality,” it actually relies almost exclusively on earlier-generation observational studies with controls for observed confounding. The researchers try to get around this by meticulously grading how well the studies were controlled and whether they can, perhaps, plausibly tell us anything about causality. After doing all of that they find, as I already mentioned, that a lot of research is of very poor quality for the purpose. Thus, they conclude:

The available evidence does not suggest causality, although it remains methodologically flawed and limited, with very few studies using natural experimental approaches or examining income inequality at the national level.

Life satisfaction and mental health

The largest meta-analysis to date finds that, after correcting for publication bias, there’s no general association between income inequality and life satisfaction or mental health. In fact, in poor societies, more inequality is associated with a higher risk of depression, but in rich societies, there’s nothing. That’s the reverse of what Wilkinson and Pickett expected (said) in their book.

Another interesting moderation “effect” has to do with inflation. Where inflation is low and inequality is high, life satisfaction is actually somewhat higher than usual. But where both inflation and inequality are high, people report lower life satisfaction. The researchers speculate this has to do with financial stress (under high inflation) and upward mobility (under low inflation). Where prices are stable and the economy is more predictable (as is usually the case in the developed world), higher inequality can serve as a signal of economic opportunity and upward mobility, so people feel better than they do under low inequality. Again, this is a finding that’s not quite in the spirit of The Spirit Level.

(Oh, is any of this causal? We don’t know because it’s just standard observational research.)

Violence and crime

One recent meta-analysis focused on Europe confirmed that income inequality is related to more crime. However, the association is small. And when I say “small,” I mean really small. It accounts for “only 3% of the variance in crime outcomes.” More specifically, the relationship shows up for Eastern/Northern Europe but not Western/Southern Europe. Here’s the author:

the meta-analysis results in the current study support the position that, compared to other regions of the world, income inequality in Europe has a much smaller impact on general crime and homicide.

And Europe is very important, because for Wilkinson and Pickett, inequality’s bad effects should be visible (only) in developed countries, like the ones in Europe!

Another, larger meta-analysis corroborates the impression that nothing much is going on between inequality and crime. As the researcher says:

The findings indicate a statistically significant but economically insignificant true effect of inequality on crime, ranging between 0.007 and 0.123 using UWLS FAT-PET and advanced methods. In essence, if there is an impact of inequality on crime, it is, at best, minimal. Additionally, there is some limited evidence suggesting positive publication bias.

As they further qualify the “effect” size in the paper:

the true values of the partial correlation coefficients – net of publication bias – are statistically but not economically significant. They are in the range 0.007–0.123, which represents non-existent or small effects, according to the guidelines provided by Doucouliagos (2011).

Democracy and democratic integrity

I don’t remember Wilkinson and Pickett writing a lot about inequality eroding democracy in their book. However, in a recent report they issued – The Spirit Level at 15: The Enduring Impact of Inequality – they do have a section on democracy. There, they claim that “rising income inequality … erodes … democracy. … When inequality worsens, so do trust, democracy, [and many other things].” They then proceed to show how inequality is statistically significantly and negatively associated with countries’ democratic scores in a cross-sectional scatterplot of around 20 societies.

That’s bad. You might remember that I showed there’s no robust evidence for the connection (let alone a causal connection) between the two variables. This is true both if we look at the extant research literature or if we go to the data directly ourselves.

Beware claims of causality

Note also that this recent report by the authors, in contrast to their book, relies solely on bivariate scatterplots, even though causal language is used throughout. Inequality, the authors claim, “drives,” “hurts,” “determines,” “leads to,” “reduces,” “erodes,” etc. That’s doubly bad.

As I said at the beginning, there are many other topics covered in the The Spirit Level, and for some of them, the evidence is more encouraging! But some of the key, most attention-grabbing claims from the book have more or less unraveled in light of more (and more systematic) evidence.

Look, I don’t particularly like (high) inequality, nor do I think we should treat it as something irrelevant. For instance, I don’t agree with Deirdre McCloskey that expressing a certain amount of normative concern about inequality is simply to be possessed by “a vague feeling of envy.” I’m a social democrat and thus naturally lean egalitarian. I’m also sure inequality has at least some (even if only small/modest) bad social effects, which makes me like it even less. But let’s not distort the truth. Inequality is definitely not, as Wilkinson and Pickett mistakenly claim, “at the root” of all of our crises.