Breaking: Wilkinson admits there is no statistically significant relationship between life expectancy and inequality!

Blogger August has done some nice journalism.

Wilkinson writes:


“These problems look serious only to the uninitiated, unaware of the vast literature. There are now about 200 peer reviewed analyses of the relation between various measures of health and inequality in different settings (see attached review). There have been times in the past where, because of rapid changes in income distribution, cross-sectional relationships have temporarily disappeared only to reappear when the new levels of inequality have had time to work their way through to affect culture and then health. As I said, death rates at older ages which now dominate life expectancy, are likely to be influenced by inequality throughout life. Although I started off working on health in relation to inequality, we now know that health is more weakly related to inequality than many of the other outcomes we look at.

Kate has quickly run through our data to see how the outcomes relate to the OECD inequality figures (see attached file). There are a few changes, some stronger, some weaker, but the vast majority much as before – see attached table. Inevitably things change, there are lag periods and there are different measures of income and income distribution. She has also included correlations for some of the relationships in question using the new inequality measures in the UN Human Dev Rpt 2009.”

What you will notice in his two graphs is that Life expectancy never had a statistically significant relationship to Gini! This is even if you use his 21 cherry picked countries, and even if you use the UN HDI index instead of the OECD.

I encourage everyone to go ahead and klick on the tables. You will see with your own eyes that inequality is not related in a statistically significant way with life expectancy. P value 0.24 means that it is not even statistically significant at a 10% level (usually you require 5%).

Wilkinson is through this table ADMITTING that his main variable, inequality, does not have a robust tie to the main measure of health, life expectancy.

Mission accomplished.

He refuses to admit this in words however, choosing to avoid the subject, and claim you have to be “uninitiated” to want statistically significant results between simple and straightforward variables that his theory predicts.

Wilkinson want to remove Japan from the sample. Even if he does this it is not statistically significant at a 5% level. And what justification does he have to arbitrarily remove one country from the sample, a country with 127 million inhabitants? Can I remove the U.S from the sample when doing obesity? There is a name for this: data mining. There is another name, but let us not get nasty.

I can tell you that this is not how you do science. If you are an undergraduate student, and remove one country with no justification from your small sample in order to get closer to statistical significance, your professor will likely fail you.

Does Wilkinson admit in his lectures and in his book that life expectancy is not linked to inequality?

If he doesn’t, puts up those famous diagrams and gives the audience the opposite impression (“Inequality Kills”) he is committing academic fraud.

Instead we get to know the number of studies does about the issue. There is a “vast literature”. Ooh, I am impressed! What he does not tell you is that many of the of those studies find no effect (I linked to a few as examples in earlier posts). The number of studies tell us nothing. There are hundreds of studies made on taxes and growth, but there is no accepted robust relationship.

Would you believe me if I told you: Taxes are proven to be bad for growth. I have no robust evidence for this, but here are references to 200 studies with mixed findings and endogeniety problems. With so many studies it must be true!

(I don’t quite understand why The Spirit Level pushed all those nice convincing cross country correlations in the first place, if the argument now is not the cross country relationship, but the “vast literature”. Oh well. I must be very “uninitiated” to ask these rude questions).

Of claiming something is true because there are many studies about it will not impress academics. Wilkinston, in choosing to ignore addressing the causality problem in his variables, and in ignoring statistical significance as a standard, has decided he does not need academic credibility. He will target an unsophisticated audience who don’t ask too many difficult questions, such as Social Democratic party activist in need for their ideology to be supported by “science”.

By the way, notice yesterday he told me to look at mortality for working age adults. “look at mortality among… the working age populations”. I did, no statistically significant relationship.

He does not mention why. Ignoring this problem, he changes his story, and today writes that inequality will instead influence the old.

What is good about life expectancy is that you can account for both mortality among the old and middle ages. But OK. Let us use his latest standard: the life expectancy of the old.

I look at remaining life expectancy at 65 for men and women (the OECD does not have a table for both sexes, and since there are many more women alive at 65 we can’t average them easily).

In both cases, the relationship with inequality is not statistically significant, and of the wrong sign (more unequal live slightly longer).

For women:


For men:


Now, since I am super-nice, I will graph life expectancy at 65 in 2005 with inequality 2 decades ago, for all countries that there is data.
For women:


For men:


Nothing, the opposite sign of what The Spirit Level claims again, and not statistically significant.

How many specifications have I run so far? 20? I have done the UN, the OECD, the young, the old. Except infant mortality, a robust relationship just isn’t there.

This sounds like a good life though. I can find not statistically significant variables, and tour around the world selling stories to naive Social Democrats.

The Spirit Level is junk science part deux (updated)

Obs: Don’t forget to read my updated post about the Spirit Level, where the author responds to me.

Obs II: He responded again, and it is as weak as the last response. Go the the bottom of the post.

Obs III: Their claim about innovation cannot be replicated either (scroll down).

The “Evidence” presented in the Spirit Level falls apart very easily if you poke even a little at it.

I first redid the main argument, the relationship between life expectancy and inequality, now using OECD data (instead of UN data). The good thing about the OECD data is that it measures inequality after taxes and government subsidies are taken into account.

Again there is no statistically significant relationship between inequality among 28 OECD countries and life expectancy. The p value is 0.78. As a rule of thumb you need a p value no higher than 0.1, preferably 0.05, to be able to argue that something is statistically significant. The core argument of the book is not significant with the standard measure of inequality. This is a joke.

The country with the highest life expectancy, Japan, has the 9th highest level in inequality measured by Gini after taxes and subsidies. The Spirit Level gets around this “problem” by using a very selective measure of inequality, the ratio of two groups of people, instead of the standard measure in social sciences, Gini , which looks at every member of society, including the poor. Furthermore, they use an “index”, instead of the straigtforwards measure: how long people live.

As every trained economist knows, correlation is not causation. The relationship between life expectancy and income equality is problematic, because of reverse causality, and because third factors (basically all social problems) cause income inequality AND lower life expectancy.

One way to at least mitigate the problem is to not to look at levels, but at change. If the theory of the book is correct, as countries become more unequal life expectancy should fall, relative to countries are are becoming more equal.

Here is the change in Gini for 19 OECD countries where there was data between what the OECD defines as “mid 1980s” and “mid 2000s”. As you see the relationship is again not statistically significant (p 0.35), and again the opposite of what the Spirit Level claims.

The countries whose income distribution became more unequal had faster growth in life expectancy!

Since we have higher standards than the authors of the Spirit Level, let us not pretend this means anything, dreaming up ad-hoc stories how more equality kills people. Probably just a coincidence or an artifact of the complex relationship between inequality and other factors that influence life expectancy (such as GDP growth, human capital accumulation).

Microeconomic studies can mitigate (but not solve) the causality problem, by controlling for variables. I did a quick search of the literature. It seems that microeconomic studies, including a thorough study in Sweden and one in Norway that tried to reduce endogeneity problem do not find a relationship between inequality and mortality. Same result in Finland and the U.K.

The Social Democrats in Sweden need to learn about requirements in empirical social sciences, such as statistical significance and robustness. Most importantly, they need to learn about causality. If the Social Democrats, the postmodernists and the Social liberals had not destroyed the school system, basic foundations of logic would be taught at least in high school.

Let me give another example of Social Democratic inability to understand causality: crime and punishment. The Swedish left, including often leftist libertarians, believe the have “proven” that long prison sentences does not reduce crime, because countries with longer sentences have more crime. In Swedish leftists logic, this means that long punishments CAUSE high crime! Just look at the U.S!

But punishments are a costly countermeasure to crime: you only increase punishments when crime becomes a problem. Countries with high underlying crime are forced to respond by increasing punishments. If these countries had less strict punishment they would have even higher crime.

The analogy that I use is head-ache pills and head-aches. We don’t go around claiming that aspirin causes head-pain, just because people with head-ache are more likely to take aspirin. Why would you accept that the positive correlation between crime (the illness) and punishment (the medicine) implies that the medicine caused the illness?

Many leftists are smart, but they have not been taught how to think in a stringent way (worse, they have been ideologically indoctrinated against objectivity and reason).

The first thing someone should have asked in the seminar presenting The Spirit Level is “how did you get around the endogeniety problem?”. Instead the left turned it into a religious meeting, embracing the claims made in the book, even though they are not supported by real evidence.

What you need to get around complex relationships such as prison length and crime, or inequality and life expectancy, is some form of exogenous treatment. You need controlled experiments, quasi-experiments, or good so called instruments.

In crime versus punishment scientists worked hard at finding these type of experiments, such as the recent prison reform in Italy which almost randomly gave different prisoners different punishments for the same crime. This experiment showed us that punishment does reduce crime. One such study with a clear identification is worth 1000 correlations, when endogeneity is a problem. Social Democrats would understand this if they understood causality.

I don’t want to single out Social Democrats either. The Swedish Social Liberals (Folkparti) are almost as bad, as evident by the editorial page of DN.

More on this:

In order to put the nail in the coffin of these charlatans, I further investigate the claim by Spirit Level that inequality causes bad health.

For 28 OECD countries, I measure 11 health outcomes from OECD Health Data 2009, and compare them with inequality as measured by post taxes and transfers Gini coefficient defined as “mid 2000s” by the OECD.

All health measures are the average for the years 1998-2007, in order to correspond with mid 2000s. I include more years in order to be generous to The Spirit Level, if I only look at 2005 their claims do even worse (for example infant mortality is no longer statistically significant). The reason is that for some of these measures some countries have missing variables in many years.

The 11 health measures are:

1. Life expectancy at Birth
2. Infant mortality, Deaths per 1,000 live births
3. Suicides, deaths per 100 000 population
4. Cerebro-vascular diseases, deaths per 100,000 population
5. Cancer, deaths per 100,000 population
6. Diseases of the respiratory system, deaths per 100,000 population
7. Diabetes, deaths per 100,000 population
8. Tobacco consumption, % of population
9. Alcohol consumption, Litres per capita
10. Obesity, percentage, percentage of population
11. Overweight, percentage, percentage of population

Out of 11 health measures, Inequality only had a statistically significant relationship at the generous 10% level with two variables: Infant mortality (p 3.2%) and obesity (p 6.1%).

(Obesity is driven entirely by one fat and unequal country, the U.S. Without the U.S the p-value is 34.4%.) Infant mortality is the one variable I would definitely give them. Here are these two graphs.


Again, life expectancy at birth has no statistically significant relationship with inequality (p value 74.1%). I have already included the graph for 2005, the 1998-2007 average is close to identical. Bear with me as I show you the graphs for the other 8 variables.

Remember the nice, convincing graphs The Spirit Level presents to it’s readers? You will not find them here, when reproducing some of the claims.

Suicides are negatively related to inequality, the more unequal the country, the fewer suicides. The relationship is not statistically significant however (p 10.9%)


Deaths in Cerebro-vascular diseases are not statistically related to inequality (p 31.7%).


Deaths from Cancer are not statistically related to inequality (p 60.1%). In fact countries with higher inequality have fewer deaths in cancer.


Deaths from Diseases of the respiratory system are not statistically related to inequality (p 19.4%).


Death from Diabetes is not statistically related to inequality (p 16.3%).


Smoking is not statistically related to inequality (p 76.2%). In fact people smoke slightly less in countries with more inequality.


Drinking is not statistically related to inequality (p 59.2%). In fact people drink slightly less in countries with more inequality.


The share of overweight people is not statistically related to inequality (p 55.5%). (remember, obesity was related, this is the share of overweight but not obese people).


2 out of 11 ofvariables being statistically significant is not particularly impressive at all. 4 out of the 11 health measures have the opposite sign that they predict, with unequal countries doing better than equal ones! Yet the Spirit Level has through selective presentation given the impression that inequality is very strongly related to health outcomes. They must have known how weak the unerlying data was. I conclude that they are simply misleading their readers.

One more, just for fun. The under-appreciated blogger Dan Nordling noticed an odd claim by The Spirit Level, that more than 25% of Americans and Brits were mentally ill, compared to only 10% of Germans and Italians. Such enormous disparities in illness between similar countries seems oddly high, quite frankly. I had never heard anywhere that Brits had two and a half time more mental ilnes than Germans.

I tracked down international comparisons of Mental Health from the WHO. The WHO measure, age standardized DALY per 100.000, for what they refer to as “Neuropsychiatric conditions”, which includes:

1. Unipolar depressive disorders
2. Bipolar disorder
3. Schizophrenia
4. Epilepsy
5. Alcohol use disorders
6. Alzheimer and other dementias
7. Parkinson disease
8. Multiple sclerosis
9. Drug use disorders
10. Post-traumatic stress disorder
11. Obsessive-compulsive disorder
12. Panic disorder
13. Insomnia (primary)
14. Migraine

The WHO results are much more intuitive. the U.K has 10% more mental illness as Germany, not two and a half times more. The U.S has 25% more mental illness than Belgium, not more than twice as much (the U.S has much more drug use). Does the Spirit Level stand closer scrutiny?

First, here is the graph from The Spirit Level, too convincing by half as they say.


Here is the graph linking Gini from the OECD to mental health problems.


As you notice, we have another variable that is not only not statistically significant (p value 28.6%), but that goes in the opposite direction of what The Spirit Level claims: according to WHO data more unequal countries have less mental illness!

Again, the lesson is: Don’t trust anything written by these people. They believe in their story so much that they are willing to fudge the data if that is what is needed to convince the pubic.

Update II

It just gets better and better. Just one more for the heck of it.

Another fishy looking claim in “The Spirit Level” is that more equal countries are more innovative. Here is another one of those really convincing graphs:


Notice that the United States is one of the least innovative countries according to “The Spirit Level”. Now, no matter how dogmatically leftists you are, it is hard to claim that the U.S., the most technologically advanced country in the world, winner of 60% of scientific Nobel prizes in the post war period, is one of the least innovative advanced nations on earth, no more innovative than Portugal.

So I went to the homepage of the World Intellectual Patent Organization. They had themselves calculated a measure of patents adjusted for population “Resident patent filings per million population (1995-2007)”.

Here is the relationship between patents and inequality (the outliers are Japan and South Korea, I knew from before that Japan is much more patent intensive than others, although I don’t know why).


The correlation is not statistically significant, and once again goes in the opposite direction of what the books claims (inequality is correlated with more patents).

Another one of the books graphs disappear when you try to independently replicate it. If I was not afraid of taxing the patience of my readers, I think I could do this for days, deconstructing this house of cards.

In the comments I suggested people simply refute Wilkinson by poiting out that there is no statistically significant relationship between life expectancy and Gini in the OECD.

I should say that this is the easy question. He does not have a response to it, so they hide it in their book. Everyone will understand this critique.

It is not the true core of the problem. A trained economist would ask him: How do you account for the endogeneity problem. This means: how does he establish causality when he has variables that are related in complex ways.

He claims Inequality causes bad health outcomes. But bad health can cause inequality. More importantly, third factors, basically all social problems, simultaneously lead to low health outcomes and to low income, causing what is referred to as a spurious correlation.

For example in Sweden a social problem is poor integration of immigrants. The unemployed immigrants in Rosengård have lower health outcomes, they smoke more, use more drugs, are often victims of crime. And they have lower income than other Swedes. In the data this looks like a correlation between inequality and bad health. But it is just a correlation, not causality.

It is not the fact that Swedes in in Malmö and Vellinge are rich that is CAUSING Rosengård dysfunctional, as Wilkinson claims. If Vellinge had an economic crisis and became poor, this would have no effect of health in Rosengård (or probably a negative effect, since the hospitals would have less money). The causality is more complex.

People in corrupt southern Italy have lower health outcomes and lower economic outcomes than North Italians. The casual link is that Mafia, lack of trust, and low education make south Italians poorer, and it makes them have lower health outcomes.

The Spirit Level thinking instead childishly interprets the complex relationship that North Italians are rich makes South Italians unhealthy, because of the stress of knowing they are doing worse than North Italy.

Because of the problem of what econometricians call reverse causality and missing variables, the correlation studies used in the Spirit Level are not accepted as scientific evidence by trained economists.

A very bad sign for their hypothesis is that as soon as you put some controls the relationship between inequality and health vanishes. The micro-studies with controls that I linked to find no (and some factors are unobservable, so we cannot even control for them). Using levels for OECD countries we have no statistically significant relationship. Using levels for UN we still have no statistically significant relationship, and even find the opposite of what the book claims. Using change we find no statistically significant relationship, and the opposite of what they claim.

This does not mean Wilkinson is wrong. It just means he has no evidence for his hypothesis. Wilkinson and people who think inequality causes lower health (for example through stress) need to find exogenous experiment to verify their hypothesis. Until they have done that we cannot accept they claims as science. But not only have they not done that (to my knowledge), they are going ahead and selling their story as if they had evidence!

This is deeply unethical, because ordinary people trust academics. Ordinary people do not necessarily know about endogeneity problem in empirics. They think if a Social Scientists claims they have 800 studies, and puts up some correlations on a powerpoint, than the weight and status of science is behind him. Naïve DN readers trust “science”.

What the Spirit Level is doing is essentially fraud.

UPDATE

Wilkinson reponds again:

Wilkinson:

“Sorry – far to busy to follow up this stuff. I know the right will be doing everything to get rid of our material but we don’t have time for blogs etc now. I notice country names are not given and assume he has included countries which are not among the richest in the world and so should control for GNP before looking at inequality. Should also try looking at mortality among infants and working age populations.”

My answer:

“* I do include country names. My first regression is EXACTLY the same 21 countries Wilkinson uses, the only difference is that I use Life Expectancy, instead of the “index” they have built. The relationship is not statistically significant.

* I have run a simple regression of life expectancy on Gini and Per capita income (all data from OECD for mid 2000s and 2005) for the 28 OECD countries:

p value for Gini: 0.783
p value for per capita GDP: 0.55

This means neither value is even close to being statistically significant (why is GDP not statistically significant? One reasons is that the OECD countries already come from a selected sample of high income, per capita income is a restricted variable).

Now Wilkinson’s: 21 countries
p value for Gini: 0.451
p value for per capita GDP: 0.986

No statistically significant. Not even close.

* I have already run infant mortality, and have written that infant mortality (and to a lesser extent obesity) are the only health variables out of (now) 13 investigated that are linked to inequality in a statistically significant way.

* I traced down adult mortality for people aged 15 to 60 from the WHO. It not even close to being related in a statistically significant way to inequality. For the full 28 sample the p value is 0.63, for his selected sample of 21 countries the p value is 0.67.


Please don’t waste my time more with wild goose chases that just weakens you own story, and answer the direct questions: Why is life expectancy not related in a statistically significant way to inequality, if inequality is a major killer?”

By the way, notice that he had time for a 750 word response as recent as yesterday. How convenient that he doesn’t have time now…

The Spirit Level is junk science (updated: Wikinson Responds)

There is a new book called The Spirit Level, which claims to prove that most social problems are directly caused by income inequality. So if people in the U.S are more obese than Swedes, it is because middle class Americans they are stressed because they earn less than rich Americans, and eat more (not joking). The “proof” for this is cross country correlation between inequality and various measures.

As a social science student, I am taught to take causality seriously. If you want to argue that inequality causes obesity, you have to actually prove it. Correlations are not scientific proof.

From a theoretical perspective, social problems cause inequality, or are often both caused by deeper ills. In fact it is difficult to think about a social problem that does not cause inequality! Low level of human capital makes your more likely to not take care of your health and causes low income. Bad norms cause crime and low income. Drug use causes problems and low income. Family disruptions causes social problems and low income. And so on.

If The Spirit Level wants to make extraordinary claims (middle class people become more likely to die if the rich grow richer) they need extraordinary evidence. But they have no evidence at all, just correlations. That is why no one in scientific circles takes this book very seriously. However in Sweden it is making a huge impact.

Blogger Danne Nordling pointed out a strange fact about the book. Its measure of inequality, the most important factor in the book, is not gini, the standard inequality result. They seem to use 20/20 richest and poorest ratio. Why make this strange choice, when their source (UN Human Development Index) has gini? I smell data mining.

First for fun I did a simple regression of life expectancy on income inequality and per capita GDP for all countries the UN has data for. The correlation between inequality and health is not statistically significant.

Second I approximately redid the exercise in their book, I did a regression of inequality as measures by Gini and life expectancy for 28 OECD countries, again from the UN HDI. The result is not only that it again is not statistically significat, income inequality is positively correlated with life expectancy!

If I add per capita GDP to the regression, the p value is 0.29 for a positive relationship between health and inequality.

If a Social Democrat 5 minutes ago was convinced that “inequality kills” based on a cross country correlation, are they now convinced if I claim that I have “proven” than income inequality leads to longer life expectancy? How about my “proof” that income inequality makes you happy?

Swedish Social Democrats do not understand causality. They are excited about a book with a horrible methodological problem to start with (reverse causality), weak evidence (correlations) whose correlations are not even robust to using the standard measure of inequality.

PS. From a personal perspective, this book bothers me. It’s not at all the leftist message, but that they are giving cross country regressions a bad name. This kind of scientific abuse has caused economists to be very dismissive of any use of cross country regressions. Basically, as a rule of thumb, they completely dismiss cross country regressions altogether, because unserious people misuse it so much.

However I think that if you have solid theory and there are no serious endogeneity problems, you should be able to use countries as quasi-experiments, as long as you are careful. Have a solid story about causality, no reverse causality or third factor problems, and don’t push your results too hard, and go ahead and use cross country. For some problems, such as the effect of taxation on long run outcome, there is no other good measure of treatment.

Update:

Richard Wilkinson responds to me, and it is a amazingly week responce.

Wilkinson claims that the relationship between inequality and life expectancy remains if you use Gini. Look at his nice graph. Convincing, isn’t it?

It shouldn’t be. First of all he amazingly does not look at the straightforward measure “Life expectancy”. He uses some index. Here is the exact same graph if we look at the most intuitive measure of health, the standard, straightforward “life expectancy”, with the same countries Wilkinson used:

Not quite as nice, is it? Your eyes are not deceiving you, there is no statistically significant relationship between Gini and Life expectancy in the 21 countries Wilkinson looks at. The p value is 0.46.

That’s really it. End of story. He has written a book about the fact that inequality kills you, even though there is no statistically significant relationship, even a weakly statistically significant relationship (say p 0.1), between life expectancy and inequality.

Second, why are there 21 countries, when the OECD has more members? Out of the OECD countries, Wilkinson excludes 9 nations. I can understand excluding Mexico and Turkey, as they are third world nations. But He also excludes South Korea, Czech Republic, Hungary, Luxemburg, Poland, Slovak Republic and Iceland. I included the full list in my other post, let me put here here is well:

The relationship between Gini and Life expectancy is as you see essentially zero, and no where close to statistically significant, p value 0.78. (At least, unlike the U.N data, with the OECD data the relationship is not the opposite of what Wilkinson claims…)

Why would he include Portugal, but not Korea and Czech Republic (that are richer than Portugal), or Slovak Republic (that is as rich as Portugal)? One reason he excludes them is that Czech Republic and Slovak Republic have very even distribution of income, but low life expectancy. Portugal fits his story, it is unequal and does bad. Czech Republic and Slovak Republic don’t fit his story, since they are equal and do bad.

Now, if you ask me, I would say that the reason these two nations have low life expectancy is that they have low income. But once we control for income, we have to control for income for every country. In that case the relationship between gini and life expectancy becomes even weaker. As you remember I controlled for income using EVERY country in the U.N list, and the relationship between gini and life expectancy was no longer statistically significant!

This man has very weak evidence, and is data-mining like crazy. He does not use Gini, the standard measure, because it doesn’t tell him what he wants. He does not use life expectancy, the standard and intuitive measure, because there is no statistically significant relationship between life expectancy and inequality. He removed 7 out of 28 standard OECD countries, because some of them have low inequality and bad outcomes.

The lesson is: Don’t believe a word Wilkinson says before you have had a chance to verify it by looking at the source data yourself. He is not a objective scientists giving you realible data, he is selling you a story.

Of course the book is full of this type of graph. The central argument, the supposedly solid relationship between health and inequality, was a fraud, an optical trick created by data mining. How reliable do we expect the rest of the graphs in the book to be? The naive Swedish Social Democrats, perhaps desperate for new ideas, have been fooled by these guys. The Social Dems should ask for their money back.

I will give you the data, so you can run your own regressions.

If you see Wilkinson, ask him:

* How can he claim that inequality kills, when there is no statistically significant relationship between the standard measure of inequality (Gini) and the standard measure of health (Life expectancy)?

Remember: This is true regardless if he uses OECD data or UN data. In fact the OECD data is kinder to him than the UN HDI data, according to both sources there is no statistically significant relationship, not even close, and in the UN HDI the relationship between inequality and life expectancy is even mildly positive.

* Why did he use an odd measure of inequality and some index instead of the standard measures?

*When the standard measures refute his story, why does he not mention this to his audience, as a scientist would?