The trouble with bad social science

I don’t often read Foreign Affairs – the leading political science magazine published by the Council on Foreign Relations – but when, ahead of a 14-hour flight back to South Africa, I found an airport bookshop displaying the March/April issue with the provocative title ‘The Trouble with Race’, I decided that it would make for good airplane reading. Here was one of the world’s most authoritative magazines writing about an issue that affects everyone everywhere, but it is in South Africa during March 2015 that racial injustices has been brought into the limelight. Race, and racial prejudice, is the underlying theme of #RhodesMustFall, a movement by thousands of students across the country agitating for the dismantling of white power structures two decades after apartheid formally ended. Discussions of race and racial exclusion has emerged at campuses across the country. Rhodes will fall, but the future is murky.

And then I spot the article in Foreign Affairs titled ‘Apartheid’s Long Shadow: How Racial Divides Distort South Africa’s Democracy‘. Finally, I thought, some answers to our country’s qualms. From the leading thinkers of Foreign Affairs.

I turn to page 41. The paper is by James Gibson, Professor of Government at Washington University in St. Louis and (to my surprise) Professor Extraordinary in Political Science at Stellenbosch University. It starts brightly: South Africa is a diverse and unequal society. ‘The future of South Africa’s multiracial democracy depends heavily on minimizing animosity and hostility among these groups. Can South Africans of different races continue to get along?’ Wonderful. Answers.

Then Gibson attempts to explain who these peoples are: Black South Africans, the ‘descendants of the original inhabitants of the area’, who constitute different ethnic and linguistic subgroups. ‘Apartheid eliminated such distinctions by grouping all blacks together and, in essence, expelling them from South Africa.’ This is awkward: had Gibson attended a first-year History class (or even my second-year Econ History class), he would know that there existed a group of original inhabitants known as the Khoesan. (Or Khoisan. Actually they are not, in fact, a single group but for the purposes of this post, let’s assume they are. More about them here.) Blacks did arrive south of the Limpopo (around 300 AD) long before whites , but they did not enter vacant land, no more than the Voortrekkers trekked into vacant territory when they crossed the Gariep (as the apartheid apologists wished to believe). The sad truth is that the original inhabitants of South Africa have been mostly wiped out.

And he goes on: ‘A third group is referred to as Coloured, a term that applies to an extremely diverse group of people whose mixed heritage attests to decades of intimate contact between three different peoples: Europeans, certain groups of black Africans, and slaves brought to the area by European colonists from Madagascar and Southeast Asia.’ Again, where are the Khoesan? Surely they can’t be classified as ‘certain groups of black Africans’? (In a paper with Erik Green, I call them the ‘Missing People‘. Gibson’s paper shows why.)

But these factual errors aside, Gibson’s paper is about evolving racial prejudice in South Africa. He argues that race relations are not really improving, despite two decades of democracy. Using the Institute for Justice and Reconciliation’s South African Reconciliation Barometer, a survey among a representative sample of the population, he finds that ‘intergroup prejudice is not the exclusive problem of black people in South Africa, but as a group, they express considerably more prejudice against their fellow South Africans than do the members of the minority races’. And ‘Whites and Coloured South Africans who have more contact with people of others (sic) races are considerably less prejudiced. Among black and Asian South Africans, however, not only does contact with members of other groups not reduce prejudice; it actually seems to increase it.’

These are powerful conclusions on an emotionally-charged topic, yet Gibson provides little empirical justification to support these claims. Yes, he shows that some race groups have higher self-reported prejudice than others, but as any social scientist should know, correlation does not equate causation. To say that black South Africans are more prejudiced suggests that you’ve controlled for other observable characteristics that may also affect prejudice, like income, location, and education to name a few. But perhaps it’s not black South Africans that are more prejudiced. Consider, instead, that it is poor education that cause people to be prejudiced. But because poor education outcomes are highly correlated with race in South Africa, by implication you will also find a high correlation between race and prejudice. Race, as Gibson uses it, is likely to just be a proxy for other things.

Social scientists solve these issues by using statistical techniques such as regression analysis. Even with a standard regression, though, causality is difficult to determine. Usually we use experiments or, because this is often not possible in a social setting, natural experiments. One* excellent example of this is Martin Abel’s work on forced removals during apartheid. Abel, a PhD student at Harvard University, investigates whether the people that were moved during apartheid are more (or less) prejudiced towards people of a different race today than their counterparts who were not moved. He finds:

…that those living close to former resettlement camps have higher levels of trust towards members of their social network and people in general. They are also more trusting towards members of both their own and other ethnic groups. This effect is larger for people born after 1975 who did not witness the forced removals, which suggests that events had long-term effects on social capital. This is also reflected by lower crime rates in these communities. Exploring causal mechanisms, I document that resettlement camp areas are more ethnically diverse and that diversity is positively correlated with measures of social capital only in areas affected by relocation. Interacting with people from different backgrounds and adopting a shared identity as displaced people may explain why relocation communities have higher levels of social capital despite potential short-term conflict over resources.

Abel’s careful statistical design and analysis forces him to draw exactly the opposite conclusion than Gibson does from his simple correlations. Deeper integration makes people less prejudiced, not more. The policy implications are profound.

#RhodesMustFall shows that there is indeed much value to investigating how and why prejudices evolve. But the way James Gibson does it in Foreign Affairs is not the way to do it. We should be skeptical of causal claims based on simple correlations. That Foreign Affairs accepted an article with factual errors and wonky causal claims on a topic of such importance is worrisome. We should expect better science from our social scientists.

*I received a message on Facebook from MIT PhD student Dan de Kadt: I read this article too, and had a similar reaction. Johan, in some other work Melissa Sands and I address the exact hypothesis Gibson proposes… We find that contact does partially explain cross-racial voting (whites who are in contact with blacks are more willing to vote ANC, and blacks who are in contact with whites more willing to move from the ANC). But there’s very little in our data to say that racial attitudes are affected (so little that we don’t report it in the paper). And there’s no evidence that contact has differential effects depending on race (as one assumes Gibson is proposing).