How many people really died in the Congo wars?

Today, Canadian researchers at Simon Fraser University released the 2009 Human Security Report, challenging popular perceptions of wars. Controversially, they conclude that countrywide death rates actually fall during most contemporary wars for three reasons:

• wars today are fought by smaller, often low-tech armies in smaller areas - there has been a 70% decrease in "high-intensity conflict" since the end of the Cold War
• there have been huge improvements in health care (especially immunization) in many countries
• humanitarian assistance to countries has increased dramatically in many war zones

As a case in point, they directly attack the International Rescue Committee's mortality studies in the Congo, which concluded that between 1998 and 2007 5.4 million people died as a result of the war there. They suggest that due to flaws in the IRC research, the real number could be as much as 60% lower.

This is not the first time that such questions have been raised - in 2008 two Belgian demographers carried out a study based on the results of voter registration and, using previous Congolese censuses as baselines, extrapolated how many people would have died. Their result: only 200,000, i.e. less than 5% of the IRC estimate.

I am not very qualified to grapple with all of the statistical spit balls that have been slung back and forth between Simon Fraser University and the IRC over the past few days. I do think that in general, their basic argument is a bit misleading, as they are not saying that people don't die in wars, but that those deaths are offset by improvements in health care, somehow in the process implying that we shouldn't be as alarmed by 1,000 deaths if better health care at the same time saved 1,000 other people. The death of one person is not justified or made irrelevant by saving another person's life.

But let me deal briefly with the case of the Congo, which they deal with at length. As far as I can see, they have two main critiques of the IRC: (1) That the baseline mortality rates they used for 1998 were too low, thereby exaggerating the number of deaths that could be attributed to the war in the subsequent years. And (2) that they health zones they chose do not accurately represent mortality in the eastern Congo.

For the first point, the IRC used a baseline mortality rate of 1,5 per 1,000, which was the average mortality rate for sub-Saharan Africa. They looked at the 1984 mortality rate (given by the government during its last census), which had been 1,3/1,000 and at UNICEF's mortality rate for 1996, which had been 1,2/1,000, but preferred to go with the higher 1,5 rate to make sure they did not overestimate the death toll. The Human Security Report (HSR) says their baseline was far too low and should have been 2,0/1,000, the rate for western Congo in 2000. I tend to side with IRC on this - I don't see why they should substitute a 2000 baseline for 1998; although western Congo was relatively peaceful, the effects of the war probably still had an impact there through the economic and political instability it caused (just think of the extreme inflation and unrest in Kinshasa 1999-2000, the hundreds of thousands of people who left rural areas around the country to come to western cities.)

For the second point, the IRC carried out 5 surveys over seven years. Their researchers visited several dozen different health zones throughout the Congo and surveyed as many as 19,500 households in one survey. The surveys were run by prominent researchers, epidemiologists and statisticians in top universities; the results were published in esteemed medical journals such as The Lancet. Nonetheless, as they themselves admit, there are questions whether the health zones they used were representative: they would take the mortality rate for one or several health zones (a relatively large area) and extrapolate to the whole province (which usually includes 5-40 health zones). For example, in the 2001 health survey they measured the mortality rates for Lusambo and Kisangani and extrapolated to the over 40 health zones for Province Orientale. HSR was particularly annoyed that they took rates for Moba and Kalemie, two health zones with extraordinarily high rates, and extrapolated to all of Katanga.

This is a problem, there is no doubt, and one that the IRC study admits up front. The conflict and humanitarian situation in the eastern Congo depends heavily on micro-dynamics of conflict that can vary significantly from one area to the next. How significant is this problem of representivity? First, the IRC surveyed quite a few health zones - in South Kivu they did three out of 13 health zones, Katana, Kamituga and Nyangezi. While they selected them at random, on the face of it, it isn't a bad choice: Katana and Nyangezi both include high-altitude hills as well as large towns and are close to Bukavu; Kamituga is a remote, mostly low-land jungle zone. For the rest of the Kivus, the selection of health zones doesn't reveal any immediate problems to my eye.

But the doubt does persist - as long as we don't have the real baseline mortality (we will never have it) and have good surveys of all health zones in the country, we cannot conclusively judge. But the IRC did do what I think was a thorough job, sending out dozens of teams to randomly selected GPS locations in randomly selected health zones throughout the country, often traveling on motorcycles through deep jungle and difficult terrain.

We should also bear in mind that the problem of representivity could work in both directions, by either exaggerating or underestimating the mortality rates. In particular, IRC suggests that it couldn't visit some of the most violent areas, which should bias their results towards underestimation.

The HSR report can be seen here. Unfortunately, I haven't seen the official IRC statement (I have seen an unofficial one, which I am reluctant to release). The BBC and AP stories weren't very good, I fear, and didn't explain what the disagreement is really about and make it seem like the IRC is trying to cover up a screw up, when the IRC is just saying that statistical estimates in such complex situation will also be flawed, but that they tried to deal with the challenges as best as possible.