International “unsafe abortion” studies are highly flawed

Irish advocates rally for life. Abortion remains illegal in Ireland, despite industry attacks.

In my past post citing evidence of the effectiveness of pro-life laws, I mentioned a study conducted by the WHO and the Guttmacher Institute, published in the Lancet, often used by pro-choicers to claim that abortion laws don’t change the rate of abortion.

In that post, I mentioned a statement by the UN acknowledging that the estimates published by Guttmacher are speculative because complete data isn’t available. Recently, I’ve done more research into the methods used by the Lancet study by looking into Guttmacher’s method of calculating abortion rates in individual developing countries where abortion is illegal. Because Guttmacher only has compiled estimates for a limited amount of countries, the Lancet study estimates abortion rates in world regions, rather than country by country. However, Guttmacher’s individual country estimates are used to arrive a regional estimate.

Guttmacher’s methodology for estimating a a country’s abortion rate is explained in several publicly available studies. For this post, I’ll use their Uganda study as the example, but this methodology is not limited to Uganda as they also use it for Guatemala, Kenya, Kinshasa (capital of the Democratic Republic of Congo) and many others.

Basically, their method consists of a few steps:

1. Survey a representative sample of health facilities that are likely to treat abortion complications, both from spontaneous abortion (also known as miscarriage) and from induced abortion. They call this the “health facility survey.” The survey respondent is asked to provide the number of women they have seen come in for abortion complications in the past month and during the average month, and Guttmacher uses the average of these numbers and multiplies it by twelve to get an estimate for one year.

2. Estimate what portion of the women that were treated for abortion complications had a miscarriage and which had an induced abortion. Guttmacher assumes that women who miscarry before 13 weeks never or rarely go to the hospital, and that women who miscarry after 13 weeks usually seek treatment.

3. After arriving at their estimate for the number of women that sought treatment for an induced abortion, they have to estimate what amount of women did not seek treatment, either because their abortion didn’t result in complications or because the woman was too afraid or could not access a hospital. To do this, Guttmacher comes up with a multiplier from what they call the “health professionals survey.” These professionals are chosen by Guttmacher for their experience with abortion and abortion complications, and they agree to be interviewed as to their opinion of how many women don’t go to the hospital after an abortion and how often illegal abortions occur.

4. Guttmacher takes the hospital estimate and uses the multiplier derived from the health professional survey to come up with a final estimate. In some studies, they use different estimates based on different assumptions regarding how many women were hospitalized for a miscarriage and different multipliers.

Now one doesn’t need to be a statistician to see that this method will produce a very rough estimate highly prone to error and recall bias. If a lot of women actually do go to the hospital for a miscarriage before 13 weeks in a country, or the hospital survey respondent misremembers the number of women treated for complications, then the final estimate is going to be grossly wrong. But it’s even more troubling than it appears. For instance, this is Guttmacher’s statement to justify the assumption that women who miscarry in the first trimester don’t go to the hospital:

“Although some women who miscarry at earlier gestations seek medical care, many likely are treated on an outpatient basis, and relatively few are hospitalized.”

No data is presented here, only an assertion that is assumed to be true even for developing countries with higher rates of malnutrition and poor sanitation.

Here’s Guttmacher’s description of their health professionals survey:

“The research team prepared a list of health professionals who were conversant with abortion provision and postabortion care, including medical doctors, researchers, policymakers, family planning administrators and women’s rights activists.”

So this group includes family planning advocates and feminist activists (who are very likely to be supporters of legal abortion), and are by definition willing to talk to the Guttmacher researchers about abortion (in a country where abortion is illegal and highly stigmatized, probably very few people would be willing to discuss the subject with researchers unless they favored legalization). Although Guttmacher calls this a “health professionals survey” their description makes it sound like many of the respondents (such as the policymakers and women’s rights activists) do not have medical degrees. Among those that do, it seems likely that some may be illegal abortion providers themselves and would benefit greatly from decriminalization.  Clearly, this group has various financial, legal, and political motivation to overestimate the numbers of illegal abortions occurring and women who have died from them. I’m sure they are aware that overestimation of both illegal abortions and abortion deaths has been a very effective strategy in convincing citizens and governments to legalize abortion.

All of this is troubling enough on its own, but what’s even worse is that these estimates are often uncritically accepted even by pro-lifers and very few are aware of the methodology. The estimates are made by relying on an assumption with no supporting data (that women in developing countries don’t go the hospital for an early term miscarriage) then taking that number and multiplying it based on the claims of an extremely biased group.

I’m not against the idea of trying to estimate the occurrence of abortion in countries where it’s illegal, but this isn’t the way to do it. I do think that the hospital survey method, while flawed, is probably the best given the lack of any other official data. There just needs to be a better way of estimating hospitalized miscarriages than Guttmacher currently does. I also understand that in countries with restrictive laws and high stigma around abortion, generally Guttmacher may have no choice but to talk to pro-choice advocates for information. However, Guttmacher should recognize the bias of these groups and adjust accordingly. Unfortunately, Guttmacher’s own devotion to legal abortion probably gives them too much trust in these sources (some of which are probably affiliated with groups with close ties to Guttmacher, like the International Planned Parenthood Federation).

Finally, Guttmacher should present these speculative estimates as just that, speculative estimates, not objective fact, as they often do in their infographics and annual reports. They do at least make their methodology public, but then assert: “Guttmacher research proved once again that oppressive abortion laws are not effective in reducing the incidence of abortion.” That’s an extremely strong statement to make based on incredibly weak evidence with more grounding in speculation than fact.

So next time you hear the claim that 25 million unsafe abortions take place each year, or that abortion rates are similar in countries where it is legal or illegal, or that 50,000 women die from illegal abortions every year, take it with a healthy dose of skepticism. The numbers are largely based on subjective and probably highly biased opinion with very little empirical basis. The truth is that we don’t know how many illegal abortions take place per year, but we do know from several studies that fewer women will seek and obtain abortions when they are illegal.

[Today’s guest post by Candace Stewart is part of our paid blogging program.]