Abortion Pills and Junk Science

Chris Adkins didn't mean to cause trouble.

Back in April 2023, the pharmaceutical sciences professor contacted the editor of Health Services Research and Managerial Epidemiology with concerns about a recent article. The article questioned the safety of mifepristone, an FDA-approved medication abortion widely understood to be safe and effective.

In February of this year, following a lengthy investigation, Sage, the publisher of the journal, retracted this article and two others. The lead author on all three articles was James Studnicki, Vice President and Director of Data Analytics at the pro-life Charlotte Lozier Institute.

Ordinarily, this would be the end of the story. The retraction of three papers would be no more than an embarrassment to the authors and a message to scientists researching the topic that the data can't be trusted.

This case is different.

It's different because Studnicki and colleagues have just published a new, deeply flawed study that once again falsely impugns the safety record of abortion medication.

And it's different because the new study, like two of the retracted ones, has fueled a legal effort to restrict access to mifepristone. The U.S. Supreme Court has heard, and will likely hear again, an argument, based on Studnicki's research, that mifepristone is unsafe.

If that argument is persuasive, access to legal abortions, heavily curtailed following the demise of Roe v Wade in 2022, will be further restricted. Medication abortion is currently banned in 18 states and limited in 10 others. Even if the case fails, it may draw undue attention to false claims about mifepristone's safety record. As the Washington Post noted recently, women are already struggling to reconcile these claims with the mainstream view that medication abortion is generally safe.

(If you consider yourself pro-life, or otherwise opposed to medication abortion, I invite you to continue reading this newsletter. I won't be judging you or your opinions. I will only be suggesting that you don't want James Studnicki on your side. His research is junk science: Shoddy in design and deceptive in presentation. In a word, he appears to be a swindler.)

Background

Parts of this section are drawn from a February newsletter I wrote about the retracted studies. For more details, see this June 2024 paper written Drs. Ushma Upadhyay and Chris Adkins. Sofia Resnick provides broader context here, here, and here.

In 2000, the FDA authorized the use of two pills, mifepristone and misoprostol, to bring about an abortion. Typically, mifepristone is taken to end the pregnancy, and, one to two days later, misoprostol is used to induce a miscarriage. (Mifepristone is sold under the brand name Mifiprex or as the generic Mifepristone Tablets; misoprostol is sold as Cytotec.)  

In recent years, the FDA has made it easier for women to obtain mifepristone and misoprostol. For example, in 2021, acknowledging logistical challenges created by the pandemic, the agency allowed the pills to be prescribed by telemedicine and mailed directly to patients, a change it later made permanent. Roughly two-thirds of abortions are now carried out via medication.

Hundreds of studies have shown that medication abortions are effective and safe. The American College of Obstetrics and Gynecology has pointed out that there's a greater risk of complications from wisdom tooth removal, colonoscopies, or Viagra use. 

Nonetheless, in April 2023, Matthew Kacsmaryk, a U.S. District Court judge in Amarillo, Texas issued a ruling that suspended the FDA's approval of mifepristone. Kacsmaryk is a Trump appointee with a history of anti-abortion advocacy.

Judge Kacsmaryk's decision was based in part on the contention that the FDA hasn't properly acknowledged mifepristone's safety risks. His ruling did not review mifepristone's extensive safety record. Instead, much of the evidence for safety risks came from two of the subsequently-retracted studies led by James Studnicki.

Following other lower court rulings and intervention by the Department of Justice, the case was brought to the U.S. Supreme Court. On June 13 of this year, SCOTUS unanimously reversed  Kacsmaryk's decision and remanded the case back to the district court.

Standing

The reason for the Supreme Court's reversal is central to this story.

The Court ruled that the plaintiffs in this case, the Alliance for Hippocratic Medicine (AHM), had no standing, meaning that the anti-abortion groups and individuals that comprise the AHM were not directly harmed by the use of mifepristone.

Unfortunately, as Yeats put it, the worst are full of passionate intensity.

This month, the AHM returned to Matthew Kacsmaryk's district court with an amended brief. The states of Missouri, Kansas, and Idaho are now included among the plaintiffs, the hope being that standing will now be established via evidence that these states are harmed by women's use of mifepristone.

The amended brief no longer mentions the retracted studies. Instead, it cites a new study, led by...you guessed it…James Studnicki.

Given Judge Kacsmaryk's previous decision and history of conservative advocacy, it seems likely that he will rule unfavorably and, eventually, the U.S. Supreme Court will be asked to once again review a case where safety concerns about a drug known to be safe are grounded in flawed, deceptively-presented research.

What follows is a description of the new study and some explanation for why I've been referring to it so harshly. At the end I'll discuss why we should be concerned and what we can do when science seems overly influenced by ideology.

Editorial process

Ordinarily I would leap right into the data, but a look at the journal itself and its editorial process is revealing.

The new study, led by Dr. Studnicki and colleagues at the Lozier Institute and two other aggressive, anti-abortion advocacy groups, appeared this August 20 in the International Journal of Epidemiology and Public Health, a strikingly low-quality, "pay-to-play" journal. The study was funded by the Lozier Institute. Judging from the journal's processing fee page, the Institute spent $1899 to publish the study.

The journal's website claims that its articles are peer-reviewed. According to the site,

"you can rest guaranteed that whether you are a working researcher, scientist, or research student, you could get the handiest information ever by reading the journal."

The handiest information ever? I beg to differ. Looking through the archives, you can find articles published by individuals whose formal affiliations are listed as "independent researcher" or "self employed", and whose papers develop themes such as "it's important to be healthy" with no more evidence than you'd expect to find in a 9th grade English essay. Some of the papers read like Wikipedia entries, though in many cases, Wikipedia is clearer and more thorough.

(Keep in mind, if I seem excessively snarky, that this study is driving a legal effort to restrict women's access to a medication shown to be safer than Viagra. Keep in mind that Supreme Court justices are unlikely to peruse the archives of International Journal of Epidemiology and Public Health and discover for themselves the wretchedness of the research.)

It's not clear that the journal's editorial board actually includes any epidemiological experts, but in any case, here's a screenshot of one of the reddest of red flags in the editorial process:

If you're not familiar with peer review, I'll just say this: At a reputable journal, it's never completed in four days. Even four weeks would be a stretch.

Acceptance of a paper four days after being received indicates a sham review process, one that's designed to quickly accumulate revenue from authors who pay to publish.

Methods

Nothing I've written so far proves that the study is bad or deceptive. We need to look at how it was actually conducted and written up.

Studnicki and colleagues reviewed data on Emergency Department (ED) visits for the 17 states that fund abortion services through Medicaid. The time period was 2004 through 2015. An ED visit was recorded within 30 days after a live birth, procedural abortion, or medication abortion (likely a combination of mifepristone and misoprostol in most cases, although this is not clarified). Data on women who'd never given birth were also included.

Each ED visit was coded for severity by the hospitals themselves using a standard 5-point scale. Studnicki and colleagues focused on the two highest ratings ("severe and "critical"), combining these into a single measure of ED visit acuity. Thus, an "acute" ED visit was defined as one that hospital staff rated as severe or critical.

Findings

Four findings were presented for the time period 2004 through 2015:

1. The total number of ED visits following medication abortion increased. 

2. The acuity of ED visits following medication abortion increased. 

3. The number of acute ED visits following medication abortion increased. 

4. ED acuity was consistently higher for medication abortion than for procedural abortion or live birth.

Shoddiness and deception

Unlike most studies, I find nothing positive about this one. Here are a few of the key problems.

1. Lack of context.

The first three findings tell us nothing about safety, because the availability and use of mifepristone increased dramatically during this time period too. Studnicki and colleagues failed to mention this.

Specifically, CDC data shows that from 2004 through 2015, medication abortions in the U.S. increased from tens of thousands per year to hundreds of thousands per year.

The more people who take a medication ­(any medication), the greater the number of side effects we can expect to observe. That doesn't mean the medication is unsafe or increasing in harmfulness. Here's a simple illustration:

—In 1986, 10 women take a medication and one of them subsequently visits the ED.

—In 1987, 100 women take the medication and 5 of them subsequently visit the ED.

Studnicki and colleagues would say that from 1986 to 1987, there was a 400% increase in ED visits. This is true: 5 visits is 400% greater than one visit. But the percent of women who visit the ED actually decreases. In 1986 it's 10% (1 in 10). In 1987, it's 5% (5 in 100).

In short, saying that ED visits following a medication abortion increased in number and acuity is literally meaningless with respect to safety. You have to also take into account how many women took the medication but did not visit an ED.

Studnicki knows this, judging from other studies he has published. In this particular study, he and his colleagues neither contextualize the ED findings, nor acknowledge that the data is meaningless without context.

2. Lack of information.

(a) We are told nothing about the women in this study other the fact that they were supported by Medicaid. Studnicki and colleagues neither describe the groups nor address the possibility that they differ in anything other than method of abortion.

This is reminiscent of the early 20th century psychologists who tested children's IQs and concluded, for instance, that Black children tend to be less intelligent than White children.

We know now (as many psychologists knew then) that simple group comparisons were inappropriate. Compared to their White peers, Black children also tended to be poorer, served by lower quality schools, and tested by someone whose race differed from their own (i.e., a White adult). In short, the group difference in IQ was not attributable to race, but rather to many other variables (including racially-biased test content).

As for the group comparisons in Studnicki et al., the women who chose medication abortion may have had fewer resources than women who could afford the time and expense necessary to travel (often great distances) to obtain a procedural abortion. Women who chose medication abortion may have had physical conditions that precluded travel. Perhaps they had personal or professional obligations that created more stress or undermined their capacity for self-care. Any of these differences could've increased the number and/or acuity of their ED visits.

I'm not saying any of this is true. My point is that we shouldn't have to speculate. The researchers should've addressed these possibilities statistically or at least acknowledged them as limitations of the findings. Once again, judging from other published studies, Studnicki knows this.

(b) Studnicki and colleagues don't explain why they combined the two highest ED acuity ratings and compared them to a collapsed aggregate of the other three. (The lowest rating is "non-urgent" and the next-lowest is "urgent". Wouldn't you want to know, for instance, how many women visited an ED for "non-urgent" reasons versus "urgent" ones?)

3. Lack of coherence.

So far, I've described some reasons why none of the findings can be trusted (point 2), and why the first three findings would be meaningless anyway (point 1). I want to close with two additional concerns.

(a) Finding 2 (the acuity of ED visits following medication abortion increased from 2004 through 2015) could mean two different things, and neither interpretation is supported by the data:

–> Among women who chose medication abortion, the total number of acute ED visits increased. As I mentioned under point 1, this is meaningless, because the total number of women using medication methods increased during the same time period,

–> Among women who chose medication abortion and visited an ED, the proportion of ED visits that were most severe (i.e., acute visits) increased. What Studnicki and colleagues actually found is that for each of the groups (no pregnancy, live birth, procedural abortion, medication abortion), the proportion of acute visits increased from 2004 through 2015. There's no evidence of a significantly greater increase among the medication group.

(b) The fourth finding (from 2004 through 2015, ED acuity was consistently higher for medication abortion than for procedural abortion or live birth) is based on incomplete data.

To obtain this finding, Studnicki and colleagues collapsed data for all years from 2004 through 2015 for each group (medication abortion, procedural abortion, live birth, no pregnancy), then calculated the likelihood that women in each group would have an acute (vs. non-acute) ED visit. The likelihood of an acute visit was found greater for women who'd chosen a medication abortion.

Studnicki and colleagues actually obtained data from 1999 through 2015 but used 2004 as the starting point on the grounds of having insufficient data from earlier years. This makes no sense, because time is not treated as a variable in their analysis. In other words, they collapsed data across years and simply compared groups. Even if in, say, 2003, only one woman had an abortion, that datapoint should've been included.

What the researchers did here is incoherent at best. I suspect they chose 2004 as a starting point because it yielded data most favorable to their theme.

There are other problems with this study, but I think you get the idea. I interviewed James Studnicki for my February newsletter on his retracted papers but elected not to do so this time, because he seemed hostile and did no more than reiterate one of the methodological choices partially responsible for the retractions. If we can call him a swindler, I would say that he's a passionately committed one – and funded by the Lozier Institute for his ongoing efforts to undermine women's reproductive rights.

Causes for concern

In spite of the hundreds of studies attesting to the safety records of mifepristone and misoprostol, Judge Matthew Kacsmaryk will be once again ruling on the complaint initiated by the AHM. Assuming an unfavorable decision, the case will likely proceed to the Supreme Court. Studnicki et al.'s study is discussed in two separate passages of the AHM's amended brief that Kacsmaryk will be reviewing at the district court level.

What makes this especially concerning is a separate SCOTUS ruling in June of this year that overturned a legal precedent known as "Chevron deference."

The short version of the story is that when Congresses passes broadly-worded regulatory laws, federal agencies, following the guidance of their own experts, used to be more empowered in how to interpret those laws. The new SCOTUS ruling, pushed strongly by conservatives and the Trump administration, gives the courts more authority over interpretation.

This could mean, for instance, that the FDA approval of mifepristone on the grounds of safety and effectiveness data can now be more easily overturned by a conservatively-minded judge or court.

Federal agencies aren't perfect. The FDA has made decisions that clash with the advice of its own expert advisory panels. But, at least in theory, the scientific expertise driving FDA decision-making will hew more closely to reputable science than an ideologically-motivated judge who's empowered to decide which evidence to trust.

What you can do

Chris Adkins was courageous and, I would say, noble to reach out to a journal editor with concerns, but most of us aren't reading scientific articles every day, and in any case, Dr. Adkins couldn't stop junk science from continuing to inform legal efforts to restrict abortion medication. Like the hydra of ancient Greek mythology, cutting off three of Studnicki's flawed papers has only spurred the growth of new ones.

Ideally, the courts would entertain the competing narratives of experts but find ways to sideline the shoddiest, most ideologically-driven data. I'm not sure how this could be accomplished. Do we teach research methods classes to law students? Do we create guidelines for judges? I don't know. In the meantime, if you have concerns about women's reproductive rights and their imperilment by scientific disinformation, you can find out more about advocacy here and here.

Thanks for reading!