Child Maltreatment: Still Waters Run Deep
It's Shark Week again. For the 35th year in a row, the Discovery Channel is devoting an entire week to shark-oriented programming.
I haven't been watching, but I learned a few days ago that Greenland sharks are the longest-living vertebrates. One female apparently survived for nearly 400 years.
I also learned that radiocarbon dating was used to estimate her lifespan, and that statistical procedures are essential to the calculations. I had no idea. I may geek out about this in a future newsletter.
Radiocarbon dating illustrates how statistics can operate "behind the scenes", by supporting scientists in ways that most of the public may be unaware of. We accept that Greenland sharks live long, just like we accept that our medicines work, our teachers are trained, and our planet is warming. But most of us don't understand the role of statistics in drug testing, teacher training, and the measurement of changes in global temperature.
Some of that behind-the-scenes action isn't especially important for you and me. We'll be fine if the radiocarbon estimates are off by a century, let's say, or if the statistical algorithms used to analyze Shakespeare's plays have falsely identified Marlowe as a co-author. It's only the marine biologists and Shakespearean scholars who'd be in a kerfuffle.
Often, though, the statistics behind the scenes are critical to our health and well-being. Bad or misunderstood statistics have been responsible for ineffective medicines, racial bias in educational assessment, needless skepticism about climate change, and many other actual and potential harms.
In this newsletter I'll be describing a new study on the psychological aftermath of child abuse and neglect. One of the conclusions is that victims who don't discuss their experiences will not be at risk of emotional problems.
The study is brilliantly designed, but the statistical analyses are flawed and badly misinterpreted. The data actually contradict the researchers' own conclusions, and could lead to diminished support for certain victims of child abuse and neglect.
I find this deeply troubling. Although the study barely made the news, the organizations that did cover it are highly influential (e.g., the New York Times, WebMD, Science Daily, etc.), and, given the prominence of the journal that published the study, the findings could have an impact on mental health practitioners.
In short, this newsletter is a case study of how behind-the-scenes statistical flaws can give rise to inaccurate, potentially harmful conclusions.
The new study
The study, published online in JAMA Psychiatry this July 5, was conducted by Drs. Andrea Danese (King's College London) and Cathy Widom (City University of New York), both leading experts on childhood trauma and maltreatment. The July 5 article is the latest installment of a longitudinal project that began in 1986.
The original sample consisted of 908 adults who had experienced maltreatment (abuse or neglect) as children, as well as a comparison group of 667 adults who reported no maltreatment. Participants were first interviewed during their late 20s or early 30s. A second interview was conducted roughly a decade later, followed by a third interview roughly two years after that.
The main question was whether childhood maltreatment increased the risk of anxiety and depression in adulthood. Maltreatment was defined in two ways.
Objective vs. subjective maltreatment
Some members of the maltreatment group had court-substantiated records of abuse or neglect prior to age 12. Danese and Widom called this an objective indicator of maltreatment.
Other members of the maltreatment group indicated during the first interview that they had experienced abuse and/or neglect prior to age 12. The researchers treated this as a subjective indicator.
You might expect these two indicators to be redundant, but they weren't necessarily. The sample turned out to consist of four groups:
—Objective-plus-subjective: 492 people with court records of childhood maltreatment described their maltreatment during that first interview conducted when they were adults.
—Objective-only: 173 people with court records of childhood maltreatment did not describe any maltreatment during the interview.
—Subjective-only: 252 people with no court records of childhood maltreatment nonetheless described some form of maltreatment during the interview.
—No-maltreatment: 262 people with no court records of childhood maltreatment also did not report any during the interview. (This was the comparison group.)
(Participants were not told the purpose of the study or why they'd been selected. Methodologically speaking, this is a strength, because it enabled the distinction between objective and subjective maltreatment. Ethically speaking, you might feel uncomfortable that people who'd been maltreated weren't told why they were recruited. As a former Institutional Review Board chair, I'm on the fence about it.)
Psychopathology
During the first interview, the researchers used the NIMH Diagnostic Interview Schedule to determine whether each participant had experienced episodes of clinical depression or anxiety during the previous year and at any other point in their lives.
During the second and third interviews, the researchers determined whether each participant showed signs of clinical depression or anxiety during the previous week (above 20 on the CES Depression Scale and/or above 16 on the Beck Anxiety Inventory).
Results
The main finding – the one that received the most attention in news reports – was that mental health was undermined by the subjective perception of maltreatment rather than the objective experience of it.
Informally speaking, what Danese and Widom found is that it doesn't matter what you experienced as a child. If you say you were maltreated, you're more likely to experience anxiety or depression as an adult. This will be the case whether or not court records have documented the maltreatment.
On other hand, if court records show that you had been maltreated, but you don't talk about it when asked, you're no more likely to be anxious or depressed than someone who has never been maltreated.
(Here's a jargony but more specific version of what I just wrote: During the second and third interviews, people in the objective-plus-subjective and subjective-only groups exhibited significantly higher rates of anxiety and depression as compared to the no-maltreatment group. In contrast, the objective-only group and the no-maltreatment group did not differ in anxiety or depression at the time of either interview. These patterns emerged even after controlling for prior psychopathology.)
Implications
The results have two important implications for mental health professionals.
First, as Danese and Widom note:
"The clinical implications of these findings are that, even in the absence of documented evidence of childhood maltreatment, clinicians can use information provided by their clients to identify those at greater risk for a subsequent poor course of emotional disorders."
In other words, if you tell your therapist you were abused or neglected as a child, the therapist should recognize that you're at a greater risk for emotional disorders, even if there's no other evidence that you were maltreated. (I'm not sure how useful this is, because mental health professionals often have no evidence of early maltreatment other than the client's own description of it.)
Second, with respect to the objective-only group, the researchers indicated that if you were abused or neglected as a child, but you don't talk about it later, you're not especially likely to experience anxiety or depression. Here's how Dr. Danese put it in an interview with the New York Times two weeks ago:
"If the meaning you give to these experiences [childhood neglect or abuse] is not central to how you remember your childhood, so you don’t feel like you need to report it, then you are more likely to have better mental health over time."
Statistical misrepresentation
Ultimately, what concerns me most about the study is this conclusion that we don't need to worry about the mental health of the objective-only group – i.e., the people who were abused or neglected as children but don't mention it later when asked. This conclusion is not supported by the data. Here are four reasons why not:
1. The researchers ignored one of their own findings.
The conclusion that we don't need to worry about the objective-only group stems from comparisons with the no-maltreatment group. Overall, the groups showed no significant differences in anxiety or depression.
However, Danese and Widom also ran analyses for each of three specific types of maltreatment (physical abuse, sexual abuse, and neglect). Here they did find one significant difference: Participants in the objective-only group who had been sexually abused in childhood had exactly twice the rate of depression as people in the no-maltreatment group.
Although this finding was clearly noted in the Results section and in a key table, Danese and Widom apparently found it inconvenient, or unimportant, or something, because the Discussion section, like Danese's public statements, indicate that the objective-only group was not at risk of mental health problems. Here's how the researchers put it:
"…the objective experience of childhood maltreatment...was not associated with greater chronicity of emotional disorders in the absence of a subjective appraisal. The findings were largely invariant across different types of maltreatment."
This is one of the most shameful statements I've seen in a research report in a long time. Three different types of maltreatment were considered. There were no significant group differences for two of them (physical abuse and neglect). A significant difference emerged for one of them (sexual abuse). Two out of three is not "largely invariant."
I don't know whether Danese and Widom are callous, or driven by some theoretical agenda, or what, but here's a more responsible conclusion from the same data: People who were sexually abused as children but don't talk about it when asked are at risk of depression. However, if they were physically abused or neglected, their risk of anxiety and depression is not elevated.
With respect to maltreatment, no significant finding is unimportant. Even if the researchers had compared 50 different types of maltreatment, but only sexual abuse turned out to be linked to depression, this finding would need to be highlighted, not ignored.
2. The researchers played the significance game badly.
Even when maltreatment was not disaggregated, that absence of significant differences between the objective-only and no-maltreatment groups is questionable.
The incidence of depression was 37% greater in the objective-only group than the no-maltreatment group. The incidence of anxiety was 40% greater. Since these differences were not significant, the researchers concluded that the groups don't differ. Here's a more sensible conclusion: These analyses can't be trusted, because the sample was too small, and the rates of anxiety and depression within each group were too low.
Although you can't conclude anything from this kind of data, there is one detail that's striking and constitutes a datapoint in and of itself: The researchers ran a total of 24 separate comparisons between the objective-only group and the no-maltreatment group. In 23 of the 24 comparisons, the objective-only group had higher anxiety or depression scores. That doesn't prove anything, but it's a hint that the objective-only group fared more poorly than people who hadn't experienced maltreatment.
(A footnote for stats people: those 37% and 40% figures I cited come fron Incidence Rate Ratios, and the 95% confidence intervals for each IRR were 0.89–2.11 and 0.84–2.31, respectively. The researchers noted the use of Poisson regression to make comparisons, but they said nothing about the structure of their models.)
3. The researchers disregarded individual variability.
I also found it disturbing that the researchers seemed so nonchalant about two sources of variability in their sample: the age at which maltreatment occurred, and the severity of the maltreatment.
Because people who experienced maltreatment at any time prior to age 12 were included in the study, and because the severity of the experience probably varied widely, the objective-only group may consist of at least three different types of people:
Subgroup 1: People who forgot about the maltreatment, because it occurred at a very young age. (Decades of studies show that memories for personal experiences rarely go back earlier than about 3 years of age.)
Subgroup 2: People who can remember the maltreatment, but who don't mention it because it was so minor, or because they've come to terms with it so successfully, that they rarely think of it anymore.
Subgroup 3: People who can remember the maltreatment, because it was a horrible experience, but who choose to repress it or avoid discussing it whenever possible.
These are very different types of people, but the researchers did not attempt any distinctions. For instance, they didn't record the age at which maltreatment occurred or the severity of maltreatment.
Why is this important? For one thing, the results for the objective-only group could vary widely depending on the mix of subgroups. Suppose this group included a lot of people who'd been maltreated at age 3 or younger (subgroup 1), but not so many people who'd been severely maltreated later in childhood (subgroup 3). This might lead to underestimates of the impact of the objective-only designation.
(Nationally, the number of maltreatment victims aged 0 to 3 is not that different from the number in the age range 4 to 11. A small sample could easily yield an overrepresentation of either subgroup.)
Meanwhile, I found the researchers somewhat callous in the way they described the objective-only group. Here again is what Danese told the New York Times:
“If the meaning you give to these experiences [childhood neglect or abuse] is not central to how you remember your childhood, so you don’t feel like you need to report it, then you are more likely to have better mental health over time."
In this quote Danese is only alluding to subgroup 2. By failing to acknowledge the possibility of that third subgroup, it's easy to arrive at such a cheery conclusion about the mental health of the objective-only group.
4. The researchers ignored a plausible confound.
If you're in subgroup 3, you may just be a stoic person, or a repressed person, or whichever adjective best applies. Thus, it's possible that you choose to remain silent about your childhood maltreatment, and you choose not to speak up when you're feeling anxious or depressed.
If the objective-only group contained a lot of people like that, then the results for this group would be invalidated. Why? Because these individuals wouldn't necessarily be experiencing better mental health. Rather, there would be a confounding variable (call it "stoicism") that explains why they didn't mention their early maltreatment or their current symptoms of anxiety or depression. In a word, they're suffering in silence.
Regarding my third and fourth concerns, I reached out to Dr. Danese for clarification. (In fairness, I should acknowledge that I didn't ask him to address my charges of callousness, or my specific concerns about the statistics.) Here are the questions I emailed him:
"Could some of [the objective-only group] simply be more stoic, both when talking about their childhoods and when answering questions about anxiety and depression?
For [the youngest members of the objective-only group], could it be that what protects them isn't their appraisal of the maltreatment but rather that there's no episodic memory left to appraise? (Were you able to [do age comparisons]?"
Here is the entirety of his response:
"Both of your suggestions are certainly possible but are untested."
I appreciate the personal acknowledgment that these are possibilities, but I see no attention to them in the study itself or in Danese's remarks to the New York Times. Meanwhile, the Times focused too exclusively on subgroup 1, summarizing the findings like this: "A study of adults who were mistreated in childhood found that those who did not recall it showed fewer psychological aftereffects." That's not right either. Without knowing the ages at which participants were maltreated, we can't even guess as to how many forgot versus simply chose not to discuss their maltreatment.
All of this leaves me concerned about the possibility of mental health professionals and others coming to assume that if someone does not discuss the child abuse or neglect they experienced, we don't need to worry about their mental health. That's not what the data recommends.
Conclusion and final thoughts
According to several estimates, at least 600,000 American children are abused or neglected each year. Maltreatment in childhood is associated with a range of undesirable outcomes, including higher rates of anxiety and depression later in life. This doesn't mean that an abused or neglected child will automatically experience a mental health issue. Rather, early maltreatment increases the risk of one or more of those issues emerging over time.
The statistical analyses and inferences of the new study are deeply flawed and fail to support any conclusion. Most importantly, the data provide zero support for the researchers' view that we needn't worry about people who don't discuss their early maltreatment when asked. Rather, the data give us cause for concern.
To gain more perspective on the issues, I reached out to Carly Wielstein, a licensed marriage and family therapist who specializes in trauma. In one of her comments, she pointed out that it may be too simple to refer to some people as having forgotten their maltreatment:
"Simply “forgetting” seems to be an oversimplification of what goes on in a person’s brain and psyche when they are exposed to traumatic stress. A person might not have current conscious awareness of the experiences of abuse or neglect, but that would appear to be because of the adaptive (for the time) defense mechanisms unconsciously employed to “manage” the stressor. From a brain-based perspective, traumatic experiences such as childhood abuse or neglect have long lasting impact on a person’s brain functioning..."
Wielstein went on to note a number of ways that traumatic experiences can impact the brain, disrupt memory systems, and affect behavior, even among people who do not recall or at least describe the trauma. She also attested to the importance of subjective impressions:
"A lot of current traumatology literature supports that how we remember something or have come to make meaning of it can have a direct impact on how we feel about it. So, in this case, a person’s subjective memory of childhood abuse or neglect influences their current thoughts, feelings, and behaviors."
In other words, the mental health impact of early maltreatment depends on how victims remember and feel about it. Danese and Widom did acknowledge this. But one of the wrong turns they made was to assume that if people don't talk about their maltreatment, they either don't remember it or they no longer view it as important. Obviously there are many other reasons why a person might not talk about experiences like that, even if it might be helpful for them to do so.
In sum, Danese and Widom concluded that we needn't worry about an elevated risk of anxiety and depression among people who don't discuss early maltreatment, but this conclusion is inaccurate and potentially harmful. Among victims of maltreatment, still waters run deep, and there may be turbulence below.
Thanks for reading!
If you're an adult survivor of child abuse or neglect and are seeking more information or support, a list of organizations is provided at the Child Welfare Information Gateway here.
If you have other questions or concerns related to child abuse or neglect, you can contact the Childhelp National Child Abuse Hotline at 1-800-4-A-CHILD or 1-800-422-4453. Calls are answered 24 hours a day, seven days a week.