Pandemic Brain
What are the chances that you'll have a car accident during the next year?
It might be tempting to answer: 2.6%. Because there are roughly 233 million licensed drivers in the U.S. and roughly 6 million collisions per year. (6/233 = .0257.)
That answer is almost surely wrong. Not everyone with a license drives. Not every driver is licensed. Not every accident gets reported. Some people have more than one collision per year.
More importantly, personal risk can't be inferred from group data. The chances of having a car accident depend on where you drive, how often you drive, how well you drive, etc. Still, we can be absolutely sure about a few things. For example, the chances of having one accident can only increase when you consider longer stretches of time. A 1.5% chance of having one accident in the next year implies that the chance of having one accident in the next five years is greater than 1.5%.
What I've just written makes sense to most people. But we don't always apply these ideas consistently. When completing a test, a person might estimate a 4% chance of having one accident in the next year. Later, during the same test, they might estimate a 3% chance of having one accident in the next five years. That's illogical, because those five years include the first year.
In this newsletter I'll be describing evidence that during the pandemic, college students got worse at answering questions like these. In a new study, published by Dr. Melissa Buelow and colleagues at Ohio State Newark, the authors describe a broad decline in decision-making ability that illustrates "pandemic brain". Toward the end of the newsletter, I'll suggest that studies like this illustrate the historically distinctive power of statistics at detecting the influence of society-wide events such as pandemics.
Pandemic brain
We talk about it. Many of us experienced it. But what exactly is "pandemic brain"?
There's no question that the pandemic undermined our mental health. In many countries, levels of anxiety, depression, drug abuse, eating disorders, and suicide began to spike after February 2020. In K-12 settings, these effects exacerbated the pandemic learning loss already arising from disruptions to classroom instruction.
Data aside, most of us seem to have had an unpleasant time, even if we managed not to get sick. For me, every day was a smorgasbord of negative emotions. Boredom, isolation, stress. Grief for the victims, fear for loved ones. Anger at vaccine skeptics. And a sort of mental fogginess that I didn't expect to feel until I reached my 90s.
The phrase "pandemic brain" has meant different things to different people since it first bubbled up in the news and social media in 2020. Most of the time it's used in reference to that fogginess, though people have often complained too about distractibility, forgetfulness, or information overload. For me, it just seemed more difficult than usual to think clearly and make rational decisions. This is the kind of "pandemic brain" that was the focus of the new study.
Before I get to that study, I want to mention that "pandemic brain" is no longer just a catchy phrase. Evidence is accumulating that our brains literally changed during the pandemic.
Pandemic brain: Neurological data
The best of the neurological studies were "accidental", in the sense that the researchers were in the midst of gathering brain scan data when the pandemic struck. This created an opportunity to compare people scanned before vs. after February 2020. (COVID-19 antibody tests were also used to ensure comparisons among people who'd never been infected.)
Last year, a team led by Dr. Ludovica Brusaferri at Harvard Medical School reported that neuroinflammation increased slightly during the pandemic. According to Brusaferri and colleagues, inflammation of neural tissue is a kind of immune response that creates symptoms of illness such as anxiety, depression, and fatigue. Crudely speaking, the stress of the pandemic caused our brains to respond as if we were sick, and many of us experienced illness symptoms. One might speculate that "brain fog" is among those symptoms.
Another team, led by Dr. Ian Gotlib at Stanford, reported last year that during the pandemic, adolescent brains exhibited physical changes, such as cortical thinning, that are more typical of older people. In a word, adolescent brains began to mature more rapidly during the pandemic. (Ordinarily, greater maturity among teenagers is a good thing; it's unclear whether that's the case her.)
In short, "pandemic brain" does seem to be a neurological phenomenon. Certainly it has undermined mental health. The question underlying the new study is how it affects our thinking.
The new study
In this study, published in the Journal of American College Health, Dr. Buelow and colleagues looked at decision-making among undergraduates tested before and after the onset of the pandemic. Here again, the study was fortuitous; data collection was already underway at the time the pandemic struck.
The researchers' main analyses compared 722 undergraduates tested before the pandemic to 161 undergraduates tested on two occasions in Fall 2020. The test used was the Adult Decision Making Competence Scale (ADMC).
Impulsive vs. deliberative decision-making
The ADMC is based on a distinction between two "systems" or ways of thinking that a person can use to make decisions. One system is impulsive, the other is more deliberative. This distinction was explored by Nobel Prize-winner Daniel Kahneman, among others, and popularized in his 2011 bestseller "Thinking, Fast and Slow."
As the names suggest, the impulsive system is rapid and instinctive, while the deliberative one is more thoughtful. Impulsive decision-making leads to more risky choices, while deliberative decision-making, being more rational, reduces risk and leads to more desirable outcomes.
Studies have shown that high levels of stress undermine deliberative decision-making processes. Thus, Dr. Buelow and colleagues predicted that once the pandemic hit, even people who had not been sick would reason more impulsively and less deliberately, because the pandemic became a huge stressor for pretty much everyone.
The ADMC measures the extent to which a person responds to questions deliberatively as opposed to impulsively. In the Appendix, I provide details on how the test is structured and scored.
Main findings
During the first Fall 2020 testing, the pandemic cohort showed more impulsive decision-making than the pre-pandemic cohort had previously shown. No changes occurred from the first to the second Fall testing. Exploratory analyses revealed that the same pattern extended through Spring 2022.
Importantly, ADMC scores were generally unrelated to the time during the semester when students were tested. In other words, even though students tend to become more stressed as the semester wears on, ADMC scores were not affected by that increase in stress. Rather, the key distinction was between test administration before vs. during the pandemic.
If you're a stats person, the effect sizes for each component of decision-making (see Appendix) were medium or nearly so, with Cohen's d values ranging from 0.32 to 0.73. This might be considered a strength of the study, given how many large-sample COVID-19 studies report significant but tiny effects.
Why this study is important
Dr. Buelow and colleagues concluded that "pandemic brain" is a real phenomenon that's reflected in impaired decision-making. Historically speaking, this is an impressive demonstration.
At the outset of the Decameron, Boccaccio describes how Florentines responded to the outbreak of bubonic plague in 1348. Now, seven centuries later, the variety of their responses would be familiar to us: Some people fled the city, some isolated, some continued partying, and some took precautions while attempting to carry on. Fear and misery ruled the day.
Presumably, survivors of the plague experienced the same kinds of mental health problems we've experienced over the past three years, including what we call pandemic brain. But we have no direct evidence for that. This is merely a plausible guess, based on studies of how people respond to extreme stress.
Beginning in the 20th century, the widespread use of statistics allowed scientists to document, for the first time, exactly how people respond to society-wide stressors. What distinguishes studies like Dr. Buelow's is the granularity of the findings: Scores on a test of decision-making ability dipped slightly during the pandemic. This is useful information. We know that people need extra support during a pandemic for things like anxiety, depression, substance abuse, eating disorders, and suicidal intent. Now we have evidence that support for decision-making is needed too. "Brain fog" is not just a matter of fatigue and lack of motivation.
Interpretive questions
Can we be sure that the pandemic is responsible for the changes that Dr. Buelow and colleagues observed? Experimental design was not an option, because there could be no control group (i.e., we all experienced the pandemic). This opens the door to alternative interpretations, at least in principle, as does reliance on different cohorts for the two time periods (pre-pandemic vs. pandemic). Maybe the pre-pandemic cohort showed better decision-making because they were older on average, or contained a higher proportion of women. Maybe the pandemic cohort completed the ADMC during an unusually stressful time (e.g., the week some new variant of COVID-19 caused widespread alarm).
The researchers' own data undercut these alternative interpretations. For instance, the pre-pandemic and pandemic cohorts were comparable in distributions of age, gender, and race. The consistency of the pandemic cohort's data across multiple testings suggests that their performance wasn't influenced much by whatever stress they'd been experiencing in recent days or weeks.
Here's one final interpretive concern. According to the Methods section, the pre-pandemic cohort completed the ADMC in the lab, while the pandemic cohort filled it out online. If the pre-pandemic group did a paper-and-pencil version, could the results have been influenced by the shift in testing format? (Online testing formats can make it difficult or impossible to check prior answers, thereby increasing the prevalence of inconsistencies associated with impulsive decision-making.) I emailed Dr. Buelow to ask about this. Here's part of her reply:
"[T]he pre-pandemic participants completed the study in the lab but the pandemic participants completed the study online due to the closure of labs during the Fall 2020 semester. When the students completed the study in the lab, they were seated at a computer and worked on the electronic task individually. Although the pandemic participants could complete the study in a setting of their choosing, they still completed an electronic version of the task."
The fact that all groups completed the ADMC online puts to rest my concern about testing format. (One might speculate about differences between taking the test in a lab versus doing so at home, but it's not clear how that difference could account for the findings.)
In sum, I found the results of the study to be quite persuasive, and not readily accounted for by some alternative explanation.
In her email, Dr. Buelow also noted that data collection is ongoing. It will be interesting to see whether decision-making skills "rebound" to pre-pandemic levels. One can only hope.
Conclusion
The first draft of this newsletter was entitled "Pandemic Brian". When I spotted the typo, I found it amusing. (Who's Brian? Was he one of the superspreaders?) Then I began to wonder: Am I still suffering from pandemic brain?
Maybe, maybe not. I definitely still feel a bit foggy at times, though nowhere near as much as in 2020 and 2021.
One way statistics are useful is that they can capture trends that aren't consistent from person to person, or even from day to day for the same person. This is unprecedented, historically speaking. We can say now that pandemic brain undermines decision-making, not because that was my personal experience or what I heard others say, but because of data from Dr. Buelow's team confirming the existence of this hopefully temporary cognitive decline.
You might say that thanks to statistics, it has become easier now to reason deliberatively as opposed to impulsively about things like the state of our reasoning ability.
Thanks for reading! Feel free to blame any "brain fog" on the length of this newsletter.
Appendix: ADMC scoring
When you administer a test like the ADMC, how can you tell which type of reasoning people are using?
I'm answering this question in an appendix, because the rationale here is a bit more involved than simply measuring how quickly people answer questions, or how reflectively they claim to have answered.
Dr. Buelow and colleagues focused on three dimensions measured by the ADMC.
1. Resistance to framing.
Framing refers to differences in the way the same question is presented. For instance, I could ask for your views of an institution where 35% of graduating seniors claim they never cheated, or I could ask for your views of an institution where 65% of graduate seniors claim they cheated at least once. These are two different ways of framing the same question, and on a test like the ADMC each version appears in different parts of the test. Scoring of the ADMC assumes that deliberative reasoning is reflected insofar as a person responds the same way to each version of the question. In short, deliberative reasoning is resistant to framing effects. Impulsive thinking is more sensitive to the way questions are framed.
In this study, the pre-pandemic cohort showed significantly greater resistance to framing than the pandemic cohort did each time the latter group was tested.
2. Confidence.
When you reason deliberatively, confidence in the accuracy of what you say will be related to how accurate you actually are. For example, when you make an incorrect statement, you'll tend to be less confident in your accuracy than you would've been if you had spoken correctly. The ADMC assumes that when people respond to a set of True/False statements (e.g., "muscles do not burn calories when you are at rest"), the greater the correlation between confidence and accuracy, the more deliberative the decision-making. Impulsive thinking is evidenced by a more constant level of confidence (which looks like overconfidence whenever the wrong answer is provided).
In this study, no significant group differences were found for confidence.
3. Consistency.
When you reason deliberatively, your estimates of risk will be consistent across different time frames. As I mentioned at the outset, the chances of having one car accident in the next year must be less than the chances of having one car accident in the next five years. If you estimate your chances at 3% for a one-year period, for example, the ADMC assumes that a deliberative response to a question about a five-year period would be any estimate greater than 3%. Such an estimate would illustrate consistency. In contrast, any estimate of 3% or less would be scored as impulsive. (Respondents are not expected to know probability theory. Rather, they're classified as deliberative if they simply recognize that the chances of something occurring once are greater when we consider longer periods of time.)
In this study, the pre-pandemic cohort showed significantly greater consistency than the pandemic cohort did each time the latter group was tested.