Who Cheats?
Question: What do the Navy SEALs, Major League Baseball players, international chess masters, and Florida teachers have in common?
Answer: Each group made the news last week in connection with allegations of cheating. (See Appendix A for details.)
Here's another question: Why do people cheat in competitive situations?
The answer to that question might seem too obvious to be worth mentioning: People cheat to increase their chances of winning. (Or to avoid disadvantage, if they assume that other competitors are cheating.)
Here's a twist though: Not everyone cheats, even when the stakes are high. We'll never know exactly how many Olympians, CEOs, or undergraduate applicants cheat, because some get away with it – and, in anonymous surveys, some won't admit to it – but there's no evidence of universal cheating in any high-stakes competitive domain. Even when it's considered rampant, numerical estimates of cheating are less than 100% and vary widely. My favorite example is captured by the title of a VeloNews article: "CIRC Report: 20–90 percent of modern peloton still doping." (I smiled when I saw that. 20 to 90% of professional cyclists are doping? It's hard to be wrong with an estimate that broad. Even so, the headline is misleading. In the original CIRC report, one professional cyclist estimated that 20% of his fellow competitors use performance-enhancing drugs, while another estimated a rate of 90%. However, most of the interviewees guessed that the percentages of cyclists who do vs. do not dope are roughly the same. That suggests a lot of cheating, but nowhere near 100%.)
So, given that not all competitors cheat, let's rephrase the question: Why do some people cheat in competitive situations? Who are these people? What motivates them?
Overview
I'll start with a pair of studies, one published in 2016 and the other, a rebuttal, published last month. Both studies illustrate that competently-performed statistics can't save weak methods. After I've grumbled a bit about these studies, I'll describe what we've learned from better-quality research on cheating, and I'll show that a simple statistical observation can help us make sense of an otherwise overwhelming variety of findings.
Do winners cheat more?
According to an old proverb, "winners never cheat, and cheaters never win", but a 2016 study seems to show that winners actually cheat more than losers do.
This study, reported in the National Academy of Sciences, got a lot of attention, both in and out of academia (it was covered by the Washington Post, the BBC, and Scientific American, among others). The researchers, Drs. Amos Schurr and Ilana Ritov at Ben-Gurion University of the Negev, addressed a simple question: Who's more likely to cheat after participating in a competition: Winners or losers? You might expect it's the losers, because they've learned that they're at a disadvantage. However, that's not what the researchers found.
In the main experiment, undergraduates were randomly paired up for a simple game of estimating the number of objects on a screen. The winner received a set of JVC ear buds. Next, students were randomly reassigned to different pairings in order to play a brief game of dice. Each pair of students was given a packet of money and told that one throw of the dice would determine how much money each person would get. One player served as the "thrower" and the other was the "receiver". The thrower placed a cup over the dice, shook it, and then looked through a hole in the cup to determine how much money each player should get. (The receiver could not see.) Cheating occurred to the extent that throwers claimed more money for themselves than the dice actually indicated.
The researchers found that throwers who had won the earlier estimation game cheated more than the throwers who had lost, and more than a control group who hadn't played the estimation game.
Why did the winners cheat more? According to the researchers, winning creates a sense of entitlement. The logic goes like this: If I win a competition, I'm a winner, and thus I deserve to win. The fact that I deserve to win justifies any means of winning, including cheating.
Study limitations
I don't want call this an awful study, but that's the kindest adjective I can think of. In the experiment I described, data was only available for 23 students. Cheating wasn't directly observed, because the researchers couldn't see into the cup either. Rather, cheating was inferred from how much money each group of throwers claimed for themselves, on average, assuming that the outcomes of the dice for each group would represent what we expect to see by chance. The researchers thus succumbed to a well-known statistical fallacy. To illustrate, notice that if you throw a pair of dice and sum up the result, the chances of getting, say, a 12 (i.e., a pair of sixes) is 1 in 36, or about 2.78%. Now, if you throw the dice a billion times, you will get 12s something very, very close to 2.78% of the time, if not exactly that often. However, if 23 people throw the dice once, as they did in the present study, there's likely to be a lot more deviation from that 2.78% figure, and from the expected frequencies for every other outcome (11s, 10s, 9s, etc.). The researchers' fallacy was in assuming, for example, that if throwers reported 12s more than 2.78% of the time, they must've been cheating. In fact, that's not necessarily the case, because a small sample of throws will often yield more or less than that number of 12s. In short, the inferred rates of cheating in this study were inaccurate to some unknown extent.
More broadly, the study can be faulted for reliance on on a highly contrived, low-stakes competition. The total amount of money at stake in the dice game was the equivalent of 3 US dollars, yet the researchers claimed to have revealed something about cheating in real-life, high-stakes situations. For example, they referenced the 2015 Volkswagen emissions scandal, in which VW intentionally created software that would cause their diesel engines to meet emissions control standards only during testing, but not during actual road use. Volkswagen sold approximately 11 million vehicles with this software between 2009 and 2015. Clearly there was a lot more at stake for the company than 3 dollars.
2022 rebuttal
In August of this year, for a Royal Society Open Science article, a team led by Andrew Colman at the University of Leicester attempted to replicate and improve upon the study I just described. For instance, Colman and colleagues ran a power analysis which showed, mathematically, that the 2016 sample was too small, and that their own sample of 259 undergraduates was more than sufficient. An additional improvement was that participants were asked to complete psychological tests that measured self-confidence, perceived luckiness, entitlement, shame, and aversion to inequality. The methodology was otherwise highly similar to that of the 2016 study.
Colman et al. reported three main findings:
—Winners, losers, and a control group all showed a small extent of cheating behavior in the dice game.
—The three groups did not differ in extent of cheating. This is the key finding, because it directly contradicts the 2016 study.
—The only psychological variable linked to cheating was aversion to inequality. There was a (very) slight tendency for people to cheat more if they were more accepting of unfairness in society.
Study limitations
This study is generating some buzz among researchers in the field (and in the media, although to a lesser extent than the 2016 study did). And yet, once again, though I hate to say it, it's an awful study.
Part of the reason for my gloomy assessment is that the researchers used the same kind of contrived, low-stakes competition as in the earlier study, but they also claimed to reveal something about real-world cheating – they refer, for example, to academic dishonesty and "burgeoning problems of tax avoidance and evasion by wealthy people in developed economies." Here again, I suspect that what undergraduates do in an quirky lab study where $3 is at stake doesn't necessarily generalize to what millionaires do when they cheat on their taxes.
Even if I'm wrong, and the results do generalize, this study doesn't tell us much of anything about cheating. It's good to know that winners are not more likely to cheat than losers, but who is more likely to cheat? The association between cheating and acceptance of injustice was, statistically speaking, very weak. (In Appendix B I describe an additional concern with this kind of research.)
Here's the best part of the study: In the discussion section, Colman and colleagues summarize their work by writing that: "Our studies have not provided much enlightenment as to what leads some people to cheat." Kudos to the researchers for acknowledging that so openly!
Who cheats?
There are enormous literatures on predictors of dishonest behavior such as cheating in competitive situations. I slogged through or skimmed a number of high-quality meta-analyses and individual studies this week, and here's my conclusion: Scads of variables influence cheating behavior. I'll provide a few examples here, then describe a statistical detail that can help us frame this information.
1. Situational variables.
Reward size:
In many studies, no relationship is found between cheating and the size of the anticipated reward. This is counterintuitive, as we tend to expect greater rewards to consistently spur more cheating. Only a few studies show that greater rewards inspire more cheating (and when they do, they only do so by a small margin).
Risks:
In many studies, greater perceived risk of getting caught is associated with less cheating, by a small margin. (In most studies, the "rewards" and "risks" are small, because the studies are carried out in lab settings. However, some studies report interviews with people who have the opportunity to cheat in actual high-stakes settings – e.g., professional athletes who discuss their views on performance-enhancing drugs.)
Social norms:
In many studies, normative pressure to cheat increases cheating behavior, by a small margin. People are slightly more likely to cheat when they perceive that cheating is widespread among competitors, and/or when they're pressured to do so by employers, colleagues, coaches, peers, and in some cases even family members. Merely competing against larger numbers of opponents is predictive of more cheating, perhaps owing to the assumption that when there are more competitors, the incidence of cheating will be greater.
Ethical reminders:
In many studies, reminding people about ethical norms reduces cheating behavior, by a small margin. Reminders could be in the form of existing rules (e.g., honor codes) or just comments made immediately before a competitive activity. (Data on social norms and ethical reminders come from lab studies as well as interviews with people such as CEOs and professional athletes.)
2. Personal variables
Prior experience:
In many studies, having cheated in the past predicts more cheating in the future, by a small margin. (As with other contributors to cheating discussed here, the influence of prior cheating could be direct, indirect, or explained by other variables. For example, if a person thinks that cheating is just fine, it's not accurate to say they'll cheat in the future because they've cheated in the past. Rather, they cheated in the past and will continue to do so in the future because they think cheating is acceptable.)
Gender:
In some studies, men cheat more than women do, by a small margin. (Sorry guys.) In other studies, gender differences are not observed.
Personality et cetera:
Personality, self-esteem, moral beliefs, and attitudes toward winning have all been linked to cheating. For example, in some studies, higher levels of the so-called Dark Triad predict more extensive cheating, by a small margin. The "Dark Triad" consists of Machiavellianism (i.e., manipulativeness), narcissism, and psychopathy (antisocial tendencies such as selfishness, callousness, and lack of remorse).
This is an incomplete list, and I've left out countless interactions that have been identified. There's also a seemingly endless list of variables that predict cheating behavior within specific domains. For example, athletes with more positive attitudes towards performance-enhancing drugs are, unsurprisingly, more likely to engage in doping themselves, by a small margin. And, assuming that academic performance can be considered a competitive activity, some studies show that undergraduates cheat more than older adults do, by a small margin, though the differences may be attributable to age. Even more specifically, some studies show that business and economics majors cheat more than students from other majors do, by a small margin.
You probably noticed that my descriptions of every variable included the phrase "by a small margin." That's an important qualification. None of the variables, by themselves, strongly predicts cheating.
Good news
When many variables affect some outcome, but none of those variables is very influential in itself, then, in some cases, there's no point in trying to predict the outcome, because you'll never achieve much accuracy. You can focus instead on promoting that outcome, or, if it's undesirable, like cheating behavior, you can focus on preventing it.
To illustrate what I have in mind here, consider for a moment a classic, highly influential model of academic cheating developed by Dr. Bernard Whitley in 1998 and still discussed in the literature. This model is summarized in the figure below. You don't actually need to read every detail in the figure. Just notice that it contains a large number of boxes and arrows. Each box identifies a variable that's related in some way to cheating behavior (which is the box at the far right, in the middle.) The arrows indicate where one variable is a significant predictor of another one. For example, starting from the lower left, we can see that greater test anxiety leads to lower expected performance on upcoming tests, which then increases the expected benefits of cheating. Greater expected benefits of cheating create the intention to cheat, and the more intent a person is on cheating, the more likely they'll engage in actual cheating behavior. This is just one example of the dozens of interrelationships among variables presented in the figure.
If we added statistical information to this figure, we'd find that each effect size was small. (Appendix C contains a non-mathematical explanation of effect sizes.) Even though the numerous arrows in the figure indicate significant relationships, the strengths of these relationships are small in every case. And, these small effects don't add up in a very useful way. You can't get more accurate prediction of cheating behavior through any means of combining information from all the boxes.
So, if you were an instructor, what should you do with this information? I would argue: Nothing. Because even if you had information on each of your students for these variables, you still wouldn't gain much accuracy at predicting who will vs. won't cheat on your next test. Concretely speaking, you'd be wrong more often than you'd be right.
So, the "good news" is that whether you're an instructor, a college admissions officer, a professional sports adminstrator, a financial markets regulator, or anyone else involved in some competitive enterprise, there's no need to waste time and resources trying to predict who's more or less likely to cheat. Trying to prevent cheating is worthwhile, for obvious reasons, but predicting it is close to futile. This is a bit of a radical conclusion, but it's the only one that makes sense to me, given how weak the statistical effects are. People are complicated. Whether or not a person cheats in a particular situation is complicated, and it just can't be predicted with much accuracy in most cases. This should be a relief to everyone except the researchers who spend time on identifying predictors of cheaters. Hopefully, they will someday be able to create figures like the one above in which the variables are strongly interrelated.
Thanks for reading!
Appendix A: Cheating in the news since last week
1. The death of a Navy SEALs candidate earlier this year lead to a NY Times article, updated September 5, pointing to widespread use of performance-enhancing drugs among men attempting to survive the SEALs grueling selection and training course (over 70% fail). Drugs of choice include steroids and even Viagra for coping with swimming-induced pulmonary edema.
2. As for Major League Baseball, stories this week continued to track the fallout from Fernando Tatis Jr.'s 80-game suspension in August for doping with Clostebol. Tatis, a star for the San Diego Padres, claims that he unintentionally exposed himself to the banned substance via a medication for ringworm.
3. On September 5, world chess champion Magnus Carlsen withdrew from an international competition after losing the previous day to a much lesser-renowned player. That player, Hans Niemann, has confessed to cheating in online chess competitions, but denies ever doing so in face-to-face matches, including the latest one with Carlsen. Many in the chess world nonetheless suspect Niemann of cheating in that match. If he did cheat, his methods haven't been identified yet. (Elon Musk, probably unhelpfully, has suggested vibrating anal beads.)
4. On September 9, three high school teachers in Pasco County, Florida, were arrested for fraudulently helping students pass agricultural certification exams. Teachers gave students the exams as “study guides”, took the exams with students, and persuaded the students to let them photograph their exams, apparently intending to share the photos with other test-takers.
These are the most widely-covered cheating-related stories these past two weeks, but there are others. For instance, on September 8 Lisa Gianelli was sentenced to 42 months in prison for conspiring to sell and distribute performance-enhancing drugs such as blood-builders and vasodilators to trainers and others in the horseracing industry. Gianelli is just one of 30 people in her profession currently being prosecuted by the federal government in connection with PEDs.
Appendix B: Winning and cheating in lab studies
Lab studies on winning and cheating like the two I described here are arguably doomed from the outset, because they rely on a simple classification of people as "winners" or "losers". Winning and losing are not discrete outcomes in many real-world situations. (How does a CEO view her work over the past year? How does a producer view a movie that turns a small profit and garners mixed reviews from critics? How does an applicant view acceptance to Texas A & M when the preferred school was UT Austin but the applicant would be satisfied with either option?)
Even in scenarios like athletic contests, where winning and losing are objectively defined, competitors typically typically win some and lose some, and so, when a person enters their next contest, how they performed in the previous one might not capture how they view themselves. If they won last time, they might still not consider themselves "winners" if they feel they got lucky, or that on the whole they're not performing as well as they'd like. On the other hand, if they lost recently, they might nonetheless consider themselves "winners", because they're still in the game – they're maintaining their status as professional athletes, or whatever.
Finally, even if competitors could be divided into winners and losers, any association between winning and cheating, in real-world scenarios, would be difficult to interpret. For some people, like Lance Armstrong and Mark McGwire, cheating helped them win. In other words, they didn't cheat because they were winners. They won (in part) because they cheated. For others, there's a slightly different causal trajectory. They're devoted to winning at all costs, and so they'll do anything to win. For these people, whether or not cheating helps them win, they don't cheat because they're winners. They win owing to some combination of innate talent, hard work, luck, etc., and it just so happens that when a suitable opportunity arises, they cheat. This is the interpretation that folks like Lance Armstrong and his supporters favor: The argument is that Armstrong would've been a winner no matter what, but he chose to cheat because opportunities presented themselves (and many of his competitors were already doing so).
Appendix C: Effect sizes
"Effect size" can be informally defined as the strength of the relationship between variables. This is different from statistical significance.
For example, by means of correlational methods, you would probably find a significant relationship between body weight and foot size, because bigger people tend to have bigger feet. But the relationship is "significant" only in the sense that if you sample enough people, the correlation between body weight and foot size would be greater than expected by chance. The effect size would be small, because even though bigger people tend to have bigger feet, there are a lot of exceptions to that pattern.
On the other hand, if we look at shoe size and foot size, we'll not only find a significant relationship, but also a huge effect size. The larger the shoes, the larger the feet, with very few exceptions.
As you can see, not all variables would perform equally well at predicting foot size. IQ wouldn't work at all. Body weight would work moderately well, but not very precisely. Shoe size would work nearly perfectly.
When it comes to predicting cheating in competitive situations, we don't have anything like shoe size to work with. We don't even have anything quite as good as body weight. We just have innumerable variables that are weakly related to cheating – significant relationships, small effect sizes. (If you've taken some stats, you've probably already inferred that my discussion of effect sizes in this newsletter mostly refers to standardized beta coefficients. For example, if these coefficients were added to the figure I provided, almost all of them would be quite small.)