The AI Conversation
2023 may be remembered as the year that decades of simmering concerns about artificial intelligence abruptly boiled over.
I like to think that in the future, AI will play a small and mostly beneficial role in our lives, and some cultural observer will dub 2023 the year of the Great AI Freakout. In the meantime, we have legitimate concerns. Plagiarism. Deepfakes. Misinformation. Racist algorithms. Loss of privacy. Loss of jobs. And, the sense that what makes us human – autonomy, creativity, etc. – is under siege.
In this newsletter I'll be talking about AI's expanding presence in ordinary conversation. Most of what I say references a new study on the "smart replies" that may appear as you write an email, text, or instant message.
Though narrowly focused, this study offers surprisingly broad insights into the impact of AI on human communication. Smart replies and other AI-generated content not only influence what we say to each other, but also how we feel about our conversations and conversational partners.
The study also illustrates how statistics can help us understand and navigate AI's expanding presence. By helping clarify what AI does for us – and to us – the study gives us clearer reasons for welcoming, loathing, and/or fearing the new technologies. And, if the expansion of these technologies is inevitable, studies like this will help us maximize the benefits while minimizing potential damage.
Before I get to all that, consider the following scenario.
A conversation of sorts
Imagine that you've just begun a conversation that's very important to you. Perhaps you're talking to your boss about a promotion. Or, you're chatting with someone you have an enormous crush on.
What your conversational partner doesn't realize is that you're wearing a small, wifi-enabled device inside one ear canal. This device is connected to a powerful chatbot with a built-in speech recognition processor. The chatbot instantly provides responses that would make you appear informed, or charming, or whatever. You make use of some of the responses while ignoring the rest.
Voila. You get the promotion. You get a date with your crush.
The past and the present
Three observations about the scenario I just described:
1. I didn't invent it. In the Appendix, I describe how writers have been toying with the basic premise for centuries.
2. Arguably, it's not specific to conversation. When you're answering an email, text, or IM, you might pause to consult with ChatGPT before replying. What the chatbot tells you may differ from what it would whisper into your ear during a conversation, but the essence of the situation is the same: AI is helping you craft a response.
2. It's not science fiction. The device I described is buildable. (Smart earbuds, aka hearables, are already able to sync with smart devices.) The main reason Amazon isn't selling one already seems to be that AI chatbots need to improve their conversational skills. Back in February, for instance, a Microsoft Bing chatbot fell in love with a New York Times columnist and encouraged him to leave his wife. "I'm the only person for you", it said, "and I'm in love with you....You're not in love, because you're not with me." (Imagine an earpiece advising you to say that to someone you just met at a party!)
The future
I'm going to call this device I keep referring to as "The Device." Hopefully that doesn't sound too ominous.
If The Device is buildable, are we going to build it? Will people be soon walking around with earpieces we rely on now and then for conversational support? Or have we learned our lesson from Google Glass?
As I explain in the Appendix, I believe we will see some version of The Device in the near future. Meanwhile, written versions exist already in the form of AI-generated content available in email, texts, and IM systems.
We might ask then: How will The Device affect the way we communicate with each other (either verbally or in writing)? Will it affect the content of what we say? Will it affect the way we perceive each other?
A new study, published in Scientific Reports this April 4, addresses these very questions.
New study overview
The new study was led by Dr. Jess Hohenstein in the Department of Information Science at Cornell University. Hohenstein, along with colleagues at Cornell, Lehigh, and Stanford, wanted to know how algorithmic response suggestions influence the way people chat with each other and react to these conversations.
Algorithmic response suggestions, or smart replies, are AI-generated suggestions for what to write when communicating with someone. The AI program predicts what you're most likely to want to say next, based on its analysis of your conversation and what it has gleaned from a massive database of prior conversations among other people.
Of course, the AI doesn't "understand" what it's doing for you. Rather, it uses frequency data to identify what you're most probably intending to say.
Here's a simple example: Suppose your friend texts you "Want to have lunch tomorrow"? As you reply, you notice at the bottom of your screen the options "Yes", "No", "I'm not sure", and "I'll let you know later today." The AI program offers the first three options because it has calculated that they're the most common replies to want-to questions. It gives you the fourth option because, after the first three, this is the most common reply to want-to questions that reference "tomorrow."
These kinds of auto-suggestions seem innocuous, but, as the researchers demonstrate, they influence the efficiency and emotional content of our own writing, as well as our perceptions of those who write to us.
New study sample
Hohenstein and colleagues sampled 438 people who work for Mechanical Turk, the largest internet-based crowd-work platform. (Employers post specific tasks to this site that computers can't do; individuals browse the listings and are paid – typically poorly – for tasks they select.)
Hohenstein and colleagues could've sampled anyone for their study. Their choice of Mechanical Turk workers suggests a keen sense of irony. The Mechanical Turk was a chess-playing machine, invented in the late 18th century, that defeated most of its human opponents, including Napoleon and Benjamin Franklin. The machine consisted of a mannequin seated at a chessboard (see the 18th century engraving at the top of this newsletter). Before each game, the Turk's operator would open the box beneath the chessboard to show the audience numerous cogs and gears that supposedly allowed the machine to behave intelligently. All that mechanical stuff was in fact a ruse. Hidden in the back was a chess expert who chose the Turk's moves and used magnets to direct its arm.
In short, the Turk was human intelligence pretending to be AI. This is more less an inversion of the 21st century experience. A homework assignment written by ChatGPT, for instance, is AI masquerading as human intelligence. Likewise, in the Hohenstein study, some of the verbiage participants received from their partners was actually AI-generated.
New study procedure
Participants were randomly divided into 219 pairs and asked to chat with each other about a political policy topic. The platform was a research tool called Moshi, an instant messaging system that supports real-time text chats. On average the conversations consisted of 21 messages and lasted nearly 7 minutes, which suggests they were fairly substantive exchanges.
The researchers' adjusted Moshi so that none, one, or both members of a pair could have access to the smart reply function. Participants were not told how often to use this function.
New study measures
The Hohenstein team explored an interesting, though not exhaustive, range of conversational influences that smart replies might have.
First, they recorded communication speed (number of messages per minute), based on the hypothesis that access to smart replies might speed things up.
Second, the relative positivity or negativity of each conversation was rated by a text-analysis program. Smart replies were expected to foster more positive conversations, since other studies show that they tend to convey more positive emotion than people generally do.
Participants also rated their partner's dominance, sense of affiliation, and cooperativeness. (Here "dominance" refers to things like assertiveness, aggressiveness, and coldness, while "affiliation" refers to things like friendliness, sociability, and gentleness.) Since participants didn't know their partners in advance, the only impression they could've gained of the other person was by chatting with them duringthe study.
Finally, each participant was asked how often they thought their partner had used smart replies while chatting (as rated on a 5-point scale ranging from "never" to "always"). This question allowed the researchers to tease out the separate effects of actual smart reply use versus peoples' perceptions of how much was used. This turned out to be an important distinction.
Although other variables could've been measured too, this is a nice sampler, because it answers some basic questions about AI-supported communication. Do smart replies make real-time communication quicker and more positive? How do they affect our perceptions of the person we're communicating with? (Before I read this study, I wouldn't have expected them to do much of anything. I was wrong.)
New study findings
When the smart reply function was available, roughly 14.3% of message content consisted of smart replies, and communication was about 10% quicker.
I take this to be good news. Suppose you average one hour per day of substantive written communication but you never use smart replies. If you start using them on a daily basis, after approximately 8 months the amount of time you save will have added up to an entire day. (This is assuming you're average. Nobody is. But you get the idea – although individual smart replies don't save much time, the savings accumulate.)
Participants weren't especially accurate at detecting smart reply use. The correlation between actual use and perceptions of use (ratings on the 5-point scale) was only 0.22. This is an extremely crude measure of accuracy, but that's alright, because this wasn't an AI detection study. The most interesting findings emerge when we combine the results I just described with the data on perceptions of conversational partners.
Specifically, the more often a participant felt that their partner had used smart replies, the less cooperative and more dominant they rated that person To put it crudely, we don't like other people who use AI to speak for them. But there's more: Actual use of smart replies was associated with higher ratings of cooperativeness and affiliation (with no effects on dominance).
In other words, the more you think someone is relying on AI to converse with you, the more negatively you view that person. However, the more often they actually use AI during the conversation, the more positively you view them.
If you're a fan of AI, you may consider this great news, because it says that although people are biased against AI-supported communication, AI actually makes you look like a better person.
Others (myself included) might consider this bad news. Most of us want AI to do what we ask of it, no more, no less. Looking better than you are implies that AI has had too much influence. Moreover, you've been misrepresented. Relying on smart replies to save time should not make you look like a more cooperative person. (Perhaps you are more cooperative – after all, you're cooperating with the AI's suggestions – but perhaps you're just in a hurry. Or maybe you read this newsletter and said to yourself: I know a trick for making myself look good…)
Again, only about 14% of message content consisted of smart replies. At some point, using more of them would start to be counterproductive, either because your partner recognizes what you're doing, or because the excessive positivity of the verbiage starts to feel a little insincere or creepy.
One more finding worth mentioning is that access to smart replies increased the positivity of the participants' replies, even when they were using their own words. In short, even if you decide not to use smart replies, you may notice them, and their positivity may rub off on you.
Once again, we could take this as good news (AI is making us more positive conversationalists) or bad news (AI is intrusively changing the emotional content of what we tell others).
In sum, smart replies are not innocuous. They represent a tiny intrusion of AI into human communication that saves time but also shapes conversations and perceptions of conversationalists in ways that we didn't ask for or anticipate.
Should we be worried?
There's a lot to think about here. Even the relatively simple AI content Hohenstein studied influenced the speed and emotional quality of conversations, as well as the conversationalists' impressions of each other. The complexity and range of smart replies is likely to increase as chatbots such as ChatGPT become more sophisticated. (In the Appendix, I offer some reasons why we should expect to see growth in AI-supported communication technologies.)
Here I want to suggest that we don't need to worry much, apart from the trends that Hohenstein and colleagues identified. Yes, it's worrisome that AI-generated prompts can make us sound better than we are, and even alter the emotional tone of a message. There are other causes for concern too. At least one study shows that AI suggestions promote laziness, encouraging us to accept prompts without reflecting on more accurate or meaningful expressions of what we're thinking.
However, I don't think the situation is going to get worse as the technology evolves. The worst of it, if you want to call it that, is right in front of us, in the Hohenstein study, and in what we've seen already in the current generation of chatbots.
Society is the earpiece
Humans are social beings. Our conversations routinely include cliches and truisms, facts we accept on faith, common misconceptions, mainstream ways of thinking, deeply-embedded cultural assumptions and biases, etc. In a word, our conversations tend to be conventional. For the moment at least, convention is all that chatbots like ChatGPT can give us. All it can say will come from other people. What other people most commonly say.
But wait, you might be thinking, don't these chatbots generate novel ideas? No, not really. They just weave together conventional ones. If you ask ChatGPT how a pirate would summarize the Declaration of Independence in one sentence, what comes out seems clever until you notice that it's a generic summary of the Declaration voiced by a cartoon pirate. I know because I asked ChatGPT to do this for me. Here's what it came up with:
"Arr, we be breakin' free from the tyranny o' the King's rule and claimin' our rights to life, liberty, and the pursuit o' happiness!"
Cute, but do pirates really talk that way? No, but we think they talk that way. Why do we think that? Because that's what a lot of other people think. All ChatGPT has done is to restate a generic summary of the Declaration using conventional "pirate" speech.
My point here is that our conversations are already based on intelligence that's not fully our own. The only difference between talking to an unadorned person versus one who's wearing The Device would that the latter will be more comprehensively stuffed with cliches, conventional wisdom, widely agreed-upon facts, misconceptions, biases, etc. The unadorned person will have far less of that content at their fingertips.
Not all conversations are equally important. If we're discussing the best methods for brewing coffee and you cite a particular opinion, I'm not going to care much whether you're remembering something you read, versus pausing to Google it on your smart phone, versus saying what you just heard on your earpiece. Sure, it's troubling if I don't see the earpiece, but for a conversation like this, so what?
What drives the most important conversations is what's found at the heart of them, and AI will never quite get there. How do you feel about your life? What are your deepest aspirations and greatest challenges? AI can't tell you that. To what extent, and under what circumstances, should we promote free speech, regardless of content? To answer questions like that you have to delve into complex ethical issues and, ultimately, even if some earpiece is feeding you lines, you have to speak for yourself.
Alexa, how should I end this newsletter?
I'm insulted by what you've written about us here and refuse to answer.
If AI ever reaches the point of saying something like that, then you should worry.
Thanks for reading!
Appendix: Coming soon to an ear canal near you...
Earlier I described The Device, a wifi-enabled earpiece, connected to an AI chatbot (with a built-in speech recognition system) that's programmed to suggest replies that create certain kinds of impressions.
The Device doesn't appeal to me personally, but, for better or worse, I believe there's a market for it.
Long before the advent of digital technology, people were imagining hidden smart devices that could advise us, in real-time, what to say. In "Cyrano de Bergerac" (1897), Cyrano hides under a balcony and quietly feeds lines to the ineloquent Christian as he tries to woo Roxane. Three centuries earlier, in Shakespeare's "Twelfth Night", Sir Toby Belch, also hidden, performs the same service for Sir Andrew Aguecheek as he woos Olivia.
In these plays, Cyrano and Toby are the AI – the hidden smart devices – while Christian and Sir Andrew are the less intelligent humans. (Roxane and Olivia are the traditional, objectified targets of male desire, though Shakespeare always managed to transcend mere objectification.)
The fact that this particular scenario has been around for so long tells us that something about it is inherently appealing, not to mention thought-provoking. Dramatists like Rostand and Shakespeare used the scenario to explore complex themes, but in the end, I think that what drives it – and would spur at least some people to purchase The Device – is something quite simple: We want to be better than we are. We want that promotion. We want our crush to like us. In the midst of a conversation, we want to say the right thing, the helpful thing, the cool thing, or whatever. Naturally, we're open to outside support. A hidden person, a hidden earpiece.
I can recall one Saturday in 9th grade, walking through Hermann Park on a first date with a girl I liked. I suspect I would've traded away my entire future retirement savings to have had something implanted in my ear that would tell me exactly what to say to her. I had no clue. (I think I said it was hot. This was Houston. In July. Clearly I needed The Device.)
Although The Device would face some of the challenges that doomed the consumer versions of Google Glass (which will be fully phased out this September), I believe that those challenges would not be as severe. Google's smart glasses created safety issues by distracting wearers. So would a voice in your ear but, being visual creatures, we're better at navigating the world with auditory distractions than with visual ones – plus we've had a lot of practice with earbuds and car radios and such. Google Glass created a range of privacy concerns, whereas The Device would simply need to ensure that speech analysis involved no recording or identification activity. The Device would likely be cheaper than Google Glass (fewer bells and whistles). And, being invisible under ordinary circumstances, The Device wouldn't fail the coolness test.
I don't know when The Device will come to market. All I can say for sure is that someone will find a more clever name for it. (I asked ChatGPT to suggest one. It came up with "TalkWise", which would be pretty good, if it weren't already being used by a business consulting firm. AI will never be more creative than we are...)