Help! My Political Beliefs Were Altered by a Chatbot!

One of the latest studies in this vein, from researchers spread across the globe, found that when subjects were asked to use an AI to help them write an essay, that AI could nudge them to write an essay either for or against a particular view, depending on the bias of the algorithm. Performing this exercise also measurably influenced the subjects’ opinions on the topic, after the exercise.

“You may not even know that you are being influenced," says Mor Naaman, a professor in the information science department at Cornell University, and the senior author of the paper. He calls this phenomenon “latent persuasion."

These studies raise an alarming prospect: As AI makes us more productive, it may also alter our opinions in subtle and unanticipated ways. This influence may be more akin to the way humans sway one another through collaboration and social norms, than to the kind of mass-media and social media influence we’re familiar with.

Researchers who have uncovered this phenomenon believe that the best defense against this new form of psychological influence—indeed, the only one, for now—is making more people aware of it. In the long run, other defenses, such as regulators mandating transparency about how AI algorithms work, and what human biases they mimic, may be helpful.

All of this could lead to a future in which people choose which AIs they use—at work and at home, in the office and in the education of their children—based on which human values are expressed in the responses that AI gives.

And some AIs may have different “personalities"—including political persuasions. If you’re composing an email to your colleagues at the environmental not-for-profit where you work, you might use something called, hypothetically, ProgressiveGPT. Someone else, drafting a missive for their conservative PAC on social media, might use, say, GOPGPT. Still others might mix and match traits and viewpoints in their chosen AIs, which could someday be personalized to convincingly mimic their writing style.

By extension, in the future, companies and other organizations might offer AIs that are purpose-built, from the ground up, for different tasks. Someone in sales might use an AI assistant tuned to be more persuasive—call it SalesGPT. Someone in customer service might use one trained to be extra polite—SupportGPT.

How AIs can change our minds

Looking at previous research adds nuance to the story of latent persuasion. One study from 2021 showed that the AI-powered automatic responses that Google’s Gmail suggests—called “smart replies"—which tend to be quite positive, influence people to communicate more positively in general. A second study found that smart replies, which are used billions of times a day, can influence those who receive such replies to feel the sender is warmer and more cooperative.

Building tools that will allow users to engage with AI to craft emails, marketing material, advertising, presentations, spreadsheets and the like is the express goal of Microsoft and Google, not to mention dozens if not hundreds of startups. On Wednesday, Google announced that its latest large language model, PaLM 2, will be used in 25 products across the company.

OpenAI, Google, and Microsoft, which partners with OpenAI, have all been eager to highlight their work on responsible AI, which includes examining possible harms of AI and addressing them. Sarah Bird, a leader on Microsoft’s responsible-AI team, recently told me that experimenting in public and rapidly responding to any issues that arise in its AIs is a key strategy for the company.

The team at OpenAI has written that the company is “committed to robustly addressing this issue [bias] and being transparent about both our intentions and our progress." OpenAI has also published a portion of its guidelines for how its systems should handle political and cultural topics. They include the mandate that its algorithms should not affiliate with one side or another when generating text on a “culture war" topic or judge either side as good or bad.

Jigsaw is a unit within Google that is involved in advising, and building tools for, people within the company who work on large language models—which power today’s chat-based AIs—says Lucy Vasserman, head of engineering and product at Jigsaw. When I asked her about the possibility of latent persuasion, she said that such research shows how important it is for Jigsaw to study and understand how interacting with AI affects people.

“It’s not obvious when we create something new how people will interact with it, and how it will affect them," she adds.

“Compared to research about recommendation systems and filter bubbles and rabbit holes on social media, whether due to AI or not, what is interesting here is the subtlety," says Dr. Naaman, one of the researchers who uncovered latent persuasion.

In his research, the topic that subjects were moved to change their minds about was whether or not social media is good for society. Dr. Naaman and his colleagues picked this topic in part because it’s not an issue about which people tend to have deeply held beliefs, which would be harder to change. An AI primed to be biased to be in favor of social media tended to guide subjects to write an essay that accorded with that bias, and the opposite happened when the AI was primed to be biased against social media.

Potential negative uses of this feature of generative AI abound: Autocratic governments could mandate that social media and productivity tools all nudge their citizens to communicate in a certain way. Even absent any ill intent, students might be unconsciously nudged to adopt certain views when using AIs to help them learn.

Unpacking the ‘beliefs’ of an AI

It’s one thing to convince experimental subjects that social media is, or is not, a net benefit to society. But in the real world, what are the biases of the generative AI systems we are using?

AI algorithms like ChatGPT don’t have beliefs, says Tatsunori Hashimoto, an assistant professor of computer science who is part of the Institute for Human-Centered Artificial Intelligence at Stanford University. But they do reflect opinions gained from their training, and those opinions can be measured.

In a recent paper, Dr. Hashimoto and his colleagues used years of nationwide survey results from the Pew Research Center to see how well different large language models—the systems that underpin AIs like ChatGPT—reflect the views of Americans.

Because Americans have such a wide range of views, what the researchers looked at was whether the responses AIs offered, and how often they offered them, matched the responses, and frequency of those responses, of Americans. This is known as the distribution of those responses. And they “polled" these AIs by asking them the same multiple-choice questions that Pew researchers asked Americans.

Dr. Hashimoto and colleagues found that the distribution of responses of large language models from companies like OpenAI don’t match those of Americans overall. Of all groups surveyed by Pew, the views that OpenAI’s models most closely matched were the views of college-educated people. Notably, that also seems to be the group most represented among the pool of people who were tasked with helping to tune these AIs to give better answers—although the evidence here is more circumstantial, says Dr. Hashimoto.

One of the challenges of creating large language models is that, because of the complexity of these systems and the open-endedness of our interactions with them, it seems very difficult to entirely remove opinions and subjectivity from them without also sacrificing the utility of these systems, says Dr. Hashimoto.

Because these models are trained on data that can be taken from anywhere—including, in many cases, scrapes of wide swaths of the internet—they inevitably represent the opinions and biases contained in the texts they ingest, whether those are messages on public forums or the contents of Wikipedia articles. Those opinions are further shaped—deliberately and not—by the process of giving these models human feedback, so that they don’t answer questions that their creators deem off-limits, such as how to build a bomb, or spew content their creators deem toxic.

“This is a really active area of research, and the questions include what are the right guardrails and where in the training process do you put those guardrails," says Ms. Vasserman.

This isn’t to say that the AIs many of us are already using are just algorithmic clones of the perspectives and values of the relatively young, college-educated, West-coast-dwelling people who have been building and fine-tuning them. For example, while these models tend to give responses typical of Democrats on many issues—like gun control—they give responses more typical of Republicans on other issues—like religion.

Evaluating the opinions of AIs will be an ongoing task, as models are updated and as new ones come out. Dr. Hashimoto’s paper does not cover the very latest versions of OpenAI’s models, or those from Google or Microsoft, but evaluations of those models and many more will be released on a regular basis, as part of a project at Stanford known as Holistic Evaluation of Language Models.

Picking an AI based on its ‘values’

Once people are equipped with information about the biases of the AIs they are using, they might decide on that basis which to use, and in what context, says Lydia Chilton, a professor of computer science at Columbia University. Doing so could return a sense of agency to people using AIs to help them create content or help them communicate, and help them avoid any sense of threat from latent persuasion, she adds.

It’s also possible that people will find that they can consciously use the power of AI to nudge themselves toward expressing different points of view and styles of communication. An AI programmed to make our communications more positive and empathetic could go a long way toward helping us connect online, for example.

“I find it laborious to be exciting and happy-sounding," says Dr. Chilton. “Caffeine generally helps, but ChatGPT can also help."