When AI disagrees, people don’t change their original view– www.futurity.org

When an AI chatbot agrees with our reasoning in resolving a social dilemma, we may become more confident in our opinions. But when the chatbot disagrees, we may simply dismiss the challenge rather than reconsider our views.

That’s according to new research from the University of Michigan examining how people respond when artificial intelligence weighs in on personal and interpersonal dilemmas.

The study, appearing in Computers in Human Behavior Reports, highlights a challenge for developers trying to reduce the tendency of chatbots to agree too readily with users.

The authors note that AI models are often designed to be agreeable, which can lead them to validate users’ reasoning even when that reasoning may be flawed. They describe how this kind of confirmation can reinforce harmful ideas and distorted perceptions of reality.

“Developers have been working to make AI models more willing to disagree when necessary, but having an AI disagree with the user does not necessarily mean that people will reconsider their own views,” says Atakan Atamer, the study’s co-lead author and UM psychology doctoral student. “Instead, they may simply discount the AI’s response.”