This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility:

fact-checked

trusted source

proofread

Just believing that an AI is helping boosts your performance, study finds

Just believing that an AI is helping boosts your performance
The participants interacted with the system with their dominant hand using a keyboard and a mouse. Credit: Proceedings of the CHI Conference on Human Factors in Computing Systems (2024). DOI: 10.1145/3613904.3642633

Sometimes it seems like an AI is helping, but the benefit is actually a placebo effect—people performing better simply because they expect to be doing so—according to new research from Aalto University. The study also shows how difficult it is to shake people's trust in the capabilities of AI systems.

In this study, participants were tasked with a simple letter recognition exercise. They performed the task once on their own and once supposedly aided by an AI system. Half of the participants were told the system was reliable and it would enhance their performance, and the other half was told that it was unreliable and would worsen their performance.

The findings are published in Proceedings of the CHI Conference on Human Factors in Computing Systems.

"In fact, neither AI system ever existed. Participants were led to believe an AI system was assisting them, when in reality, what the sham-AI was doing was completely random," explains doctoral researcher Agnes Kloft.

The participants had to pair letters that popped up on screen at varying speeds. Surprisingly, both groups performed the exercise more efficiently—more quickly and attentively—when they believed an AI was involved.

"What we discovered is that people have extremely high expectations of these systems, and we can't make them AI doomers simply by telling them a program doesn't work," says Assistant Professor Robin Welsch.

Following the initial experiments, the researchers conducted an online replication study that produced similar results. They also introduced a qualitative component, inviting participants to describe their expectations of performing with an AI. Most had a toward AI and, surprisingly even skeptical people still had positive expectations about its performance.

The findings pose a problem for the methods generally used to evaluate emerging AI systems. "This is the big realization coming from our study—that it's hard to evaluate programs that promise to help you because of this ," Welsch says.

While powerful technologies like undoubtedly streamline certain tasks, subtle differences between versions may be amplified or masked by the placebo effect—and this is effectively harnessed through marketing.

The results also pose a significant challenge for research on , since expectations would influence the outcome unless placebo control studies were used.

"These results suggest that many studies in the field may have been skewed in favor of AI systems," concludes Welsch.

More information: Agnes Mercedes Kloft et al, "AI enhances our performance, I have no doubt this one will do the same": The Placebo effect is robust to negative descriptions of AI, Proceedings of the CHI Conference on Human Factors in Computing Systems (2024). DOI: 10.1145/3613904.3642633

Provided by Aalto University
Citation: Just believing that an AI is helping boosts your performance, study finds (2024, May 13) retrieved 26 May 2024 from https://techxplore.com/news/2024-05-believing-ai-boosts.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

Blind trust in enhancement technologies encourages risk-taking even if the tech is a sham, finds study

89 shares

Feedback to editors