This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility:

fact-checked

peer-reviewed publication

trusted source

proofread

Can large language models replace human participants in some future market research?

ChatGPT
Credit: Unsplash/CC0 Public Domain

Do market researchers still need to conduct original research using human participants in their work? Not always, according to a new study. The study found that thanks to the increasing sophistication of large language models (LLMs), human participants can be substituted with LLMs and still generate similar outputs as those generated from human surveys.

The study is published in Marketing Science. The article, "Determining the Validity of Large Language Models for Automated Perceptual Analysis," is authored by Peiyao Li and Zsolt Katona, both of the University of California, Berkeley; Noah Castelo of the University of Alberta and Miklos Sarvary of Columbia University.

According to the research, agreement rates between human- and LLM-generated data sets reached 75%–85%.

"LLMs can be used to generate text when given a prompt on certain generative Artificial Intelligence (AI) platforms," says Li. "Our research focused on perceptual analysis and the use of automated for certain product categories."

To conduct their research, the study authors used LLMs to tap data that is broadly available on the internet. They developed a and workflow that allows market researchers to rely only on an LLM to conduct market research. As a result, they demonstrated that LLM-powered market research can produce meaningful results and even replicate human results.

"It is important to note that with LLMs, while market researchers may not require interviews with human research subjects, the ultimate data does originate from human beings, using available data," says Katona. "LLMs have been engineered to accurately replicate human responses based on machine learning of actual human perceptions, attitudes, and preferences."

Castelo added, "The core LLM takes a prompt as an input and generates a continuation of text as output. With proper prompting, the LLM can then generate comparisons and assessments of various brands or products in a given category and produce results that are, at the moment, 75%–85% in agreement with research featuring ."

The researchers believe that for some product and brand categories, their new method of fully or partially automating market research will increase the efficiency of market research by speeding up the process and potentially reducing cost. At the same time, they caution that fully automated market research without human input may not be accurate for all product categories.

"While we are very excited about the possibilities we've seen through our research, we recognize that this is just the beginning and going forward, LLM-based market research will be able to answer more nuanced questions as the market research field begins to tap and develop its potential," says Sarvary.

More information: Peiyao Li et al, Frontiers: Determining the Validity of Large Language Models for Automated Perceptual Analysis, Marketing Science (2024). DOI: 10.1287/mksc.2023.0454

Journal information: Marketing Science
Citation: Can large language models replace human participants in some future market research? (2024, April 9) retrieved 27 May 2024 from https://techxplore.com/news/2024-04-large-language-human-future.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

Large language models in health: Useful, but not a miracle cure

14 shares

Feedback to editors