Emotional AI and gen Z: The attitude towards new technology and its concerns

AI has ubiquitous presence in technology. Yet, it had been lacking a crucial feature: the ability to engage human emotions.

This is changing, however. Algorithms that can sense human emotions and interact with them are quickly becoming mainstream as they come embedded in existing systems. Known as "emotional AI," the new technology achieves this feat through a process called "non-conscious data collection"(NCDC), in which the algorithm collects data on the user's heart and respiration rate, voice tones, micro-facial expressions, gestures, etc. to analyze their moods and personalize its response accordingly.

However, the unregulated nature of this technology has raised many ethical and privacy concerns. In particular, it is important to know the attitude of the current largest demographic towards NCDC, namely Generation Z (Gen Z). Making up 36% of the global workforce, Gen Z is likely to be the most exposed to emotional AI. Moreover, AI algorithms are rarely calibrated for socio-cultural differences, making their implementation all the more concerning.

In a new study made available online on June 9 2022 and published in Volume 70 of Technology in Society on 22 June 2022, a team of researchers, including Prof. Peter Mantello and Prof. Nader Ghotbi of Ritsumeikan Asia Pacific University, Japan, strove to uncover the factors governing Gen Z's response towards emotional AI. "NCDC represents a new development in human-machine relations, and are far more invasive compared to previous AI technologies. In light of this, there is an urgent need to better understand their impact and acceptance among the Gen Z members," says Prof. Mantello.

The team surveyed 1,015 Gen Z respondents spanning 48 countries and eight regions worldwide. The participants were asked about their attitudes towards NCDC, used both by commercial and state actors. They then used a Bayesian multilevel analysis to control for variables and observe the effect of each variable at a time.

The team found that, overall, more than 50% of the respondents were concerned about the use of NCDC. However, the attitude varied based on gender, income, education level, and religion.

"We found that being male and having high income were both correlated with having positive attitudes towards accepting NCDC. In addition, business majors were more likely to be more tolerant towards NCDC," says Prof. Ghotbi. Cultural factors, such as region and religion, were also found to have an impact, with people from Southeast Asia, Muslims, and Christians reporting concern over NCDC.

"Our study clearly demonstrates that sociocultural factors deeply impact the acceptance of new technology. This means that theories based on the traditional technology acceptance model by Davis, which does not account for these factors, need to be modified," explains Prof. Mantello.

The study addressed this issue by proposing a "mind-sponge" model-based approach that accounts for socio-cultural factors in assessing the acceptance of AI technology. Additionally, it also suggested a thorough understanding of the potential risks of the technology to enable effective governance and ethical design. "Public outreach initiatives are needed to sensitize the population about the ethical implications of NCDC. These initiatives need to consider the demographic and cultural differences to be successful," says Dr. Nguyen.

Overall, the study highlights the extent to which emotional AI and NCDC technologies are already deployed and the privacy trade-offs they imply for the younger generation. Thus, there is an urgent need to make sure that these technologies serve both individuals and societies well.