Credit: Unsplash/CC0 Public Domain

Creating a machine learning agent to ease interactions between participants in virtual and augmented reality environments—such as the metaverse—in order to help them achieve their goals and address conflict situations is the main objective of GuestXR, the European project under the coordination of professor Mel Slater, from the Faculty of Phycology and the Institute of Neurosciences of the UB (UBNeuro).

As part of the GuestXR project (A Machine Learning Agent for Social Harmony in eXtended Reality), coordinated by the Eurecat technological center, a called guest will be designed, and it will work as an agent able to examine the individual and group behavior of participants by drawing on existing models from a neuroscience and social psychology standpoint. Moreover, will train the Guest to learn over time—both in simulations and real meetings—and it will be more efficient in social interactions.

Along these lines, the agent "will take part in the conversation in different ways, performing a range of multisensory actions through music or changes in space, among others," notes the director of Eurecat Audiovisual Technologies, Adan Garriga.

"Each meeting in virtual and augmented reality has a purpose, be it explicit (for instance, reaching an agreement on a topic) or implicit (such as enjoyment). The basic idea of the Guest is to help the group of people to accomplish this purpose. This involves exciting research with a strong interdisciplinary element, which opens new paths, and hopefully, contributes to the success of virtual and augmented reality," notes Professor Mel Slater.

In order to modulate this social interaction, the Guest will carry out a series of multisensory actions "through, for instance, visual and auditory features to create specific states of mind and stimulate a relaxed environment when it identifies a conflict," says researcher Umut Saying, from Eurecat Audiovisual Technologies.

This project "is designed to help offer solutions to the existing challenges in online environments of such as online harassment, for which there isn't a specific European legislation," says the coordinator of the Eurecat GuestXR project, Aurora Sesé.

The intervention of the machine learning system will initially be tested in conflict resolution situations, interactions with participants with hearing difficulties and contexts that lead to polarized debates, such as climate change.

Two open calls to include other innovative use cases to test the effectiveness of the system are expected. One case will be aimed at society in general—including companies and associations, among others—while the other will aimed at the field of arts.

The GuestXR technology will be carried out under the ethics by design approach, which entails considering the potential ethical problems derived from the use of artificial intelligence (respect for human autonomy, prevention of harm, fairness, etc.).

The GuestXR consortium, coordinated by the UB and the Eurecat technology center, is formed by eight organizations from six countries with multidisciplinary teams of the field of the expanded reality, , artificial intelligence, social psychology, neuroscience of emotions, , ethics of research and technology transfer.