Allowing social robots to learn relations between users' routines and their mood

Social robots, robots that can interact with humans and assist them in their daily lives, are gradually being introduced in numerous real-world settings. These robots could be particularly valuable for helping older adults to complete everyday tasks more autonomously, thus potentially enhancing their independence and well-being.

Researchers at University of Bari have been investigating the potential using social robots for ambient assisted living applications for numerous years. Their most recent paper, published in UMAP'22 Adjunct: Adjunct Proceedings of the 30th ACM Conference on User Modeling, Adaptation and Personalization, specifically explores the value of allowing social robots who are assisting seniors to learn the relationships between a user's routines and his/her mood.

"Social robots should support older adults with daily activity and, at the same time, they should contribute to emotional wellness by considering affective factors in everyday situations," Berardina De Carolis, Stefano Ferilli and Nicola Macciarulo wrote in their paper. "The main goal of this research is to investigate whether it is possible to learn relations between the user's affective state state and daily routines, made by activities, with the aid of a social robot, Pepper in this case."

The "mood-personalized" assisted living approach devised by De Carolis and her colleagues is based on a system called WoMan (Workflor Management). This system, introduced in one of their previous papers, can continuously learn and refine the routines of users, using first-order logic reasoning.

In their recent study, the team implemented the WoMan system using the Pepper robot. Pepper is a semi-humanoid mobile robot manufactured by SoftBank Robotics, which can interact with humans and detect their basic emotions.

"WoMan will be used as a back-end module of the Daily Diary application running on the Pepper robot to collect data concerning daily activities and their relation to emotions and mood," De Carolis, Ferilli and Macciarulo wrote in their paper.

So far, the team outlined their proposed approach and investigated whether it can effectively learn the relationships between a user's daily routines and his/her mood. In addition, they created a proof of concept of their approach, designing a dialog application for the Pepper robot called the DailyDiary (DD).

As its name suggests, the DD application can be used to collect annotations summarizing a user's daily routine, activities and moods. The data collected by this application is then analyzed by a machine learning model, to better understand how users are feeling when completing different tasks.

"Results of this phase of the research will be used to assess the validity of the approach in ambient assisted living houses for seniors to make the social robot able to provide not only proactive service assistance but also an affective empathic experience," the researchers added in their paper.

In their initial evaluations, De Carolis and her colleagues found that their learning model could learn to predict daily routines and the relations between activities and a user's mood with excellent accuracy. In their next studies, they plan to develop a complete version of their application and test it in a real-world setting.

These future tests will re-assess their learning model's accuracy in predicting users' activities and moods in a real-world setting, In addition, they will try to determine how senior users experience the DD application and the Pepper robot running it, confirming whether the team's approach would be beneficial and well-accepted by seniors.

If the researchers' future evaluations go well, their work could aid the development of more "seemingly empathic" robots, which can adapt their behavior to meet both the physical and emotional needs of vulnerable users.