OHBOT SOCIAL ROBOTS EMOTION MODELING USING MARKOV CHAINS AND YOLOV5 NEURAL NETWORK

Social robotics is a strongly developing and multidisciplinary field that combines solutions from robotics, computer science, psychology and many other disciplines. Within its development, there is a particular focus on enhancing human-robot interaction, by constructing robots in such a way that they are perceived as more social. The aim of this paper is to implement a module that allows a robot's facial reaction to be matched to the current state presented by a person, and to predict and attribute a person's emotional style. The aim is to increase the reactivity of the social robot and to sensitise it to sudden changes in the emotions of the interlocutor. Neural networks and Markov chains were used to complete the task. The use of YOLOv5s networks made it possible to analyse and label both single emotions and their sequences based on the acquired image of the interacting person. The implementation of a second-order Markov chain enabled the prediction of sequences of emotional states, taking into account previously acquired data and user characteristics. The designed solution was implemented into a social robot of the OhBot type, taking into account all the constraints and requirements this brings (e.g. limited computing power). The theoretical proposal for matching the social robot's facial expressions to the speaker was practically implemented, and the effectiveness of the predictions carried out was tested, which reached 82-70.5%. The results obtained in this publication can be transferred to other social robots, as well as to other IoT solutions that require emotional modelling.