Real-Time Human Interaction Intent Detection for Resource-Constrained Social Robots Using RGB-Only Behavioral and Emotional Cues

Service robots with expressive heads are increasingly being deployed in public spaces, creating a need for the real-time detection of human intent to interact, in order to provide responsive services back to the interacting human users. In this work, we present a practical and modular deep learning-based system for detecting human interaction intent, which runs in real time on compute- and resource-limited robot platforms without requiring high-cost, bulky, and power-hungry GPUs. Our approach combines human pose estimation and facial emotion analysis for both frame-level and sequence-level detection of interaction-intent behaviors. Crucially, we demonstrate robust model generalization across different cameras and environments by implementing our system in hardware (Raspberry Pi 5) and using it in the field. Remarkably, we show that even though our dataset has been collected in a controlled setting using a standard USB webcam, our trained model generalizes successfully without finetuning on a robot head platform which makes use of a different camera sensor and deployed in different uncontrolled environments not captured by the training data. In contrast to most prior works that report model precision in offline settings only, our real-world system deployment and assessment in the wild reports a high balanced detection accuracy of 91%, clearly demonstrating the usefulness of our approach in practice.