HKUST Engineering Develops Embodied Robotic Arm “EmArm” with Human-Like Touch Sensing
A research team at The Hong Kong University of Science and Technology (HKUST) has developed an Embodied Sensorimotor Robotic Arm (EmArm) capable of whole-arm tactile sensing and adaptive manipulation. By embedding a soft optical tactile skin directly into a robotic skeleton and integrating it with a real-time decision-making algorithm, EmArm can perceive and analyze contact information across its upper and lower arms, enabling human-like tactile perception and adaptive object manipulation. The system can also distinguish between different touch interactions, such as tickling and patting, respond in real time, and avoid obstacles through tactile feedback, opening new possibilities for the development of embodied intelligent robots.
The study was led by Prof. SHEN Yajing, Professor of the Department of Electronic and Computer Engineering (ECE) and Associate Director of Cheng Kar-Shun Robotics Institute, and his team at the Mevita Robotics Lab. Other authors of the paper include postdoctoral researchers Dr. TANG Yifeng (first author), Dr. LI Gen, and Dr. REN Hao; doctoral students YIN Tianci, LING Zhengrong, ZHAO Haoxiang, and ZHANG Ruijia; and lab graduates Prof. ZHANG Tieshan (Shenzhen University) and Prof. YANG Liu (Nankai University, Tianjin). The findings were published in Nature Sensors under the title “Embodied sensorimotor integration for whole-arm tactile sensing and adaptive robotic manipulation”.
Tactile sensing reaches 97.9% fidelity while preserving high-precision actuation
Inspired by the human body’s “bone-muscle-skin” structure, the team deeply embedded a large-area soft optical tactile skin inside EmArm’s rigid mechanical skeleton, integrating high-load operation with soft tactile sensing. More than 71,000 square millimeters of soft tactile skin covers the upper and lower arms, while internal sub-millimeter-scale cameras analyze the distribution of forces applied to the robotic arm. EmArm achieves a spatial resolution of 0.085 millimeters, comparable to that of human fingertip skin. Its two-point discrimination threshold can be as low as 18.2 millimeters, outperforming the typical 30-to-50-millimeter range of human arm skin and enabling finer differentiation of contact spacing. The optical sensing approach also avoids the complex wiring required by electronic skin and is inherently resistant to electromagnetic, light, temperature, humidity, and acoustic interference. Experiments showed that EmArm maintained a sensing fidelity of up to 97.9% under five types of environmental interference.
More importantly, the embedded tactile skin had no material impact on the robotic arm’s high-load operation. Experiments showed that EmArm achieved repeat positioning accuracy comparable to that of conventional robotic arms. The system could also withstand pressure loads exceeding 15 kg and torsional loads of 7.35 Nm. EmArm therefore integrates whole-arm tactile sensing with high-precision actuation in a unified embodied sensorimotor architecture.
Perception-action closed-loop framework enables five emotional feedback expressions and advances embodied intelligence
Prof. Shen, corresponding author of the study, noted that “For robots to acquire perceptual capabilities, beyond obtaining accurate raw signals from sensors, a more important aspect is how to rapidly decode these signals so the robot ‘knows’ what this tactile information means and ‘how to act’. We are dealing with a high-dimensional, continuously time-varying force field data stream. If we directly use end-to-end learning, it will not only require massive annotated data but also demand high computational power, making it difficult to run in real time on a robot.”
To transform EmArm from a sensor to an intelligent agent, the team further proposed a perception-action loop. The framework combines joint-angle data (proprioception) with tactile information (exteroception) in real time, and extracts features including contact location, area, force, and the number of contact points to support autonomous decision-making. Subsequently, the sensing data are fed directly into the robotic arm’s motion-control system. For example, when EmArm holds a basketball positioned away from its center of gravity, the system compares real-time tactile features with a pre-learned policy, generates joint-angle commands in real time, and continuously adjusts the forces applied by the upper and lower arms to maintain balance.
The research team demonstrated EmArm’s application potential across three typical scenarios. In human-robot interaction scenarios, EmArm can identify five touch behaviors, namely stroking, pinching, poking, tickling, and hitting. It interprets the interactions in real time and provides emotional feedback expressions corresponding to happiness, sadness, curiosity, shyness, or anger, achieving a recognition accuracy of up to 99.8%. In manipulation task scenarios, when a basketball is placed off-center on the arm, EmArm autonomously adjusts its posture through whole-arm tactile feedback and force control to stabilize the object. In a more challenging trajectory replanning experiment, when visually undetectable dynamic obstacles, such as a human hand, enter the motion path, EmArm can immediately detect them through touch, execute an emergency stop, determine whether the obstacle was temporary or permanent, and autonomously replan its path to bypass it. The results demonstrate that EmArm can maintain safe and flexible autonomous operation even when vision is occluded.
EmArm represents a significant technological advance in sensor technology and marks a broader shift in robotic design, from “externally attached perception” to “intrinsic intelligence”, and from “vision supremacy” to “multimodal fusion”. The research team notes that the EmArm system could be further extended to robotic legs, torsos, and other body parts, enabling safer and more adaptive interaction by humanoid robots in home services, industrial settings, medical assistance, and other application scenarios. The system could also help advance embodied AI, in which AI systems with physical bodies perceive, interact with, and learn from the real world.
This project was supported by the National Science and Technology Major Project “Brain Science and Brain-like Intelligence Technology”, Guangdong Basic and Applied Basic Research Foundation, Hong Kong Innovation and Technology Commission, and Technology Start-up Support Scheme for Universities (TSSSU).
(This news was originally published by the HKUST Public Affairs and Global Communications Office here.)
