Empathetic Robots Built to Detect Human Emotions: HKUST Professor on Human-Machine Interactions Elected as ISCA Fellow

What would happen to our lives if robots could read our emotions and act like a friend? It may sound like some plots in science fiction movies, but Prof Pascale Fung from the Department of Electronic and Computer Engineering at the Hong Kong University of Science and Technology (HKUST) has realized this dream. An expert in intelligent systems that understand and empathize with humans, Prof Fung’s latest project “Zara the Supergirl” is a prototype machine capable of chatting with humans in both English and Mandarin, it is expected to chat in more languages including French in the future.

Zara determines users’ gender and ethnicity by their appearances before choosing the right medium of communications. As an early version of what intends to be a truly empathetic robot, Zara could already grasp a person’s character by asking questions about their childhood, or experiences with their bosses. Zara could also make comments in response to what the person has said, taking into account his tones and facial expressions. The robot has received positive feedback while on display at the World Economic Forum in September 2015.

Prof Fung became the first Hong Kong-based scholar elected as a fellow of the International Speech Communication Association (ISCA) – a top professional body in speech technology and science, for her “fundamental contributions to the interdisciplinary area of spoken language human-machine interactions”. Throughout the years, Prof Fung and her team invented the world’s first multilingual voice search engine, the first Mandarin voice assistant on smartphones, and the first Chinese natural language search engine. ISCA’s fellowship recognizes and honors those who made significant contributions to the science and technology of speech communication with a minimum of ten years of experience in the field.

Prof Fung was elected as the fellow of the Institute of Electrical and Electronic Engineers (IEEE) for her contributions to human-machine interactions in this year. She is one of the founding faculty members of the Human Language Technology Center at HKUST – the first such research center in China that has produced many of the industry and academic leaders in this area, Director of InterACT@HKUST- a joint research and education center with Carnegie Mellon University, University of Karlsruhe (TH) and Waseda University, and the founding chair of the Women Faculty Association at HKUST.