Winning AI Solution Helps Remotely Monitor Patients’ Vital Signs

PanopticAI developed an AI health monitoring app using the camera on a smartphone or tablet to monitor the user’s vital signs such as heart rate, respiration rate, blood pressure etc. A pilot scheme has been deployed to 10 to 20 elderly homes across the city. [Download Photo]

The upheaval wrought by the COVID-19 pandemic has changed the world and businesses have had to quickly pivot in response to the new normal. Hong Kong start-up PanopticAI was one, seeing a demand for new methods of remote diagnosis and self-diagnosis in a time of social distancing.

“A 30-second scan can get your vital signs before you see the doctor”

The company developed a fully remote wellness monitoring app with AI-enabled smart technology. Unlike existing wearable health-monitoring watches which use LED sensors, the app simply connects to the camera on a smartphone, tablet or computer to capture subtle changes of coloration in the skin due to changes in blood flow to obtain the user’s health readings.

“We can do a quick 30-second scan to get your vital signs before you see the doctor online,” says CEO Kyle WONG.

These include heart rate, respiration rate, blood pressure, oxygen saturation, and heart rate variability. From these, further readings, such as stress, energy levels, and health risk factors, can be triangulated with the visual scanning algorithms that the team developed.

Kyle, who is currently studying for a PhD in Bioengineering at HKUST, initially started the company as an offshoot of an existing research group at HKUST which was installing smart temperature monitors for border points and government offices since 2018. His three co-founders were all from HKUST – Nick CHIN, who has since graduated with a PhD in Industrial Engineering and Logistics Management; Teric CHAN, who holds an MPhil in Industrial Engineering and Decision Analytics; and also Prof. Richard SO of the School of Engineering, who was overseeing the research group.

A chance encountered with Prof. So at a careers fair in London, where Kyle was an undergraduate in biomedical engineering at King’s College, led him to apply for the Hong Kong PhD Fellowship Scheme (HKPFS).

“We found that our research interests lined up,” says Kyle. “My research focus in biomedical engineering has always been how to make use of the latest software, whether it’s AI, big data, cloud computing, or signal processing, to improve someone’s health and to improve a person’s life,” he says.

Though not yet commercialized, the response to the product has been positive to date, winning the university/tertiary section at the inaugural Hong Kong City I&T Grand Challenge last November due to the product’s marketability and the business model, all of which was underpinned by the team’s published scientific papers and patents. “Aside from the very generous award money, they are also providing us a gateway to deploy and commercialize our idea,” Kyle says.

That deployment comes in the form of a pilot scheme at 10 to 20 elderly homes across the city from April to June. The tracking app is connected to the elderly homes’ tablets or computer devices for the staff to obtain vital signs of their residents. A health profile for the residents can also be created using the data to better monitor their health and detect any early health signals that may lead to timely assessment and care.

“The health monitoring booth can be replaced with a tablet”

While the benefits of the technology in reducing contact during a pandemic are obvious, there are also other logistical advantages, such as saving much-needed space with a potential reduction of equipment.

“Our proposal is to simplify this whole health monitoring process to just a tablet, which will essentially replace the health monitoring booth,” Kyle says.

More immediately, the trial scheme will allow PanopticAI to get valuable feedback from users and troubleshoot any problems before the product goes to market. Initially, elderly residents will be monitored via the software under staff supervision, though Kyle sees future elderly users being more autonomous, in the process benefiting from greater independence.

“Now that more and more elderly are becoming tech-savvy, or at least know how to use a smartphone, and our technology can play a big role in healthy aging,” Kyle says.

But it is not just the elderly who stand to benefit – pregnant women and people with chronic illnesses can also monitor their health, and anyone who might not be able to see a doctor immediately.

Instrumental in PanopticAI’s journey to date has been the HKUST Entrepreneurship Center (EC), which gave the fledgling company lots of support, both financially and administratively, in its early days, particularly via the Lo Kwee Seong Tech-Ship Fund, which supports HKUST faculty, students and alumni who found start-ups using HKUST intellectual property.

“We had a lot of networking opportunities, whether it’s with investors or just strategic partnerships with different institutions,” Kyle says.

Among the institutions the EC introduced PanopticAI to were Hong Kong Science Park, where the company is now based, and the Hong Kong Trade Development Council. The team now focuses on delivering their proprietary technology to a wider local market while global expansion will be their ultimate goal.

“For now, we are looking to expand locally and collaborate with different industries. In the future, our hope is to make PanopticAI code available as a software development toolkit (SDK) to companies to integrate into their own systems, and that our ‘snapshot of health’ will encourage people to monitor their own health, for which we envisage a number of innovations, such as gamification, to make it a reality.”