HKUST Develops Multiple Smart Devices to Advance Medical Innovation

A research team at the Hong Kong University of Science and Technology (HKUST) has developed three innovative smart medical devices for health monitoring, surgical assistance and rehabilitation. Integrating artificial intelligence (AI) with robotics technology, the devices seek to help doctors address challenges in treatment and diagnostics, improve medical procedures and enhance efficiency. Led by Associate Prof. SHEN Yajing from the Department of Electronic and Computer Engineering, the team is actively collaborating with public and private institutions, industry stakeholders and investors to promote the translation and clinical application of the three breakthroughs below.

AI Hand-Centric Tactile Interaction System (PhyTac):

With a dense network of nerves, human hands possess remarkable sensitivity and dexterity. When hand nerves, muscles or joints are compressed or affected by conditions such as stroke, patients may experience loss of function, numbness, or tingling. For stroke survivors in particular, hand movement impairments are common. Early detection and accurate assessment of severity of these conditions can significantly reduce their impact on patients’ daily lives.

However, the assessment of hand grip forces largely relies on doctors’ observation at present, judgements could vary according to the doctors’ experience. Prof. Shen’s team has therefore developed PhyTac, a cone-shaped AI hand-centric tactile interaction system fitted with a maximum of 368 sensing elements that could correspond to the exact points of force exertion in the hand. 

Simply by gripping the device, the force distribution across specific regions of the patient's hand will immediately be displayed on screen, allowing doctors to easily monitor the patients’ rehabilitation progress. Additionally, this AI system can integrate with virtual reality (VR) technology to create interactive games, enabling the customized design of rehabilitation exercises and the development of personalized recovery plans.

The special spiraling design of PhyTac was inspired by Aloe polyphylla – a plant with triangular leaves in a symmetrical, five-pointed spiral pattern that prevents overlapping to maximize sunlight absorption. With this innovative design, PhyTac can significantly increase sensor density while expanding its sensing range, thereby precisely reflecting the hand force distribution with an accuracy reaching 97.7%. The team is now seeking collaborations with institutions such as the Hospital Authority Community Rehabilitation Service Support Center, on translating this innovation for patients’ benefits.

A portable wireless spirometer device: 

The World Health Organization predicts that by 2030, chronic obstructive pulmonary disease (COPD) will become the third leading cause of death globally. Additionally, post-COVID-19 sequelae, commonly known as “long COVID,” have heightened the demand for regular lung monitoring in the post-pandemic era. To ease the burden of frequent hospital visits for spirometry tests, the team has developed a portable wireless spirometer, measuring just 8 cm in diameter and weighs approximately 78 grams—about half the weight of a typical smartphone. This device allows users to conveniently perform exhalation tests and perform breathing training at home.

Equipped with a "Biomimetic Biomechanical Airflow Sensor," the device enables patients to measure their Peak Expiratory Flow (PEF) with an accuracy of 99%.  The device incorporates a gamified feature that uses patients’ breath output to control a virtual bird navigating through obstacles.  It is hoped that by adding fun to the process, more chronic patients will be encouraged to conduct regular respiratory training at home. The data collected will be stored on the server, and the research team is also building a cloud-based platform to allow real time transmission of test results, so that doctors can monitor patients’ progress remotely. The team will seek to promote the mutual sharing of this test data with healthcare institutions to facilitate the product’s commercialization, allowing patients in need to monitor their health at home.

World’s Smallest Multifunctional Surgical Robot:

Micro-robotic systems are widely used in minimally invasive surgeries, but existing technologies struggle to combine endoscopic capabilities, precise navigation, and compact size. Addressing this, the team has developed the world’s smallest multifunctional surgical robot, with a diameter of just 0.95 mm - 60% smaller than comparable devices. This micro-robot integrates imaging and precise navigation functions, and can assist medical professionals in sampling tissues, delivering drugs, and performing laser thermal therapy within the human body. The robot has been tested in bronchial models and ex vivo pig lungs, and has demonstrated exceptional navigation capabilities in confined spaces, clear imaging, and the ability to perform multiple therapeutic functions in challenging areas. By optimizing the optical path of the endoscope and pursuing investment partnerships, the team aims to commence animal testing within six months in preparation for the subsequent phase of clinical trials.

Prof. Shen remarked, “In this era of AI and robotics, technology advances rapidly and there is ample room for the development of human-machine interaction, which shall become a trend for the future.. The development of this series of medical devices aims to combine the results deriving from the convergence of interdisciplinary knowledge in robotics technology, engineering, and biomedicine, with the goal of helping doctors overcome the bottlenecks encountered in the diagnosis and treatment process, thereby improving the efficiency of medical workflows. Leveraging HKUST’s strengths in robotics and AI research, our team will continue to enhance the perceptiveness of robots so as to enable safer and more natural interactions between robots and humans, paving the way for impactful research designs that support transformation across various industries, including healthcare.”

The tactile sensor series developed by Prof Shen’s team was selected as one of the first 24 projects under the Innovation and Technology Commission’s “Research, Academic and Industry Sectors One-plus Scheme” (RAISe+ Scheme) last year.  With the ability to accurately measure the contact forces on humanoid robotic fingertips in all directions, and mimick human touch, the sensor series has a broad application and can enable robots to better perceive and interpret their environments and human intentions.  The robots’ interaction with humans will be even safer and more natural if the series is combined with speech comprehension and visual processing. The team is now collaborating with research institutes, industrial automation production lines, healthcare centers, and robotics-related industries in Hong Kong to advance these innovations.