HKUST ENGINEERING SPRING 2023 ISSUE 35 Stepping Into the Metaverse
CONTENTS SPRING 2023 I NO.35 HKUST Engineering In Focus Magazine sengpr@ust.hk (852) 2358 5917 https://seng.hkust.edu.hk/node/2914 Published by: Communications and External Affairs School of Engineering The Hong Kong University of Science and Technology Editors: Diana Liu, Dorothy Yip Contributing Editor: Sally Course Designer: Kingsley Lau hkustengineering HKUST-SENG seng.hkust.edu.hk Net-zero carbon development Blueprint to assist major developer decarbonize 5 7Stepping up vaccine and drug e ectiveness Researchers engineer novel way to synthesize mRNA 10 Preventing glaucoma Students’ award-winning goggles set out to save people’s sight 12 Getting real about the metaverse Tackling the technical and societal issues of extended reality through MetaHKUST 20 Joy of innovation From hydrogels to spider silk in a faculty career lled with discovering the unknown 17 Fun pathways to full-potential engineers Motivating students’ acquisition of knowledge 26 Life as an ‘ingénieur’ Two alumni embrace wider horizons on a French government master’s degree scholarship program
At the forefront of change elcome to the latest issue of In Focus and my rst column as Dean of the HKUST School of Engineering (SENG). As commences, we move forward into a new and exciting year where the COVID- pandemic may nally recede from centerstage and fresh horizons beckon. As a specialist in smart transportation, I see this as a propitious sign for the start of my own new journey as I take up the great honor and responsibilities of the deanship. Since joining the then Department of Civil and Structural Engineering (now Civil and Environmental Engineering) in the mid- s, not long a er HKUST was established in , I have been fortunate to witness and contribute to SENG’s rapid rise to global prominence through diverse roles in education, research, and administration at the University and my active participation in the academic world and wider community beyond the campus. High global rankings are one indication of SENG’s leading achievements. However, I believe the number one reason for the School’s distinctive character is our community of hardworking, high-achieving, self-motivated faculty, students, sta , and alumni. It is their passion to work at the highest levels of engineering education, research, and the profession, along with a growing drive for knowledge transfer, entrepreneurship, and social impact, that creates such an inspiring environment across the many elds the School embraces. Established and emerging research areas cover an extensive range: advanced materials; aging & healthcare; AI; autonomous systems & robotics; communications; data science; design thinking & entrepreneurship; energy & sustainability; microelectronics; and smart cities. On the education front, the School has helped advance massive open online courses (MOOCs) and continuously explores novel engineering pedagogies. As Dean, I will be seeking to extend the School’s reach by enhancing our capabilities and platforms to tackle large-scale and complex interdisciplinary projects. In particular, we will strive to address the grand challenges and sustainable development goals that must be solved worldwide to improve the lives of the generations to come – energy, climate change, health and well-being, industry, innovation, and infrastructure, to name but a few. In this quest, I see the complementary strength of colleagues at HKUST’s cross-disciplinary-oriented Guangzhou campus, which opened in September , and collaborative ventures with national and international partners as being highly productive to scaling up our capabilities. Further priorities are to proactively recruit top faculty and students from around the world to sustain and advance SENG’s dynamic environment; strengthen the School’s connections with industry to build mutual understanding of research and technology transfer needs and challenges; and deepen engagement with alumni. Engineers create and deliver the technologies that shape the world. At SENG, we strive to be at the forefront of such forward moves. With the Hong Kong government’s focus on transforming the city into a global innovation and technology hub, and the Greater Bay Area development plan bringing a host of opportunities regionally, it is certainly a prime time to connect up and be part of the SENG community. Prof. Hong K. LO Dean of Engineering 03 IN FOCUS Dean’s View W
HKUST pacesetters take o ce t has been a time of change at the top of HKUST over recent months, bringing two outstanding University scholar-administrators and one new face – all with links to engineering – to three key leadership positions. In October , globally renowned molecular neuroscientist Prof. Nancy IP took the helm as HKUST’s h President, becoming the rst woman to head one of Hong Kong’s University Grants Committee-supported universities. Prof. Ip, who has spent years at HKUST, has played an integral role in instituting the excellence that has powered the University’s rise to a world-leading institution just three decades a er its establishment in . She is an elected member of the Chinese Academy of Sciences and an international member of the US National Academy of Sciences, among many other eminent recognitions and awards. Born and raised in Hong Kong, Prof. Ip received her PhD in Pharmacology from Harvard University under the tutelage of Prof. Richard ZIGMOND, and later developed a successful career in industry, before joining HKUST in . A highly accomplished researcher, she has made seminal contributions toward the development of diagnostic and prognostic tools for Alzheimer’s disease. She also actively collaborates with renowned researchers from around the world and is passionate about nurturing local talent. Her cross-disciplinary collaborations have led to important breakthroughs, such as the development of imaging technology with faculty at the School of Engineering (SENG), which has shed light on the functions of brain regions that were previously hard to explore. At the start of December, distinguished computer scientist Prof. GUO Yike commenced his appointment as Provost of HKUST. Prof. Guo, most recently Vice-President (Research and Development) and Dean of the Graduate School at Hong Kong Baptist University, had previously spent years at Imperial College London, including receiving his PhD there. Prof. Guo, a Tsinghua University alumnus, is an expert in data mining for large-scale scienti c applications, covering distributed data mining methods, machine learning, and informatics systems in areas ranging from biology to creative design, geophysics to healthcare, nance to social media. In addition to his responsibilities as Provost, Prof. Guo is concurrently a Chair Professor in the Department of Computer Science and Engineering. He is also the principal investigator of a HK$ .8 million Theme-based Research Scheme project, focused on AI-based art tech for human-machine symbiotic creation and funded by Hong Kong’s Research Grants Council. In January , Prof. Hong K. LO took o ce as Dean of Engineering. Prof. Lo, Chair Professor in the Department of Civil and Environmental Engineering, is a smart transportation specialist and Founder-Director of HKUST’s interdisciplinary GREAT Smart Cities Institute. The Hong Kong-born academic received his PhD in Civil Engineering from Ohio State University, moving on to Oak Ridge National Laboratory and University of California, Berkeley, before joining SENG in the mid- s. Prof. Lo has established himself as a driving force in the interdisciplinary transportation eld, steering forward intelligent transportation systems, mobility system resilience, sustainable development, and smart cities. President Prof. Nancy Ip Dean of Engineering Prof. Hong K. Lo Provost Prof. Guo Yike I 04 IN FOCUS In the News
Developing the way to net-zero carbon n a signi cant step forward for the developer sector and climate change mitigation, Ir. Prof. Irene LO, Chair Professor of Civil and Environmental Engineering, and her research team have collaborated with a major industry player to devise a practical holistic roadmap for the company to achieve net-zero carbon by . Working with Sino Land Company Ltd., one of Hong Kong’s leading property developers, Prof. Lo has assisted in drawing up a Decarbonization Blueprint and providing science-based targets. The blueprint encompasses interim goals for decarbonization, including provisions for climate risk assessment for projects, to further enhance the company’s risk management system. The roadmap is one of the rst to be publicly released by a major developer. Prof. Lo, an award-winning researcher and educator, has concentrated on originating environmental clean-up research solutions locally and internationally since joining HKUST in . She is an Academician (Technical and Environmental Sciences) of the European Academy of Sciences and Arts, becoming Hong Kong’s rst member of the prestigious body in . She is also a registered professional engineer and registered carbon auditor in Hong Kong and the US. In , Sino joined the Business Ambition for . ⁰C campaign, initiated by a global coalition of United Nations agencies, business and industry leaders to advance e orts to decarbonize through aligning corporate plans with climate science. Sino then teamed up with Prof. Lo and her researchers. Following research and data collection, they went on to together devise decarbonization strategies and science-based interim targets covering the company’s key areas of development, operations, and collaboration. These include setting percentage goals for reductions in greenhouse gas emissions and electricity use intensity as well as carrying out climate risk assessments and obtaining BEAM Plus Gold certi cation or above at Sino Land’s wholly owned new development projects, where applicable. BEAM Plus is a top initiative in Hong Kong providing independent assessment of building sustainability performance. The collaboration has provided a valuable opportunity to transform the team’s research on decarbonization and carbon reduction into impactful solutions for the building industry and wider community, according to Prof. Lo. “Together, we are making important strides to drive sustainability and tackle the grand challenge of climate change,” she said. Mr. David NG, Group Associate Director of Sino Group, also commended the “stellar support and e ort” from the HKUST team and Sino colleagues and stakeholders in propelling forward the net-zero carbon roadmap “to make progressive strides toward a low-carbon, climate resilient and sustainable future”. Prof. Lo and her team are now working on further net-zero projects with Sino related to the company’s new Landmark South o ce and retail building in Wong Chuk Hang and a construction project on Peel Street in Central. I Advancing corporate efforts to decarbonize through aligning company plans with climate science: Chair Professor Irene Lo, Civil and Environmental Engineering, and Mr. David Ng, Group Associate Director of Sino Group. 05 IN FOCUS
06 IN FOCUS In the News The “AI checker”, based on computer vision, helps to ensure all guidewires (circled, right) have been removed from a patient’s body. AI technology boosts patient safety An AI-based guidewire recognition and counting technology, developed by Prof. Gary CHAN Shueng-Han, Computer Science and Engineering, and his research team, is helping to improve patient safety by reducing the risk of wires being le in the body following a clinical procedure. The AI so ware, which rapidly and accurately recognizes all used guidewires removed from a patient, has been deployed at Tseung Kwan O Hospital in cases from November up to February , without missing a wire. Prior to the technology, medical workers had to manually count multiple times whether all guidewires had been taken out. A guidewire is a thin wire used to guide placement of a catheter into a central vein to administer fluids or medication, or monitor blood flow stability. It is used in procedures employing the Seldinger technique, where a central venous catheter is inserted to give access to blood vessels and hollow organs such as the stomach and gallbladder. The technology developed by Prof. Chan’s team provides an “AI checker” using computer vision. Medical sta rst con rm the number of guidewires as indicated by the system. They take and upload a photo of the guidewire(s), and perhaps other medical instruments, via a smartphone or tablet. Through object recognition and data augmentation techniques, the system can then detect guidewires and cross-check the manual count in real time. “Compared to humans, our AI checker does not get tired or distracted and serves as an impartial tool to double check and validate manual counting,” Prof. Chan said, noting the system maintains its high accuracy and e ciency over time. “Looking ahead, we would like to extend the technology from guidewires to other medical instruments to guard against retention inside a patient’s body.” Prof. Chan is also working on location sensing applications to e ciently search for missing dementia patients or other mentally incapacitated persons (MIPs). Turning innovation into industry impact Five HKUST advanced technologies are to be developed for application in industry following the signing of a licensing agreement with Guangdong Bright Dream Robotics Limited (BDR), which specializes in automation machinery R&D and manufacturing. The technologies mark the rst outputs to emerge from the HKUST-BDR Joint Research Institute (HBJRI), a University-industry platform set up to promote innovation and nurture talents. Established in , the joint institute is working on around projects related to robotics, AI, smart cities, materials, and big data. Most stem from real-life challenges facing industry, with faculty proposing solutions and Bright Dream Robotics testing the ideas. The innovations licensed comprise: A new inorganic coating that could help keep buildings cool in hot and humid climates by lowering the temperature by °C during day time and 6°C at night. A parallel compression framework containing three di erent compression algorithms to t di erent environments and point cloud data. GPU-accelerated algorithms can be used in large-scale D map compression or in real-time autonomous driving systems. Wear-resistant, anti-corrosion, self-cleaning, superhydrophobic surfaces for building robots’ metal components. Design and forming technology of large composites for building formwork and light-weight robots.
07 IN FOCUS Prof. Becki Kuang (center) and team members Li Cheuk-Yin (right) and Liang Zhenghua, both PhD candidates, showcase their mRNA tail sequence optimization technology. An inorganic coating to keep buildings cool is among the ve recently licensed technologies. Spurring mRNA drug and vaccine e ectiveness A synthetic biology research team led by Prof. Becki KUANG Yi, Chemical and Biological Engineering, has discovered a way to enhance the lifespan and increase synthetic mRNAs’ protein production e ciency by up to times. The exciting advance can augment the e ectiveness of mRNA vaccines and drugs, including those battling cancer, COVID- , and genetic diseases, and at the same time use smaller dosages. Synthesized mRNAs can teach cells to make proteins such as antigens and hormones that are essential to ght infections and regulate bodily functions. This means that mRNAs are o en the preferred option for vaccines and treatment for many diseases. However, the need for high dosages and repeated injections to generate enough proteins in the body has made increasing mRNA e ectiveness a key issue among researchers. Having engineered di erent mRNA tail sequences, Prof. Kuang’s team identi ed optimal sequences that could produce three to times as many proteins as unoptimized tail sequences commonly used for synthetic mRNAs for both human cells and on mice. The duration of protein production also doubled. In addition to reducing the amount and number of injections needed, the new technology could lower the cost of treatments. It can also be used along with other mRNA enhancement technologies to synergistically boost protein production. The nding was recently published online in Molecular Therapy – Nucleic Acids. Prof. Kuang’s team is now working in collaboration with Sun Yat-sen University in Guangzhou to explore optimized tails for mRNA cancer vaccines on animals. She also hopes to collaborate with pharmaceutical companies to transfer the innovation from lab to market. Integration of Building Information Modeling (BIM) data and a mobile map engine, enabling users to accurately and rapidly pinpoint people or facilities inside a building through their mobile phones. Bright Dream Robotics President Mr. WANG Kecheng said that the partnership with HKUST had built a bridge between academia and industry, enabling research and knowledge transfer. Meanwhile, Prof. Tim CHENG, Vice-President for Research and Development at HKUST and former Dean of Engineering, said that the signing of the licensing agreement exempli ed HKUST’s ability to create impact on society. “We will continue to forge and deepen our collaboration with industry to strengthen the University’s endeavors in knowledge transfer,” he added. (See also HKUST Industry Engagement Day, back cover.)
08 IN FOCUS In the News Rechargeable battery breakthrough Hearing aids, flashlights, remote control devices, and other household items could become more sustainable in the future, thanks to a novel electrode design by School of Engineering researchers that enables the alkaline zinc batteries these devices use to become rechargeable. The team, led by Prof. CHEN Qing, Mechanical & Aerospace Engineering and Chemistry, has developed a nanoporous zinc metal electrode capable of stabilizing the electrochemical transition between zinc and zinc oxide, successfully turning an alkaline zinc-air coin cell into a rechargeable battery stable for over hours. To do this, the team shaped zinc into curvy laments hundreds of nanometers wide, nested in a freestanding solid with numerous, similarly narrow pores. When the battery is discharged, a thin layer of zinc oxide nucleates on the zinc laments, preserving the metallic network and enabling the zinc electrode to return to its initial structure. In addition, the team has tested the new electrode in alkaline nickel-zinc batteries. Results showed an increase from the normal lifespan of to discharges and charges to more than , under conditions competitive with state-of-the-art lithium-ion batteries. Prof. Chen pointed out that alkaline zinc batteries had an edge over other batteries due to their safety, low cost and energy density. In industry, they are ideal for golf carts and forkli trucks, among others. They are also suitable for emerging applications, for example, back-up power for data centers, which do not need multiple discharging and charging but require the battery to be extremely safe. The research has been published in Nature Communications. Prof. Chen’s group has also been working with industrial partners since the research started in to assist development and commercialization of the battery technology. Key step forward for renewable energy An international research team led by Prof. Francesco CIUCCI, Mechanical and Aerospace Engineering, has designed an iron-based cathode material that achieves record performance for protonic ceramic fuel cells (PCFCs), marking a signi cant step forward in the commercialization of this renewable energy technology. PCFCs are generally used for distributed power generation and have the advantages of low pollutant emissions, high e ciency, and the flexibility of working well with hydrogen and other gases, including ammonia and biogas. However, a lack of high-performance and low-cost cathode materials have hindered development to date. By combining rst-principle simulations, molecular orbital analysis, and experiments, Prof. Ciucci’s team has now designed ceramics using inexpensive elements such as barium, iron, and zirconium, leading to a PCFC with record performance. The research has been published in Nature Catalysis and highlighted in Nature Reviews Materials. Team members include collaborators from Mainland China, South Korea, and Australia. A 3D model of the zinc electrode’s nanoporous structure, magni ed 10,000 times. Prof. Francesco Ciucci’s iron-based cathode material has enabled protonic ceramic fuel cells to attain record performance.
09 IN FOCUS Inspiring human-centered creativity A partnership between the School of Engineering’s Division of Integrative Systems and Design (ISD) and the business community is successfully fostering students’ empathetic design, systems thinking, and human-centered products through an enterprising scholarship award scheme. The Chinachem PrimeMovership scholarships support ISD undergraduates and postgraduates with high achievements in innovative technology design and entrepreneurial potential. One awardee is postgraduate Iain LAM, who has founded Sallux Education, a center that uses technology to help academically challenged students in primary and secondary schools. An endeavor born out of Iain’s own struggles with dyslexia and attention de cit hyperactivity disorder, his creative approach has improved the learning experience of such students by gamifying the process. With the help of headsets, students can become fully engaged in an immersive virtual learning environment, enabling them to focus on lectures and complete tasks in a non-distracting environment. Scholarships also went to an undergraduate team for a project centered on a social robotic pet, which o ers wellness support to isolated students. The palm-sized digital pet provides interaction and companionship to users with sound and visual feedback, and an emotional assessment to raise users’ awareness of their mental health. Prof. Winnie LEUNG, ISD, said: “Our ultimate goal is to nurture a new generation of innovators who can work across multiple disciplines and create disruptive innovations to solve the world’s great challenges.” Robots build engineering interest In a boost for science and technology know-how and enthusiasm among young learners, members of the School of Engineering helped the University organize the First Robot Explorer Cup competition in summer , attracting over students from primary and secondary schools and their teachers. The contest formed part of a hands-on HKUST science, technology, engineering, and mathematics (STEM) program, launched in December , whereby school students could learn about programming and D printing, and build their own robots through online workshops and a bootcamp. The winning team came from TWGHs Yau Tze Tin Memorial College, with Pui Tak Canossian College as rst runner-up. Lok Sin Tong Lau Tak Primary School was named second runner-up, and also received the Best Team Spirit Award. St. Margaret’s Co-educational English Secondary and Primary School won the Best Engineering Award. The competition was funded by Bank of China (Hong Kong). Immersive learning for academically challenged students, and (right) a social robotic pet. Young learners discover how robots work by putting together their own.
Students for their innovative O_Oley goggles. The O_O symbolizes the novel eyewear. The high-pro le international contest aims to inspire the next generation of design engineers in line with the creative outlook of competition founder and renowned industrial designer James Dyson. The team’s achievement has injected £ , (around HK$ , ) into the students’ research project, enabling them to patent the design and launch a start-up to take the invention forward. The project is led by Prof. David LAM and supervised by Dr. Stanley LEUNG (both Mechanical and Aerospace Engineering). Damage caused by glaucoma is a result of sustained build-up of internal eye pressure, known as intraocular pressure (IOP). While it is known that massage and hot towels relieve tired eyes, such applications are insu cient for relaxing intraocular tissues. The comfortable, curved-shell O_Oley goggles take this forward by allowing users’ eyes to undergo contactless thermal stretching. A shared goal to save people’s sight has led a team of mechanical engineering students to invent an award-winning device to help prevent the progression of a devastating eye condition Goggles seek to prevent glaucoma laucoma causes progressive loss of vision over time and is the second-leading cause of blindness worldwide. It is estimated that 8 million people globally su er from glaucoma, known as the “silent thief of sight” due to its capability to cause irreparable damage to the optic nerve before symptoms are detected. Yet preventive therapeutics are still in short supply. Now, four students from the Department of Mechanical and Aerospace Engineering have set out to change this through their creative invention: an easy-to-use wearable device suitable for those diagnosed with mild and pre-glaucoma, and with the potential to expand to all people who care about the health of their eyes. In September , the team comprising mechanical engineering students KWOK Kin-Nam, PhD Year (pictured, second le ), Minji SEO, PhD Year ( rst right), CHAN Kwun-Chung, BEng Year (second right), and LEUNG Yuen-Yin, BEng Year ( rst le ), took home the Hong Kong James Dyson Award G 10 IN FOCUS
The invention comprises a corneal tissue compliance improvement (CTCI) system and an ocular cell rejuvenation (OCR) system. The CTCI system regulates the goggle chamber pressure and increases the chamber temperature to massage the corneal tissue in a contactless way. The OCR system utilizes activation light at speci c wavelengths and infrared irradiation to energize intra-cellular activities as well as promote blood circulation within the ocular region. “O_Oley is like ‘hot yoga’ for your eyes” “O_Oley is like ‘hot yoga’ for your eyes,” Kin-Nam said. “It lowers your eye pressure by stretching the ocular surface and allows it to relax under a controlled, heated environment.” Given that the structure of the eyes is similar to a balloon, as the surface gets sti er and tenser, air pressure builds up from within, he explained. O_Oley is designed to reduce that stress, halting the progression of glaucoma. In a departure from other glaucoma treatment for diagnosed patients, O_Oley is non-invasive, providing a warming therapeutic experience that can be carried out daily at home. Bene ciaries include patients and people with a higher number of glaucoma risk factors, such as the elderly, individuals with high eye pressure, high myopia or hyperopia, and those with a family history of glaucoma, with the potential to expand coverage to all people su ering from eye strain associated with digital and urban lifestyles. In an unexpected further bene t, O_Oley has proved e ective in relieving dry eye symptoms. As with many innovations, the creative journey has not always been straightforward. One particularly discouraging stage occurred near the beginning when trial participants’ IOP remained constant despite all the team’s hard work. “At that time, we inched ahead as months went by. Frankly, we once thought about giving up when developing the prototype,” Kin-Nam said. “The Dyson Award came as timely assurance. It was such an honor to see our innovation win buy-ins and recognition. We bounced back and gained renewed con dence to turn things around again.” Team spirit was another major factor driving the students on. A well-mixed team of male and female, local and international, postgraduates and undergraduates, the four members proactively combined their strengths to build collective knowledge, challenged one another to think from di erent perspectives, and sought to integrate each person’s insights and contribution to create the best possible solution. Moreover, from the start, they were of one mind as to what they wanted to achieve. “Who knows… one day, we might be the bene ciary of our own invention” “Even if O_Oley hadn’t generated positive trial results, we believe we would have taken an alternative route and developed another device to achieve the same purpose,” Kin-Nam said. “Because right from the beginning, our goal has not been to commercialize a product but to create something that serves glaucoma patients. Every step we take is oriented to that goal. Who knows… one day, we might be the bene ciary of our own invention.” What’s in a word School of Engineering students also won the runner-up accolade in the Hong Kong James Dyson Award. The team’s PreDyctor is the world’s first dysgraphia identifier for Chinese handwriting. The advance o ers a rapid, inexpensive pre-screening system that can assess Chinese characters and estimate the chance that the writer has dysgraphia, to assist early intervention. Dysgraphia is a learning disorder associated with impaired writing ability. The research team, all from the Class of , comprised NIE Fei, BEng in Computer Science (pictured, bottom le ), ZHOU Xinrui, Dual Degree Program in Technology and Management (bottom right), ZHAO Yankun, BEng in Computer Science (top right), and ZHAO Yizhe, BSc in Computer Science (top le ). 11 IN FOCUS
magine taking a course in a lecture theater lled with avatars, including the speaker’s and your own. Imagine a class where the professor greets the students present, in person and apparently there but actually located at a campus around km away. These scenarios may seem futuristic. However, such total virtuality and mixed reality environments for education are almost here, with initial trials underway as part of MetaHKUST, the University’s go-ahead extended reality initiative. I Getting real about the metaverse HKUST is setting out to help shape the digital world of immersive learning that frontier augmented and virtual reality technologies are opening up, as well as alert people to the societal issues that such realms will need to address in their development Cover Story 12 IN FOCUS Prof. Pan Hui at HKUST(GZ): pioneer of virtual and augmented realities.
And this is only the beginning for the engineering and emerging technologies’ venture. By the end of this year, it is expected that the avatars populating HKUST’s D virtual classroom will possess the facial features and mannerisms of the students and lecturers they represent, with real-time reactions that echo their human counterparts. The classroom’s look’n’feel will be enhanced and scaled up to accommodate or more users. Within three years, it is planned that HKUST and its recently opened HKUST(GZ) campus in Nansha, Guangzhou, will each be operating a physical classroom, equipped with state-of-the-art sensors and motion-capture cameras for the more technically challenging mixed reality learning environment, as well as sharing use of the virtual reality classroom. These arrangements will enable students from both campuses to meet each other and learn together in real time without actually traveling between the two locations. It will allow HKUST students, guest learners and lecturers from other institutions to attend remotely. Broadening out from classes, there will also be a digital twin up and running for each campus, with the expectation that MetaHKUST will add convenience and greater e ciency to multiple aspects of University life. Examples range from visiting the Finance O ce and accessing advisory services to receiving blockchain-secured diplomas or transcripts in the form of non-fungible tokens (NFTs). Implementing MetaHKUST turns the “Uni ed HKUST, Complementary Campuses” vision behind the two campuses’ operations into a working model. Moreover, it becomes possible to identify the real research challenges of building a large-scale teaching and learning meta environment, according to Prof. Pan HUI, one of the international trailblazers of research and development in augmented and virtual realities and a core mover behind the initiative. As one of the rst higher education institutions to build and explore the challenges of such a scalable endeavor beyond gaming, it also o ers a way for HKUST researchers and educators to play a signi cant part in shaping the overall metaverse, now at its formative stage, “the very, very beginning”, Prof. Hui said. What is the metaverse? The Encyclopedia Britannica de nition is a “proposed network of immersive online worlds” where users interact with each other and consume services and goods through virtual or augmented reality technologies, Simulated experience with D near-eye displays of an immersive virtual world + user interaction. Augmented reality (digital overlaid on the physical) + user interaction. 3 Augmented, virtual, and mixed reality. with the issue of how the di erent worlds and communities will seamlessly interconnect to enable users to move across the platforms (interoperability) yet to be resolved. What these worlds will comprise, their design, and standards are also undetermined on a collective basis. For Zuckerberg followers, the metaverse is a utopian creation where people work, socialize, and play games. Business people see a burgeoning realm of market opportunities expected to reach US$8 billion in , according to a Bloomberg forecast. For Prof. Hui and his research team, it is a fascinating technical and societal quest, with the potential for more accessible education, greater social equality, and better communication across the world, as well as major issues that need to be addressed related to privacy, governance, and data security, among other thorny areas. Prof. Hui, who joined HKUST in as a member of the Computer Science and Engineering at the School of Engineering, is now Chair Professor of Computational Media and Arts, and Director of the Center for Metaverse and Computational Creativity at HKUST’s new Guangzhou campus. At the Clear Water Bay campus, he is Chair Professor of Emerging Interdisciplinary Areas, and long-time Director of HKUST-DT System and Media Laboratory, a joint arrangement with Deutsche Telekom Innovation Laboratories, researching virtual reality and augmented reality systems, social media, big data, and mobile computing. Students attend a lecture remotely in HKUST’s virtual reality classroom using their avatars during a recent trial of MetaHKUST technology. The aim is to scale up to 200-plus users. 13 IN FOCUS
14 IN FOCUS Cover Story His own introduction to di erent kinds of reality started a er his undergraduate studies at the University of Hong Kong (HKU) in the s when the internet was the hottest technology of the day for those in electrical and electronic engineering and computer science. A months-long cycling trip to internet-less Tibet in ahead of starting his HKU MPhil postgraduate studies made him curious about alternative communication networks when the internet is not working or accessible. This led on to his PhD ( 8) at the University of Cambridge exploring short-range radio, human mobility, and centered on bridging mobile and social networks. He then worked at Deutsche Telekom Innovation Laboratories in Berlin, Germany, on cutting-edge cloud computing applications before deciding that augmented reality would be the technology of the future, not only for him but everyone else. “In the beginning, nobody knew what was going on with online social networks. Now people have learned… and know they need to be more careful” In the past years, Prof. Hui has focused on building high-performance, energy-e cient and scalable so ware for mobile wearable devices and cloud systems; and devising mobile augmented reality algorithms and systems for immersive data visualization and human-data interaction. His published papers now total more than , with over , citations. He also has European and US patents. The concept of MetaHKUST arose at the end of when students were tiring of the limitations of learning via D videoconferencing technologies and extensively used during the pandemic. Rather than a sense of being outside looking in, classrooms using D virtual reality or mixed reality o er students a way to feel “present”. These perceptions are an essential part of the learning experience, with global research already showing that immersive experiences in general – such as in-country language learning – can boost learning outcomes, he said. The avatar of HKUST(GZ) President Prof. Lionel Ni (left) takes a virtual torch from HKUST Founding President Prof. Woo Chia-Wei’s avatar at the opening ceremony for the Guangzhou campus (see also P15 box). The real President Ni then placed a physical torch on the main stage.
15 IN FOCUS Wearing a headset and represented by their avatar, the students in MetaHKUST’s November trial were able to enter the University’s virtual reality classroom remotely, take a seat, move around, and go up to the professor and talk to him or her. “As they don’t see their own avatar, psychologically, they feel part of that environment,” Prof. Hui said. “You can just turn your head and see other students. You can move around. Education users want some self-presence. You feel the experience is better. With videoconferencing, I cannot move to see another person.” In a mixed reality classroom where a real person can be projected, the “immersive presence” will be even stronger, he noted. Such work involves a host of cutting-edge technical challenges for his team of postdocs, postgraduates, and undergraduates, from over di erent countries (he is a rm believer that diversity sparks creativity and new perspectives a er his own multicultural experiences at Cambridge and in Berlin, among others). System architecture for mixed reality classrooms, user interactivity and perception challenges, and latency issues are just some of the topics being tackled. HKUST academics from a range of disciplines are also participating. These currently include Prof. WANG Yang, Vice-President for Institutional Advancement, who is the leader and driver of MetaHKUST development at the Clear Water Bay campus, Prof. Kani CHEN, Industrial Engineering & Decision Analytics and Mathematics, and Prof. Tristan BRAUD, Integrative Systems & Design and Computer Science & Engineering, both in the School of Engineering. At HKUST(GZ), Prof. WU Jingshen, Vice-President for Teaching and Learning, has been pushing forward the mixed reality classroom construction and development and many students from his Red Bird MPhil Program have chosen MetaHKUST as their MPhil research projects. The aim is to expand the project to more faculty and students at both campuses to accelerate advances. But technical issues are not the only ones covered in a growing number of publications, and postgraduate theses’ topics alongside implementation of the physical and virtual classrooms of MetaHKUST. The initiative is also spurring the exploration of societal questions arising from the extension of virtual and augmented reality technologies into di erent areas of people’s lives. A particularly popular contribution is the comprehensive “All One Needs to Know About Metaverse: A Complete Survey on Technological Singularity, Virtual Ecosystem, and Research Agenda” ( ), a paper available to specialists and non-specialists via the Creative Commons. In one of the rst publications of its kind, Prof. Hui and his co-authors consider not only the impact of metaverse-related technologies but the broader ecosystem such a development encompasses: the virtual economy and content creation, social acceptability, security and privacy, trust and accountability, and the behavior and potential immortality of people’s avatars. The paper has attracted more than citations and over , individual user reads, and brought many invitations from di erent sectors for Prof. Hui to discuss the metaverse. Among recent speaking engagements, Prof. Hui was invited to join a panel discussion in October at the th INTERPOL General Assembly. INTERPOL’s membership Seeing the future Early MetaHKUST demonstrations include an animated dragon entering a mixed reality classroom (scan QR code, or available on YouTube). Meanwhile, large-scale virtual reality was in action at the opening of the HKUST(GZ) campus in September , attracting great media attention. Guests, including overseas higher education presidents and senior management, business leaders and long-term University partners, could either attend in person or send their avatar – a useful as well as memorable way to be present given the di culties of traveling during COVID- .
connects police forces across countries. Its annual assembly drew , security ministers, chiefs of police, and other delegates from around the world in New Delhi, India, including Hong Kong’s police commissioner. “I’ve never felt so safe!” Prof. Hui said. Activities at the assembly included the unveiling of the rst metaverse designed for law enforcement, which is being used for training. As a founding member of the INTERPOL expert group on the metaverse, Prof. Hui will be working closely with law enforcement specialists globally to raise concerns and try to make these realms safe. In providing societal input, Prof. Hui sees a special role for academia in evolving the metaverse. “Sometimes the big tech companies may not have such things in mind when they design their metaverse as they are more interested in the commercial aspects. But as academics, we have to push more on the ethical, societal side. While we focus on technological development of the metaverse, we can also identify what other potential issues there are and voice them as early as possible so that society knows about them. “In the beginning, nobody knew what was going on with online social networks (OSNs), that people could be so manipulated. If this can happen with OSNs, it can happen in the metaverse. But now we have learned from everything that has happened with social networking and people know they need to be more careful.” Raising awareness in this way is particularly important, for Prof. Hui. An IEEE Fellow, International Fellow of the Royal Academy of Engineering, and Member of the Academy of Europe, he views the development of the metaverse as the future that awaits us. He sees people want convenience, business wants e ciency, and both want communication. The three combined will ensure that as virtual and augmented reality technologies improve, more and more content will become available and incorporated into people’s lifestyles. They will be used to hold meetings, when people sit in restaurants, or to call up information overlaid on locations they are visiting. Prof. Hui views the development of the metaverse as the future that awaits us “Fi een years ago, we didn’t have smartphone. At that time, people wouldn’t believe that you would use your phone to do all the things we use it for today. Now we can’t go out without bringing our phones. Similarly, people will get used to virtual content,” he said. It is also why Prof. Hui feels it is imperative at this formative time that it is shaped to be a “better metaverse”. “A metaverse, with respect, accessibility and equality, enabling more global collaboration by providing a platform for people from di erent regions to communicate, study, and learn immersively,” he said. “A metaverse for good.” 16 IN FOCUS Cover Story Members of the multicultural MetaHKUST research team, with Prof. Hui (second row, third right) and Prof. Wang Yang (second row, third left).
appy classes make happy students. As a firm believer in engaging, motivating education, and in making the acquisition of knowledge fun, I always do my best to create a positive teaching environment where students feel cared about and supported. An essential component is to maximize the learning potential of each student in my classes. The positive teacher-student relationship also motivates students to participate and engage in all aspects of their learning, and thus stimulates their curiosity and interest in exploring more about computer science, my main field. “You may be shy at the beginning, but we will all become friends by the end of the semester,” I always say to my students when we first meet. An inspiring learning environment is especially important for classes with difficult theoretical subjects. My award-winning common core course on “Discrete Mathematical Tools for Computer Science” (COMP ) introduces mathematical theories that some students may consider difficult. To stop them losing interest before they get to the “fun” part, where they apply their new knowledge to real-world computer applications, I employ gamification and content such as brain teasers to engage students in critical thinking, build collaborative learning, and sustain enthusiasm. In COMP , the gamification system not only generates more sample questions, but also allows course participants to design and share their own questions. Designing questions requires students to have a good understanding of the materials covered in class and helps them gain a deeper understanding of the topics covered. “The wide adoption of blended learning and experiential learning at HKUST School of Engineering ultimately equips and inspires students to embrace boundless possibilities in their future lives” Technology o ers other pathways to a more engaged environment for learners. Take the cloud-based instant messaging channel I utilize in my classes. This enables students to raise questions on course topics and beyond and take part in discussions among themselves as well as with teaching assistants and myself. Such interaction leads to valuable peer learning and mutual support, along with great camaraderie. Meanwhile, solving real-world problems is one of the most exciting aspects of a career as an engineer. The wide adoption of blended learning and experiential learning at HKUST School of Engineering creates a host of opportunities for students to develop knowledge and skills from direct experience and, ultimately, equip and inspire them to embrace boundless possibilities in their future lives. Indeed, the University is a regional leader in implementing new pedagogies – blended learning, experiential learning, massive open online courses (MOOCs) – to enhance the student learning experience. With HKUST’s support, I have developed a series of blended learning and MOOCs to promote science, technology, engineering, and mathematics (STEM) education, and have accumulated over , learners around the world. Such pedagogies have proved especially e ective during the COVID- pandemic. They also have a significant role to play in opening up motivational, quality education to a wider audience locally, nationally, and globally. Fun pathways to full-potential engineers Prof. Kenneth LEUNG is Assistant Professor of Engineering Education in the Department of Computer Science and Engineering. He was the project leader for the Teaching Development Grant projects “Flipping the Classroom with iPad” and “Social Gaming Platform for COMP 2711”. He received the HKUST Common Core Teaching Excellence Award in 2021 and an Honorary Mention in 2019. century engineers st H By Prof. Kenneth Leung Faculty Column 17 IN FOCUS
Unlocking healthcare secrets of sweat An early career faculty member’s convenient, wearable biosensors are offering a novel technological avenue for people to gain fresh insights on perspiration and wellness erspiration is a bodily function that people o en prefer to hide. However, for Prof. Hnin Yin Yin NYEIN, Chemical and Biological Engineering, the value of sweat is due for a rethink. “Sweat,” she pointed out, “is an underutilized resource with enormous potential in facilitating active, non-invasive, and accurate health monitoring.” Now the rising academic star is setting out to bring sweat back into the spotlight, along with greater recognition of the wealth of insights it o ers into the dynamics at work inside us. Prof. Nyein, selected for MIT Technology Review’s Innovators Under Asia Paci c , is doing so by developing flexible wearable biosensors that utilize our sweating response to provide personalized and preventative healthcare in areas such as metabolism and psychological stress. Her cutting-edge sensors use resting sweat as a constant biofluid source to assess health metrics at a molecular level. The sensors are small, disposable and versatile, and can be tucked into wearables such as wristbands. They are also straightforward and cost-e ective to mass produce, providing accurate analysis of the wearer’s condition and enabling users of all ages and states of health to routinely track their physiology. The sensors represent a technological breakthrough in the eld, opening up on-going sweat monitoring without special collection procedures having to be carried out and paving the way for the potential transformation of diagnostic methods for some diseases. One candidate is cystic brosis, where the conventional test for diagnosis is to measure the chloride ion level in sweat. This requires patients to remain still while their sweat glands are stimulated for sweat to be collected. Patients then need to endure a further wait until lab results are returned. Prof. Nyein’s electrochemical sensors instead P 18 IN FOCUS Young Faculty
o er near real-time sweat stimulation, collection, and analysis in one wearable as the sensors allow for local sweat stimulation. They are optimized to enable rapid uptake of an ultralow sweat secretion rate and can perform immediate and accurate sweat composition measurement and analysis with a volume as little as nanoliters. Furthermore, the technology can be used to measure other molecular constituents of sweat, Prof. Nyein noted. “If a disease’s indicator is involved with sweat biomarkers, we can collaborate with healthcare professionals and modify the sensor technology for a speci c application.” Beyond healthcare, sports science is a likely bene ciary, with the sensors able to estimate an athlete’s loss of body fluid, electrolytes, as well as sweat. “We are already having conversations with sports labs and athletes to conduct further studies. This could potentially bene t athletes by helping them nd their best condition and optimize their performance,” she said. Born in Myanmar, Prof. Nyein earned her bachelor’s degree and PhD in Materials Science and Engineering at the University of California, Berkeley, going on to join Stanford University as a postdoctoral fellow. In addition to her Innovator recognition, Prof. Nyein’s research has been published in eminent scienti c journals such as Nature. She joined HKUST in mid- , where she is relishing the dedication and dynamism of the School of Engineering’s research environment and fellow faculty members as she prepares to take her technology to the next stage of development. In the next ve years, Prof. Nyein’s goal is to acquire a large set of data and map out a baseline representing a healthy and equilibrium body condition. If a user’s sweat measurements deviate from the healthy baseline, it might be an early indicator of a developing health condition, warning the user to get a medical check-up. To uncover correlations between sensor measurements and health status, and determine subsequent actions that could be taken, it is essential to involve experts with core competencies in data analytics, she explained. “We will need help from big data and machine learning specialists to analyze the large database and extract meaningful insights. Lots of interdisciplinary and collaborative research will be required for this idea to become successful.” As Prof. Nyein believes that a positive impact on society is the ultimate goal of every research endeavor, she is looking forward to this challenge and to passing on her expertise to students, empowering them to become creative changemakers for society in the future. “Not every career o ers the opportunity to do what we love while enabling us to mentor the younger generation and help them make a real-world impact,” she said. “I am proud and honored to be able to do both at HKUST at this stage of my career.” Sweat rate optimization Nyein sensors allow rapid uptake at low secretion rates through use of a rigid hydrophilic ller topped with a thin hydrogel in the collection well. Minimal measurement delay Fast collection reduces the time that sweat stays on the skin and facilitates near real-time sweat monitoring. On-the-spot analysis Assessment of sweat rate and compositions can be carried out inside the microfluidic channel. Molecular-level metrics Accurate molecular quanti cation with a volume as small as nanoliters and operational stability up to hours. Biosensor bene ts PCB Sweat gland Sweat rate monitor Microfluidic chip Sweat collection area 19 IN FOCUS
RkJQdWJsaXNoZXIy NDk5Njg=