Unlocking the Metaverse inside Our Universe

When social media giant Facebook rebranded itself as “Meta” in late 2021, companies and institutions – large or small, national or regional – began entering the global Metaverse space race. As a result, the term “Meta” now embraces everything from education to entertainment, from retail to telecommunications.

While visions of what the Metaverse is or might eventually become varied, experts broadly agree it will almost certainly revolutionize everyday life by changing the ways we work, learn, and interact with others.

The so called “Digital Big Bang” is the end result of steady technological and digital infrastructure advancements including augmented reality (AR), virtual reality (VR), and blockchain cloud services. Such advances have also resulted in an explosion in research into relevant areas across the higher education sector.

Leveraging over 10 years’ specialist research expertise in immersive human-data interaction, Prof. Pan HUI, HKUST’s affiliate Professor of Emerging Interdisciplinary Areas and Computer Science & Engineering and HKUST(GZ)’s Chair Professor of Computational Media and Arts (CMA) Thrust, is leading the way here at HKUST.

The Metaverse made simple

“Advances in mobile devices, wearables, and technologies mean content creation and consumption is increasingly happening in today’s ever-more three-dimensional digital world,” says Prof. Hui. “The Metaverse is an inevitable end product of these changes.”

Prof. Hui envisions the Metaverse as ideally being a series of self-sustaining, shared, perpetual, concurrent 3D virtual spaces linked together to form a perceived virtual universe. He believes that the resultant world will be rich in “surreality”, with physical and virtual objects so seamlessly intertwined that they are almost indistinguishable.

The professor goes on to add that many people’s view of the Metaverse is born of gaming and entertainment experiences. For example, several blockchain-based Metaverse platforms have enabled users to tailor unique digital identities – or avatars – to purchase, rent, or sell virtual items using cryptocurrencies.

To amplify users’ feeling of presence, some platforms allow VR headset wearers to sample many immersive experiences from a first-person perspective. Specific examples of activities taking place in the three-dimensional virtual environment include playing shoot-em-up laser games with friends or “attending” work meetings with colleagues.

Events specifically tailored for virtual spaces are also on the rise. These virtual events are free from audience capacity limits and other physical barriers, and can often enable greater audience engagement. Such activities range from personal events like weddings to industry-wide showcases such as fashion previews. 

While potential Metaverse functions such as human beings being able to holographically and instantly teleport to any location are still some way off, the market looks set to hit US$800 billion by 2024. All of which represents a huge marketing opportunity for shrewd entrepreneurs.

In his comprehensive paper “All One Needs to Know about Metaverse” in late 2021, Prof. Hui offers a timely guide for both those excited and alarmed by the innovation’s rapid expansion.

In addition to evaluating the Metaverse’s technological fundamentals, the professor’s paper examines major obstacles blocking the evolution of a healthy Metaverse ecosystem, including social acceptability, security and privacy, trust and accountability. “Building the Metaverse calls for a holistic approach as there are still many challenges preventing its full integration into our everyday lives,” says the professor.

To date, Prof. Hui’s paper has generated over 70,000 individual user reads and sparked constructive discussions across many sectors. Local educational institutions and tech companies such as Tencent eager to learn more about the Metaverse’s potential and social implications have invited the professor to share his research insights at talks and seminars.

Laying the foundation

Since joining HKUST in January 2013, Prof. Hui has done his utmost to drive the development of immersive technologies across several fields. One of his most notable roles involves directorship of HKUST-DT System and Media Laboratory.

A joint research institute founded by HKUST and leading German telecommunications company Deutsche Telekom (DT), the initiative pursues cutting-edge research in AR, big data, and mobile computing. The collaboration has already pioneered scores of AR/VR systems and co-filed some 20 patents with major telecommunications partners such as Deutsche Telekom, AT&T, Huawei, and China Mobile.

Ubii (“Ubiquitous interface and interaction”) is a classic example. Pairing an augmented user interface and physical objects, the system enables users to perform essential administrative tasks via simple hand gestures, such as dragging a file from a computer onto a projection screen.

Other research highlights include CloudAR. Aimed at software developers, this prototyping toolkit integrates a variety of technologies covering everything from computation offloading and object tracking to hand gesture/face recognition. Prof. Hui has also been involved in the development of an edge computing solution that effectively processes and plays 360-degree 16K resolution videos on wearables and mobile devices.

Heading up a team of computer science professors and integrative systems designers at HKUST’s Center for Metaverse and Computational Creativity, Prof. Hui is also involved in an especially ambitious project. Its ultimate aim is to help HKUST become the first in the world to host an opening ceremony in the Metaverse.

Founded in 2021, the Center’s mission remains to accelerate the exploration and exploitation of technology-powered creativity. Its ultimate aim is to cement HKUST’s position at the forefront of Metaverse research and collaboration.

The university campus as a physical-digital Metaverse

One of the Center’s key undertakings, the “MetaHKUST” project will kick off with this September’s landmark grand opening of our new campus in Nansha, Guangzhou. Guests from all over the world will be invited to socialize at various launch events either in person or virtually via their avatars.

Prof. Hui stresses that the MetaHKUST project is not a one-off gimmick, but will continue to empower meaningful strides forward on both HKUST campuses for many years. “MetaHKUST’s goal is multi-faceted. As well as moving the two campuses closer, it will add new dimensions to the HKUST family’s shared experience by continuing to benefit members with advanced technologies,” he says.

During MetaHKUST’s final phase, all physical and virtual experiences will be consolidated under one roof. Apart from building digital recreations of actual facilities (i.e. a digital twin), it will migrate even more virtual objects to both physical campuses.

What this means is that students wearing VR headsets in lectures will be able to learn more interactively through three-dimensional visualization of data and information superimposed on real objects. They will also be able to virtually “sit in” on lectures and events happening at other campuses many miles away.

As CMA’s Chair Professor and Associate Head for Research, Prof. Hui is already exploring ways novel technologies can help stimulate creative thinking amongst both teachers and students. The end result will be a powerful tool via which users can express their artistic vision. 

“CMA’s aim is to harness the most advanced technologies to promote social impact via the interplay of art and technology,” Prof. Hui explains. “In doing so, our art faculty and students will collaborate via AR and VR, AI-generated art, information art and design, and data visualization.”

The recent creation of a physical-digital gallery experience that aims at enabling art lovers to appreciate virtual exhibitions through AR devices or smartphones is an excellent example of this interdisciplinary hybrid in action. In addition to a smart navigation function guiding visitors around display areas, the system will virtually overlay extra information on actual exhibits.

“By shaping and reshaping how we perceive and interact with others and our world, I believe the blending of physical and digital entities will help us better understand and enrich our lives, environment, and communities,” says the professor.

Asked whether humanity should favor or fear increased integration of the physical and digital worlds, Prof. Hui feels everyone will eventually learn to accept the co-existence both technically and psychologically. “The Metaverse is inevitable whether we want it or not,” he admits. “The key is to find ways to exploit its tremendous potential while maintaining a healthy ecosystem.”

The professor makes an intriguing observation about the Metaverse’s possible color connotations: “If you type ‘Metaverse’ into a search engine, many simulated images in the color purple will appear. As this shade is not commonly found in nature, some regard it as being dystopian,” he notes. “This may well reflect humanity’s skepticism about unknown technologies.”

So which color best sums up the Metaverse at its most positive? “I prefer to view it as not one, but a kaleidoscope of colors,” says the professor with a smile. “The Metaverse is such an exciting world, it looks best in shades as wide ranging as the experiences it generates.”

(This news was originally published by the HKUST Public Affairs Office here.)