Asia’s First Transnational R&D Center on AI Chip Design Set to Realize Ubiquitous AI Applications in Society

Prof. Tim Cheng together with PhD student Wang Xiaomeng at work at the AI chip design center. [Download Photo]

A new world of mass customization in which everyday devices are enhanced by greater intelligence is set to be fast forwarded through a unique multinational research center, helmed by the head of the School of Engineering.

Be it clothes, jewelry, buildings, or hospital beds, the realization of ubiquitous AI applications in society is now moving closer through the groundbreaking work of Asia’s first transnational research and development center on AI chip design, led by Dean of Engineering Prof. Tim CHENG Kwang-Ting.

The multidisciplinary AI Chip Center for Emerging Smart Systems (ACCESS) is striving to revolutionize the global AI chip design and talent pipelines. To achieve this, it is bringing together a leading end-to-end R&D consortium of expertise that is driving forward innovation through novel AI chips that improve performance and energy efficiency by up to 1,000 times. At the same time, software-hardware co-designed solutions will enable companies of all sizes, including start-ups and smaller firms, to rapidly deploy their own customized AI-driven applications.

The pioneering center is being supported under the Hong Kong government’s collaborative InnoHK Clusters initiative, receiving HK$443.9 million in initial funding. Participating universities comprise HKUST, Stanford University, the University of Hong Kong, and Chinese University of Hong Kong.

Addressing key hardware challenges for AI & machine learning tasks

In recent years, advances in AI and machine learning accuracy have given rise to a greater number of AI-driven applications in areas such as face and object recognition, natural language processing, and autonomous vehicles. However, such developments have come at a price, with increased accuracy involving large computing and memory resources, long training periods for AI models, and major expense, limiting widespread use of such applications. ACCESS, established in 2020, is setting out to change this.

Customized computing chips for AI applications, also known as AI accelerators, are processors designed to speed up artificial intelligence and machine learning applications, including internet of things, computer vision, and other data-intensive or sensor-driven tasks. “Companies want to cram more and more intelligence into today’s sensors and devices to enhance their functions,” said Prof. Cheng, an internationally recognized leader in electronic design automation, integrated circuit design, and computer vision. “They need powerful but small, energy-efficient AI chips to carry out specific and ubiquitous tasks. But such embedded intelligence is not yet widely available.”

The center’s research areas address four key technical areas: enabling technology for emerging computer systems, architecture and heterogeneous system integration, AI-assisted electronic design automation for AI hardware, and hardware-accelerated AI applications. Among its 14 projects to date, teams are developing a new generation of computing in memory (CIM) chips that serve as independent data processors, eliminating the need for the chip to send data to the cloud or a central server for analysis and processing and then return the data. Getting rid of these steps can make the CIM chips hundreds of times faster, with current literature and results indicating that ACCESS chips are three times more efficient than the best-performing CIM chips currently available.

The center has also successfully designed a new generation of optimized neural network prototypes on computer architecture and hardware models and completed verification on a Field Programmable Gate Array (FPGA) platform. It is expected that final design and manufacturing of ultra-low-power chips will be completed in the second half of 2022.

A massive opportunity for research, business, and social impact

“Just think of all the sectors wanting such AI acceleration, from transportation to fintech, medicine to education,” Prof. Cheng said. “It represents a massive opportunity for research, business, and social impact, penetrating different market segments and product applications, and involving hundreds of millions of units.” The global market for AI was estimated at US$47.1 billion in 2020 and forecast to increase to over US$290 billion by 2026.¹

Another of ACCESS’s innovative aspects is its bid to offer an alternative to the current business model for AI chip design, enabling a greater number of companies to obtain chips tailored more precisely to their specific needs.

While giants such as Apple, Tesla, and drone-maker DJI can afford to employ large teams to design customized chips for their own devices (and sole use), such resources are beyond most companies. Firms requiring chips for AI products instead pass their algorithms and specifications to chip-design companies. However, the functions provided by the latter are often designed for a wide range of applications rather than optimized for a certain AI application and algorithm. This means client companies may only use a part of the purchased chip’s functionality and potential performance, resulting in mismatch, wasted resources, and inefficiency. It also makes it harder for companies to differentiate themselves from their competitors’ AI products.

ACCESS, on the other hand, has set out to devise open-source methodologies and tools for design automation, which enables AI customization to be rapidly carried out by relatively few designers, speeding up design productivity and time to market. “Our goal is to produce customized chips with a small team within a short period of time. Instead of years, it can be done in one to two quarters. Instead of hundreds of people being required to design these chips, it will involve a team of tens of people,” Prof. Cheng said.

Closing “valley of death” – the gap between university research and industrial deployment

To deliver both increased AI chip performance and greater automation of design, the center’s integrated R&D platform comprises an unusually broad array of research expertise spanning software, hardware, chip structure and design, and AI algorithms and applications, linking upstream to downstream. The center has taken on around 20+ researchers and engineers and is aiming to recruit a total of 80 full-time staff with extensive industrial experience. In addition, 36 top faculty members and over 100 doctoral researchers from the four universities are on board.

Its cooperative relationship with industry is another important factor, according to Prof. Cheng. Companies can gain advice from ACCESS researchers on AI algorithms and AI chip architecture offering the most efficient solutions for their businesses while ACCESS teams can learn more about the latest developments and requirements of the AI application market. Incubation of new start-ups would also result. “The need is mutual. We need to reach out for new applications and impact, and companies seeking applications already have customers that will be the beneficiaries. The next step is how to build a customized chip so their product becomes more competitive and energy efficient,” he said.

Furthermore, such cooperation should see more university integrated circuit research being commercialized. “We always say the ‘valley of death’ is the gap between university research and industrial deployment. Universities can turn their concepts into demos but face many problems before they can reach production. Industry finds interesting demos from universities but needs several years to work out how they can apply them. ACCESS has been conceived to integrate the R&D pipeline and remove the valley of death, which I think is a unique opportunity.”

In the right place, at the right time

Despite the center’s work in AI and chip design being at the intersection of two of the hottest areas of electronics research, Prof. Cheng is optimistic about ACCESS’s ability to attract top talents to Hong Kong given its distinctive integrated R&D pipeline, significant investment in resources by government, and HKUST’s commitment to drive the hub forward. “Other research centers are not connecting all the different dots together as a unified center and can only do part of what ACCESS can do,” he pointed out. It can thus appeal to both experienced researchers seeking to grow their experience in a cutting-edge transnational research center and young talents keen to gain wider knowledge and skills through the comprehensive range of expertise that the platform brings together.

Meanwhile, the location of ACCESS in Hong Kong and the openings provided by the Greater Bay Area, which already possesses world-leading domains in AI applications and infrastructure such as machine learning and computer vision technology, among others, offer tremendous potential for research and business development. “I think the center will put Hong Kong on the global map in this area. You have great customers, great technology, and the next stage is to tie this together with the AI hardware,” Prof. Cheng explained. “We are in the right place at the right time.”

¹ https://www.researchandmarkets.com/reports/1056044/artificial_intelligence_ai_global_market

Ubiquitous AI that sticks to its task

As AI extends its reach into different areas of people’s lives, concerns are frequently raised around areas such as privacy, responsibility, and ethics. In this regard, Prof. Cheng noted that ACCESS’s work focuses only on core application-specific AI, not general purpose. “We want a device to be more intelligent but intelligent only for the particular task you want for that device. Thus, we contain intelligence to a very specific application, a very specific task. Nothing more, nothing less.”