The Rise of China in the Supercomputer Race

In the SupercomputingAsia 2019 conference held in Singapore in early 2019, one of the keynote speaker, Professor Lu Yutong, director of the National Supercomputer Center in Guangzhou, discussed the new applications driving the next stage of supercomputer development in China. She noted that aside from the traditional uses of supercomputers in the computation task intensive HPC (high-performance computing) applications such as climate modeling and other scientific computation, the emerging applications in big data and artificial intelligence (AI) are also clamoring for more computing power and there is increasing convergence between HPC and big data/AI applications. The new usage is driving the new supercomputer design and pushing the race in the exascale supercomputer.

The Rise of China in the TOP500 list

Lu was the design director of the Tianhe-1A and Tianhe-2A supercomputers. Tianhe-1A surprised the supercomputing world in 2010 when it became the first Chinese supercomputer to head the TOP500 list of the world’s fastest supercomputers. That was the first time that a Chinese supercomputer was recognized as the most powerful in the world since the biannual list was published in 1993. Tianhe-1A displaced long-time leaders like the US and Japan, and the development surprised most experts as the country was not a known computing power at that time.

Tianhe-1A was the first heterogeneous large-scale supercomputer in the world, using a mix of the traditional central processing units (CPUs) and graphics processing units (GPUs). The GPU performance at that time was not as mature it is today, and the Chinese design team had to work more on the software, libraries and data transfer optimization to achieve the performance. This approach was validated in the past ten years as heterogeneous systems now make up the majority of the most powerful systems on the TOP500 list in the last ten years.

Tianhe-1A’s achievement was repeated in 2013 by its successor-Tianhe-2A, and once again in 2016 by Sunway TaihuLight. China earned the bragging rights as the developer of the world’s most powerful supercomputer for five years (2013-2017) in a row. The US reclaimed global supercomputer leadership in 2019 with the Summit and Sierra, and Sunway TaihuLight and Tianhe-2A are number 3 and 4 ranking supercomputer in the June 2019 TOP500 list.

Sunway TaihuLight was a complete made-in-China machine; it featured locally designed and fabricated CPU Shenwei 26010 processor, replacing the US Xeon chips used in Tianhe-2A and earlier generation of Chinese supercomputers. The move was prompted by the 2015 US ban on sales of high-end Intel chips to Chinese supercomputer manufacturers. The speedy replacement of Xeon chip by Shenwei chips showed the country’s ability to move on supercomputing development, with or without outside parts or talents.

The country also demonstrated its ability to tap the full potential of supercomputers and not just develop them as mere “stunt machine”. In 2017 and 2018, Chinese software developer won two succeeding Gordon Bell Prize, a prestigious prize honoring the most outstanding HPC applications. Home-grown talents are the drivers of the supercomputer development in China as the discipline is considered a national security area and close to technology exchange. Lu got all her degrees from National University of Defense Technology (NUDT).

In the latest TOP500 list, China tops the list with 219 systems running, or 43.8 percent of the total, followed by the US with 116, and Japan ranks third with 29 systems. France with 19, Britain with 18 and Germany with 14.

In terms of computing power as measured by the High-Performance Linpack (HPL) benchmark,  a software library for performing numerical linear algebra on digital computers used to measure a supercomputer’s real performance in practical use and use floating-point operations per second (flops) as a unit of measurement, the US leads by running 38.4 percent of Top500 list flops totalling 1.5 exaflops (10¹⁸ flops, where an “exaflop” refers to one quintillion floating-point computations per second), compared with China’s 29.9 percent Its 116 systems average 5.03 petaflops (10¹⁵) computing power per unit, while the 219 Chinese supercomputers average 2.05 petaflops computing power per unit. The two fastest computers in the world, Oak Ridge National Laboratory’s Summit and Lawrence Livermore National Laboratory’s Sierra accounted for a combined 243.2 petaflops between them, or a full sixth of the Top500 list’s capacity.

The computer company, Lenovo, was the top vendor both by the number of systems and combined petaflops. Its machines achieved upwards of 302 petaflops, followed by IBM’s with 207 petaflops; Cray’s with 183.4 petaflops; Hewlett-Packard with 120.1 petaflops; and Sugon’s with 96 petaflops. In terms of the number of running supercomputers, Lenovo led with 74, 71 from Inspur, and 63 from Sugon. These three companies are all Chinese. For the first time in history, the total aggregate score exceeded 1.5 exaflops, all 500 systems deliver a petaflop or more, with the entry-level to the list now at 1.022 petaflops.

Big Data and AI

The emerging big data and AI research open new application areas for supercomputers. Japan launched the AI Bridging Cloud Infrastructure (ABCI), a supercomputer purpose-built for AI and machine learning applications in the cloud in 2018. The Aurora exascale supercomputer project of the US has been earmarked for AI projects in neuroscience and personalized medicine, among others. Meanwhile, China has identified deep learning applications, such as tumor diagnosis and video analytics, as a critical focus area.

With the expected new mountains of data generated from the emerging IoT (internet of things) devices transmitted by 5G in real-time and the use of AI to utilize those data, the world needs more and more computing power. The race to develop exascale supercomputer is not an end goal, but a means to an end.

The new applications in big data and AI calls for new architecture of supercomputer to optimize performance. Three teams in China have successfully deployed pre-exascale prototypes, namely Sunway, Sugon and Tianhe, each of them has its architectural approach based on experience drawn from academia, industry and existing national supercomputer centre team.

The Tianhe prototype is a heterogenous system using an unnamed new chip design paired with an improved version of the Matrix-2000 accelerator used in Tianhe-2A. The Sugon prototype is also heterogenous using Hygon processor and accelerator, while the Sunway prototype features SW26010 chips used in TaihuLight. The Tianhe prototype is designed to work as a general-purpose exascale system rather than a specialized application exascale system, so the technical approach is different from the other two. China developed and manufactured all the processors in the three prototypes.

The strategy of investing in different architectures at the same time highlight the advantage of a vast domestic market and good availability of computer talents in China. The move will deepen and broaden the country’s expertise, with knowledge gleaned from all three approaches expected to contribute to the final exascale system to be deployed between 2020 and 2021.

The US and China are the acknowledged leaders today in the race to the exascale supercomputer, and both are working to roll out new exascale supercomputer in 2020-2021. The US Summit and Sierra now hold first and second position respectively in the latest TOP500 list, and they are stepping stones toward the goal of exascale supercomputers. To date, the US has set aside USD 430 million for its Exascale Computing Project, and at least USD 400 million for the Aurora exascale supercomputer.

Since the inception of supercomputer in the 1970s, countries have seen how supercomputing leads technology development for the entire IT industry. The US is the leader of supercomputing since inception, and its leadership on chips design and fabrication is very significant over China. However, China has progressed very fast on developing use case of supercomputers, and in the budding AI area. How the race to exascale supercomputer plays out will be a crucial area to watch in the technology competition between the two countries.