Our analysis of supercomputer efficiency over time

Sam Naffziger of AMD and I just published our report on the efficiency of supercomputers over time.

Here’s the abstract:

The energy efficiency of computing devices is a topic of ongoing research and public interest. While trends in the efficiency of laptops and desktops have been well studied, there has been surprisingly little attention paid to trends in the efficiency of high-performance computing installations (known colloquially as “supercomputers”). This article analyzes data from the industry site Top 500 (http://www.top500.org) to assess how the efficiency of supercomputers has changed over the past decade. It also compares how the efficiency and performance of a recently announced supercomputer, scheduled to be completed in 2021, compares to a simple extrapolation of those historical trends. The maximum performance of the most powerful supercomputers doubled every 2.3 years in the past decade (representing a slowdown from doubling every year from 2002 to 2009), while the efficiency of those computers doubled every 2.1 years from 2009 to 2019.

The Top 500 data have some issues, but this effort is a reasonable attempt to glean some meaning from them. We focused on analyzing each supercomputer based on the year that it started operation, so we could track meaningful technology trends.  The Top 500 tracks the same supercomputers over time as they move down the list of top 500 machines, so we eliminated all but the first instance of any particular installation’s listing in the Top 500.

We split analysis of the performance of supercomputers into two periods, 2002 to 2009 and 2009 to 2019. The 1st period shows rapid growth (doubling every year or so) while the 2nd period shows a much slower doubling time of about 2.3 years, as well as much great variance in the data.

The efficiency data only start to become reliable around 2009, so that’s where we started the data analysis. Efficiency of supercomputers in the Top 500 data doubled every 2.1 years for the top performing machine, the top 10% of the top performing machines, and for the complete set of machines reported in the Top 500, which is pretty remarkable regularity. One caveat is that the R-squared of the linear regression goes down a lot as we regress on the bigger data sets.

We then focused in on the trend for the top performing machine so we could extrapolate that trend and compare it to an upcoming supercomputer (Frontier) built using Cray and AMD technology. The performance trend data show that Frontier is significantly above the trend line when it’s expected to start operation in 2021.

The story is the same for efficiency, although Frontier’s height above the trend line is less dramatic than for performance.

The details of how Frontier is expected to achieve these results are not yet public, but the article discusses some of the most promising areas for efficiency improvements as well as focuses on the need for future work, especially in the area of co-design of hardware and software.

Koomey, Jonathan, Zachary Schmidt, and Samuel Naffziger. 2019. Supercomputing Performance And Efficiency: An Exploration Of Recent History And Near-Term Projections. Burlingame, CA: Koomey Analytics.  [https://www.amd.com/en/system/files?file=documents/Supercomputing-Performance-Efficiency.pdf]