You are viewing content from a past/completed QCon -

Presentation: Scaling Deep Learning to Petaflops and Beyond!

Track: Papers in Production: Modern CS in the Real World

Location: Cyril Magnin III

Duration: 2:20pm - 3:00pm

Day of week:

Slides: Download Slides

This presentation is now available to view on InfoQ.com

Watch video with transcript

Abstract

NERSC has successfully applied Deep Learning to a range of scientific workloads. Motivated by the volume and complexity of scientific datasets, and the computationally demanding nature of DL, we have undertaken several projects targeted at scaling DL on the largest CPU and GPU-based systems in the world. This talk will explore 2D and 3D convolutional architectures for solving pattern classification, regression and segmentation problems in high-energy physics, cosmology and climate science. Our efforts have resulted in a number of first-time results: scaling Caffe to 9600 Cori/KNL nodes obtaining 15PF performance (SC’17), scaling TensorFlow to 8192 Cori/KNL nodes obtaining 3.5PF performance (SC’18), and finally, scaling TensorFlow to 4560 Summit/Volta nodes, obtaining 1EF performance (SC’18). The talk will review lessons learnt from these projects, and outline future challenges for the DL community.

Speaker: Prabhat

Data and Analytics Group Lead @NERSC

Prabhat leads the Data and Analytics Services team at NERSC; his group is responsible for supporting over 7000 scientific users on NERSC’s HPC systems. His current research interests include Deep Learning, Machine Learning, Applied Statistics and High Performance Computing. In the past, Prabhat has worked on topics in scientific data management; he co-edited a book on ‘High Performance Parallel I/O’. 

Prabhat is the Director of the Big Data Center collaboration between NERSC, Intel, Cray, UC Berkeley, UC Davis, NYU, UBC, Oxford and Liverpool. The BDC project aims at enabling capability, data-intensive science applications on the NERSC Cori system. 

Prabhat received a B.Tech in Computer Science and Engineering from IIT-Delhi (1999) and an ScM in Computer Science from Brown University (2001). He is currently pursuing a PhD in the Earth and Planetary Sciences Department at U.C. Berkeley. Prabhat has co-authored over 150 papers spanning several domain sciences and topics in computer science. He has won 5 Best Paper Awards, 3 Industry Innovation Awards, and he was a part of the team that won the 2018 Gordon Bell Prize for their work on ‘Exascale Deep Learning’.

Find Prabhat at

Speaker page