Presentation: Scaling Deep Learning

Track: Papers in Production: Modern CS in the Real World

Location: Cyril Magnin III

Duration: 1:20pm - 2:00pm

Day of week: Tuesday

Share this on:


NERSC has successfully applied Deep Learning to a range of scientific workloads. Motivated by the volume and complexity of scientific datasets, and the computationally demanding nature of DL, we have undertaken several projects targeted at scaling DL on the largest CPU and GPU-based systems in the world. This talk will explore 2D and 3D convolutional architectures for solving pattern classification, regression and segmentation problems in high-energy physics, cosmology and climate science. Our efforts have resulted in a number of first-time results: scaling Caffe to 9600 Cori/KNL nodes obtaining 15PF performance (SC’17), scaling TensorFlow to 8192 Cori/KNL nodes obtaining 3.5PF performance (SC’18), and finally, scaling TensorFlow to 4560 Summit/Volta nodes, obtaining 1EF performance (SC’18). The talk will review lessons learnt from these projects, and outline future challenges for the DL community.

Speaker: Prabhat

Data and Analytics Group Lead @NERSC

Prabhat leads the Data and Analytics Services team at NERSC; his group is responsible for supporting over 7000 scientific users on NERSC’s HPC systems. His current research interests include Deep Learning, Machine Learning, Applied Statistics and High Performance Computing. In the past, Prabhat has worked on topics in scientific data management; he co-edited a book on ‘High Performance Parallel I/O’. 

Prabhat is the Director of the Big Data Center collaboration between NERSC, Intel, Cray, UC Berkeley, UC Davis, NYU, UBC, Oxford and Liverpool. The BDC project aims at enabling capability, data-intensive science applications on the NERSC Cori system. 

Prabhat received a B.Tech in Computer Science and Engineering from IIT-Delhi (1999) and an ScM in Computer Science from Brown University (2001). He is currently pursuing a PhD in the Earth and Planetary Sciences Department at U.C. Berkeley. Prabhat has co-authored over 150 papers spanning several domain sciences and topics in computer science. He has won 5 Best Paper Awards, 3 Industry Innovation Awards, and he was a part of the team that won the 2018 Gordon Bell Prize for their work on ‘Exascale Deep Learning’.

2019 Tracks

  • Groking Timeseries & Sequential Data

    Techniques, practices, and approaches around time series and sequential data. Expect topics including image recognition, NLP/NLU, preprocess, & crunching of related algorithms.

  • Deep Learning in Practice

    Deep learning use cases around edge computing, deep learning for search, explainability, fairness, and perception.