Presentation: Petastorm: A Light-Weight Approach to Building ML Pipelines @Uber
Share this on:
Abstract
Data produced and managed by Big Data systems like Apache Spark and Hive cannot be directly consumed by Deep Learning systems like Tensorflow and PyTorch. Petastorm bridges this gap by enabling direct consumption of data in Apache Parqet format into Tensorflow and PyTorch. In this talk, we describe how Petastorm facilitates tighter integration between Big Data and Deep Learning worlds; simplifies data management and data pipelines; and speeds up model experimentation.
2019 Tracks
-
Predictive Architectures in the Real World
Case Study focused look at end to end predictive pipelines from places like Salesforce, Uber, Linkedin, & Netflix.
-
Predictive Data Pipelines & Architectures
Case Study focused look at end to end predictive pipelines from places like Salesforce, Uber, Linkedin, & Netflix
-
Sequential Data: Natural Language, Time Series, and Sound
Techniques, practices, and approaches around time series and sequential data. Expect topics including image recognition, NLP/NLU, preprocess, & crunching of related algorithms.
-
ML in Action
Applied track demonstrating how to train, score, and handle common machine learning use cases, including heavy concentration in the space of security and fraud
-
Deep Learning in Practice
Deep learning use cases around edge computing, deep learning for search, explainability, fairness, and perception.
-
Handling Sequential Data Like an Expert / ML Applied to Operations
Discussing the complexities of time (half track) and Machine Learning in the data center (half track). Exploring topics from hyper loglog to predictive auto-scaling in each of two half-day tracks.
Half-day tracks
-
AI Meets the Physical World
Where AI touches the physical world, think drones, ROS, NVidia, TPU and more.
-
Hands-on Codelabs & Speakers Office Hours
Codelabs are a self-guided tutorial of a product, API, or tool kit followed by an Office Hour period with the lab’s creator.
-
Papers in Production: Modern CS in the Real World
Groundbreaking papers make real-world impact.