Presentation: Petastorm: A Light-Weight Approach to Building ML Pipelines @Uber
Share this on:
Abstract
Data produced and managed by Big Data systems like Apache Spark and Hive cannot be directly consumed by Deep Learning systems like Tensorflow and PyTorch. Petastorm bridges this gap by enabling direct consumption of data in Apache Parqet format into Tensorflow and PyTorch. In this talk, we describe how Petastorm facilitates tighter integration between Big Data and Deep Learning worlds; simplifies data management and data pipelines; and speeds up model experimentation.
2019 Tracks
-
AI Meets the Physical World
Where AI touches the physical world, think drones, ROS, NVidia, TPU and more.
-
Hands-on Codelabs & Speakers Office Hours
Codelabs are a self-guided tutorial of a product, API, or tool kit followed by an Office Hour period with the lab’s creator.
-
Papers in Production: Modern CS in the Real World
Groundbreaking papers make real-world impact.
-
Hands-on Codelabs & Speakers Office Hours
Codelabs are a self-guided tutorial of a product, API, or tool kit followed by an Office Hour period with the lab’s creator.
-
Solving Software Engineering Problems with Machine Learning
Interesting machine learning use cases changing how we develop software today, including planned topics touching on infrastructure optimization, developer experience, security, and more.
-
Predictive Architectures in the Real World
Case Study focused look at end to end predictive pipelines from places like Salesforce, Uber, Linkedin, & Netflix.
-
Predictive Data Pipelines & Architectures
Case Study focused look at end to end predictive pipelines from places like Salesforce, Uber, Linkedin, & Netflix
-
Sequential Data: Natural Language, Time Series, and Sound
Techniques, practices, and approaches around time series and sequential data. Expect topics including image recognition, NLP/NLU, preprocess, & crunching of related algorithms.
-
ML in Action
Applied track demonstrating how to train, score, and handle common machine learning use cases, including heavy concentration in the space of security and fraud