You are viewing content from a past/completed QCon

Presentation: Machine Learning Pipeline for Real-time Forecasting @Uber Marketplace

Track: Predictive Data Pipelines & Architectures

Location: Cyril Magnin I

Duration: 1:10pm - 2:00pm

Day of week: Tuesday

Share this on:


Uber's Marketplace is the algorithmic brain behind Uber's ride-sharing services. To help Marketplace systems make proactive and efficient decisions, the Marketplace Forecasting team builds and operates multiple machine learning models to produce forecast of many metrics, including supply and demand, over both granular time and a large number of geo-spatial dimensions.

To empower both data scientists and engineers to build and manage models that range from regressions to neural networks in production, the Marketplace Forecasting team has built an highly scalable and automated machine learning platform that supports efficient feature engineering, distributed model training, turn-key model deployment, metric-based automatic model selection, and scalable model serving.

This talk will discuss how deep learning helps improve the accuracy and efficiency of our forecasting models, the architecture of the machine learning platform, how it was evolved from a simple ad-hoc system, and lessons learned in running the platform in production.

Speaker: Chong Sun

Senior Software Engineer @Uber

Find Chong Sun at

Speaker: Danny Yuan

Real-time Streaming Lead @Uber

Danny Yuan is a software engineer in Uber. He’s currently working on streaming systems for Uber’s marketplace platform. Prior to joining Uber, he worked on building Netflix’s cloud platform. His work includes predictive autoscaling, distributed tracing service, real-time data pipeline that scaled to process hundreds of billions of events every day, and Netflix’s low-latency crypto services.

Find Danny Yuan at

2019 Tracks

  • Predictive Data Pipelines & Architectures

    Case Study focused look at end to end predictive pipelines from places like Salesforce, Uber, Linkedin, & Netflix

  • Sequential Data: Natural Language, Time Series, and Sound

    Techniques, practices, and approaches around time series and sequential data. Expect topics including image recognition, NLP/NLU, preprocess, & crunching of related algorithms.

  • ML in Action

    Applied track demonstrating how to train, score, and handle common machine learning use cases, including heavy concentration in the space of security and fraud