Data pipelines are the foundation for success in data analytics, so understanding how they work is of the utmost importance. Join us for hours of expert-led sessions that will give you insight into how data is moved, processed, and transformed to support analytics and reporting needs. You'll also learn how to address common challenges like monitoring and managing broken pipelines, explore considerations for choosing and connecting open source frameworks, commercial products, and homegrown solutions, and more.
About the Data Superstream Series: This three-part Superstream series is designed to help your organization maximize the business impact of your data. Each day covers different topics, with unique sessions lasting no more than four hours. And they're packed with insights from key innovators and the latest tools and technologies to help you stay ahead of it all.
What you'll learn and how you can apply it
Learn how to build, deploy, and run a fully functioning ETL pipeline with Airflow
Discover how to build robust data pipelines at scale
Understand challenges in managing and monitoring hundreds of thousands of pipelines—and get tips on automating them
Explore approaches to historical data preprocessing and data lifecycle management
You're a data or software engineer or solution architect interested in learning about the latest trends in moving, processing, and transforming data.
You want to learn how to address common challenges and improve the scalability and stability of your pipelines.
You want to better understand the systems that you already use and learn how to take full advantage of their capabilities.
Recommended follow-up
Read Data Pipelines Pocket Reference (book)
Read Data Science on AWS (book)
Read Data Quality Fundamentals (early release book)
Read What Is Data Observability? (report)
Explore Build a Robust Data Pipeline (four-part interactive scenario set)
Screenshots