Scaling Your Data Pipelines with Prefect and Coiled
Data engineering efforts inside an organization often start small, with a handful of relatively simple pipelines. Over time, many data teams find themselves with a large network of complex pipelines, and performance and reliability can become big issues.
In this session, Matt Rocklin, CEO of Coiled, and Chris White, CTO of Prefect, will share how, by using Prefect and Coiled together, data teams can drive higher pipeline performance at scale with less engineering overhead. They’ll also discuss trends in the Python space, and share some personal learnings and advice.
During this session you will learn:
– What Prefect and Coiled do, and how they fit into the data engineering landscape
– Challenges data engineers face when needing to scale and when the right time is to consider a scaled solution
– How to scale with Prefect and Coiled, with hands-on examples, from a single process to an auto-scaling cluster
– Matt and Chris’s views on where the modern data stack is going and trends to follow
Join us on Tuesday, October 26th at 6 PM EST by signing up here for Scaling Your Data Pipelines with Prefect and Coiled!