Workflow Orchestration with Prefect and Coiled
Jeremiah Lowin, Founder and CEO of Prefect, along with Kevin Kho, Prefect’s Open Source Community Engineer, will discuss updates about the company and demo a newly released feature called the KV Store.
Prefect is an open-source workflow orchestration tool created to handle the modern data stack. As a company, Prefect’s goal is to eliminate negative engineering, which includes failure handling, notifications, and retries. These are necessary distractions that take as much as 80% of an engineer’s time. By reducing negative engineering, engineers can focus on building the things they originally set out to.
Prefect is built on top of Dask, allowing parallel execution of workflows. This means Coiled and Prefect have a very strong synergy. In this demo example, we’ll also show how to spin up a Coiled cluster to execute Prefect jobs during runtime. The Coiled cluster will provide parallelism for dynamically mapped tasks in Prefect.
After attending, you’ll leave knowing:
– How to create a simple Prefect Flow
– How to use the new Prefect feature, the KV store
– How to use Coiled and Prefect together
– More knowledge about Prefect as a company
We’re looking forward to seeing you there!