Dask Heartbeat by Coiled: 2021-06-10

James Bourbeau June 10, 2021

, , ,


Introduction

The Dask community is highly distributed with different teams working independently. This is powerful but sometimes makes it hard for people within the community to see everything that is going on. The Dask Heartbeat by Coiled is a monthly publication intended to centralize and broadcast Dask news over the previous four weeks.  

If you want something added to this list either send an e-mail at info@coiled.io, or tweet and tag @dask_dev and we’ll try to include it. Keep reading for the latest updates.

Dask User Survey 2021

The annual Dask user survey is out! We’d love to hear about how you use Dask and how we can help make Dask better. 

Dask Distributed Summit 

The Dask Distributed Summit was last month and was attended by hundreds of Dask users. The summit featured talks, workshops, and tutorials from a wide range of different user groups. Recordings from the summit are in the process of being uploaded to Dask’s YouTube channel

Releases

Over the last month both Dask and Distributed versions 2021.05.0, 2021.05.1, and 2021.06.0 were released.

DataFrame HighLevelGraphs by Default 

Starting with Dask 2021.05.0, Dask DataFrame computations will start sending HighLevelGraph's directly from the client to the scheduler by default. Because of this, users should observe a much smaller delay between when they call .compute() and when the corresponding tasks begin running on workers for large DataFrame computations. This is part of Dask’s ongoing performance improvement efforts

“Ask a Maintainer” Office Hours 

The Dask community has begun hosting a series of office hours for Q&A between Dask maintainers and people in the life sciences community. The first session featured Benjamin Zaitlen (NVIDIA) and the next session, on June 23, will feature Julia Signell (Saturn Cloud). Full details for the next session are available here

Stability improvements 

Florian Jetter (Coiled) has had a variety of updates to generally improve Dask’s stability:

More updates are currently in progress and will be included in future releases.

The Evolution of a Dask Distributed User 

Jacob Tomlinson (NVIDIA) wrote a blog post on the path he’s seen many Dask users take when scaling from one machine to thousands.

HTML displays for Jupyter 

There has been a recent push to improve and expand Dask’s support for rich HTML output in Jupyter (see the image below). Expect to see these improvements in an upcoming Dask release. 

(Image from https://github.com/dask/distributed/pull/4857

Stability of Dask 

Matthew Rocklin (Coiled) wrote a blog post which discusses several factors which have influenced Dask’s stability recently.

Hiring 

Coiled is looking to hire software engineers to work on Dask. If you would like to learn more about this opportunity, see our OSS Engineer job posting.

You’re All Caught Up On Dask

That’s it. Thanks for reading all.

If you’re interested in taking Coiled Cloud for a spin, which provides hosted Dask clusters, docker-less managed software, and zero-click deployments, you can do so for free today when you click below.

Try Coiled Cloud