Dask Heartbeat by Coiled: 2021-06-10
• June 10, 2021
The Dask community is highly distributed with different teams working independently. This is powerful but sometimes makes it hard for people within the community to see everything that is going on. The Dask Heartbeat by Coiled is a monthly publication intended to centralize and broadcast Dask news over the previous four weeks.
Dask User Survey 2021
The annual Dask user survey is out! We’d love to hear about how you use Dask and how we can help make Dask better.
Dask Distributed Summit
The Dask Distributed Summit was last month and was attended by hundreds of Dask users. The summit featured talks, workshops, and tutorials from a wide range of different user groups. Recordings from the summit are in the process of being uploaded to Dask’s YouTube channel.
Over the last month both Dask and Distributed versions 2021.05.0, 2021.05.1, and 2021.06.0 were released.
DataFrame HighLevelGraphs by Default
Starting with Dask 2021.05.0, Dask DataFrame computations will start sending
HighLevelGraph's directly from the client to the scheduler by default. Because of this, users should observe a much smaller delay between when they call .
compute() and when the corresponding tasks begin running on workers for large DataFrame computations. This is part of Dask’s ongoing performance improvement efforts.
“Ask a Maintainer” Office Hours
The Dask community has begun hosting a series of office hours for Q&A between Dask maintainers and people in the life sciences community. The first session featured Benjamin Zaitlen (NVIDIA) and the next session, on June 23, will feature Julia Signell (Saturn Cloud). Full details for the next session are available here.
Florian Jetter (Coiled) has had a variety of updates to generally improve Dask’s stability:
More updates are currently in progress and will be included in future releases.
The Evolution of a Dask Distributed User
HTML displays for Jupyter
There has been a recent push to improve and expand Dask’s support for rich HTML output in Jupyter (see the image below). Expect to see these improvements in an upcoming Dask release.
(Image from https://github.com/dask/distributed/pull/4857)
Stability of Dask
Coiled is looking to hire software engineers to work on Dask. If you would like to learn more about this opportunity, see our OSS Engineer job posting.
You’re All Caught Up On Dask
That’s it. Thanks for reading all.
If you’re interested in taking Coiled Cloud for a spin, which provides hosted Dask clusters, docker-less managed software, and zero-click deployments, you can do so for free today when you click below.