Introduction The Dask community is highly distributed with different teams working independently. This is powerful but sometimes makes it hard for people within the community to see everything that is going on. The Dask Heartbeat by Coiled is a monthly publication intended to centralize and broadcast Dask news over the previous four weeks. If you want something …
We were recently joined by Crissman Loomis, AI Engineer at Preferred Networks, and James Bourbeau, Lead OSS Engineer at Coiled for a webinar on Training ML Models Faster: Scalable Hyperparameter Optimization with Optuna and Dask. Optuna is a framework that automates hyperparameter optimization and Dask is a library for scaling Python. In the webinar, Crissman …
Scalable Hyperparameter Optimization with Optuna and Dask Read More »
Brendan Collins (co-founder at makepath), who has created and/or contributed to libraries including Datashader, Bokeh and xarray-spatial, joins to discuss how to scale xarray-spatial with Dask.
Dask’s distributed scheduler implements work-stealing — taking pending tasks away from their assigned worker and giving them to a different worker. Work stealing improves performance by providing a counterbalance to the default data-locality-focused scheduling heuristic.
XGBoost does best when it can train on the entire dataset at once. What to do when we don’t have enough RAM?
Bursting XGBoost training from your laptop to a Dask cluster allows training on out-of-core data, and saves hours of engineering work.
It’s our firssst birthday today! 🐍 It has been an exhilarating year. We grew from a company of 1 to 6 to over 20 people, our products and our community have matured, and we’re steadily marching forward. You can read the details in Coiled, one year in. We’re also hosting a special birthday live stream …
In this article, we discuss an interesting use case of Dask and Coiled: Accelerating Volumetric X-ray Microstructural Analytics using distributed and high-performance computing.
To celebrate Coiled’s 1st birthday, we’re having a special Livestream with Matthew Rocklin, Dask maintainer and Coiled CEO, to discuss ongoing work to accelerate the Dask scheduler.
Eric Dill, Director of Data Science platform, DTN, discusses the scalable computing landscape, why Spark is still popular, and where Dask fits in the ecosystem.
Introduction The Dask community is highly distributed with different teams working independently. This is powerful but sometimes makes it hard for people within the community to see everything that is going on. The Dask Heartbeat by Coiled is a bi-weekly publication intended to centralize and broadcast Dask news over the previous two weeks. If you want something …
Coiled, a Dask company, is about one year old. We’ll have a more official celebration in mid-February (official date of incorporation), but I wanted to take this opportunity to talk a little bit about the journey over the last year, where that has placed us today, and what I think comes next.
Autoscaling can save money and improve performance by “right-sizing” your distributed compute clusters as workloads change. Dask does this by collecting a lot of data about the work you’re running, and scaling to a size that ensures all your pending work gets done quickly.
