The Dask community is highly distributed with different teams working independently. This is powerful but sometimes makes it hard for people within the community to see everything that is going on. The Dask Heartbeat by Coiled is a monthly publication intended to centralize and broadcast Dask news over the previous four weeks.
Scaling data analysis with Python has never been easier. Here are ways you can use Dask to work with larger datasets.
Matt McCormick, who works at Kitware maintaining the Insight Toolkit (ITK) library, joins Hugo Bowne-Anderson to discuss large-scale microscopic image visualization and analysis with on-demand Dask cloud clusters.
Dr. Chelle Gentemann is an oceanographer specializing in data taken from NASA satellites and robotic sailing vehicles. She is the lead scientist on an exciting new proposal for a new NASA satellite called Butterfly, and a fierce advocate for open science and inclusivity. Chelle is not only a remarkable scientist, she is also a dear …
Distributed Data Science and Oceanography with Dask Read More »
Data Scientists are increasingly using Python and the Python ecosystem of tools for their analysis. Combined with the growing popularity of big data, this brings the challenge of scaling data science workflows. Dask is a library built for this exact purpose – making it easy to scale your Python code, and serve as a toolbox …
Scale your data science workflows with Python and Dask Read More »
Hugo Bowne-Anderson, the Head of Data Science Evangelism and Marketing at Coiled, joined us last month for a webinar on this topic of increasing relevance: Making scaled data science work for people, and not the other way around. He discussed the challenges of distributed computing, the challenges of data culture in an organization, and how …
Making scaled data science work for people and not the other way around Read More »
TL;DR: In recent months, the Dask team has started refactoring task graph processing and the scheduler in order to handle large, complex workloads better and to provide a better interactive user experience. A key piece of the refactor involves moving parts of the graph processing from the Client to the Scheduler. You won’t need to …
You may have heard data scientists extol the virtues of scaling data science in Python with Dask. Their praises ring true: Dask works with all of the Python tools and libraries you love, providing an easy-to-use framework for parallel computations. With minimal code changes, you can process your data in parallel, which means it will …
And how this has and has not worked out Distributed computing is hard because of configuration consistency. Hosted Jupyter notebooks make it easy for the group that controls the distributed computing environment to also control the user’s development environment. By removing the user’s ability to control their environment we ensure consistency and a predictable experience; …
Introduction The Dask community is highly distributed with different teams working independently. This is powerful but sometimes makes it hard for people within the community to see everything that is going on. The Dask Heartbeat by Coiled is a monthly publication intended to centralize and broadcast Dask news over the previous four weeks. If you want something …
Welcome to the Coiled Employee Spotlight, a new segment on our blog where we’ll be highlighting and getting to know the fantastic members of our team. In these blogs, we’ll interview Coiled team members in order to get to know more about their background and interests and pick their brain on data science, ML, and …
Coiled Employee Spotlight: Software Engineer Jasmine Sandhu Read More »
Gil Forsyth and Mike McCarty joined us recently for a Webinar on Distributed Machine Learning at Capital One. Gil is a Senior Manager of Software Engineering and Mike is a Director of Software Engineering at Capital One. They discussed how Capital One scales machine learning workflows, the challenges of distributed computing in an industry setting, …
