We were recently joined by Holden Karau, Spark maintainer and former Princess of the Covariance Matrix (long story), for a discussion on the design of Dask, how it compares to PySpark, and why these tradeoffs were chosen.
The Dask community is highly distributed with different teams working independently. This is powerful but sometimes makes it hard for people within the community to see everything that is going on. The Dask/Coiled Heartbeat is intended to centralize and broadcast Dask news over the previous week.
As is the guiding philosophy behind OSS, Dask is a community-driven project, and the content in this post follows suit. The open-source curriculum below pulls from diverse resources, experts, and platforms to guide you in learning Dask in 2020 via the most straightforward path possible. Enjoy!
The goal of our training sessions is to empower people to solve large-scale data problems through the easiest and simplest means possible, all while leveraging skills they have already. Often, the way to do that is with Dask, Python, and PyData.
James Bourbeau, Dask maintainer and software engineer at Coiled, recently joined us for a Science Thursday session on “Scalable Python Deployments as a Service”. In this post, we summarize the key takeaways from the stream. We’ll cover: A brief overview of Dask An introduction to Coiled and its offerings Spinning up a cluster on AWS …
We recently chatted with Andy Müller, core developer of scikit-learn and Principal Research Software Development Engineer at Microsoft. Andy is one of the most influential minds in data science with a CV to match. He shares his thoughts on distributed machine learning with open-source tools like Dask-ML as well as proprietary tools from the big cloud providers. …
Florian Jetter, Sr Data Scientist at Blue Yonder, joins Hugo Bowne-Anderson and James Bourbeau to discuss supply chain analytics at scale. Blue Yonder provides software-as-a-service products around supply chain management. Along such a supply chain there are billions of billions of decisions to be made, how much to order, when to ship products, how much …
Lots of people talk about “democratizing” data science and machine learning. what could be more democratic — in the sense of widely accessible — than SQL, PyData, and scaling data science to larger datasets and models?
The Coiled team was recently joined by Brett Naul, founding engineer at Replica, where we discussed large-scale machine learning and travel simulations for urban planning. During this session, we learned more about: Interactive products for urban planning, Building synthetic populations from large data sets like the US census, Data engineering workflow with Dask and Prefect, …
Holden Karau joins Matt Rocklin & Hugo Bowne-Anderson to discuss the design of Dask, how it compares to PySpark, and why these tradeoffs were chosen. There are many different distributed systems solving what at first glance might seem like “the same problem.” Here we’ll talk about the skeletons in the respective closets of our different …
We recently chatted with Andy Müller, core developer of scikit-learn and Principal Research Software Development Engineer at Microsoft. Andy is one of the most influential minds in data science with a CV to match. He shares his thoughts on distributed machine learning with open-source tools like Dask-ML as well as with proprietary tools from the …
James Bourbeau, Dask core contributor and maintainer who works at Coiled building tools for scalable computing, joins Hugo Bowne-Anderson to discuss and code about scalable data science deployments as a service and how he thinks about these things at Coiled. Coiled Cloud is an opinionated deployment-as-a-service product/library for scaling Python data science and machine learning …
