How Double River reduced simulation time from days to hours for faster trading decisions
Building a modern data stack with Coiled, Prefect, and Polars
Introduction: Double River's Computational Challenge#
Double River Investments is a hedge fund focused on trading equities internationally and in the US. Running market-neutral strategies and statistical arbitrage models, they rely entirely on algorithms and computational power to determine what to trade and when to trade it.
We're fully algorithmic. We let computers and models decide what we're going to trade, when we're going to trade it, and execute. A lot of our work is researching what kind of data will give us insights into market movements, and then turning that into actionable insights.
Nelson Griffiths
Head of Engineering, Double River Investments
Unlike many quantitative funds that focus exclusively on US markets, Double River trades internationally, including in markets that most quant firms avoid due to data challenges or trading costs. This global approach creates unique computational demands, requiring them to process diverse datasets across multiple markets and time zones.
For a quantitative trading operation, computational efficiency directly impacts business results. The faster they can process data and run simulations, the more strategies they can test, and the more confident they can be in their trading decisions.
The Limitations of Their Previous Stack#
Double River's initial technical stack centered around Pandas for data manipulation and Google Cloud Run for execution. While this approach worked initially, they quickly encountered significant limitations as their strategies grew more sophisticated.
We went from running things on Cloud Run, which has a max of 32 gigs of RAM they give you, to not knowing what to do because we couldn't fit some of our regression outputs or our regression code on those machines. It was taking too much data, too much compute.
These constraints created real business challenges. Complex simulations that should inform trading decisions were taking days to complete when run sequentially. The team found themselves making difficult tradeoffs between the scope of their analysis and timely execution.
Cloud Run's limitations weren't just about memory constraints. The service also provided limited hardware options, typically offering only older, slower processors. For computationally intensive financial modeling, this meant longer processing times and fewer iterations on their strategies.
Additionally, managing cloud infrastructure was becoming a significant time sink for the engineering team. They found themselves spending valuable hours in the Google Cloud console, managing VMs and troubleshooting environment issues instead of focusing on their core mission of building better trading strategies.
The Modern Python Stack Transformation#
To overcome these limitations, Double River implemented a modern Python stack built around several key technologies:
- Polars for high-performance data processing and manipulation
- Prefect for reliable workflow orchestration and scheduling
- Coiled for flexible, on-demand cloud computing resources
- Snowflake for scalable data storage
Most of our data resides in Snowflake with some of it in Cloud Storage buckets. Prefect orchestrates everything. Most of our data manipulation happens in Polars. Some of it we use raw Python. Some of it we use Rust, just without Polars, depending on what it is.
The shift to Polars from Pandas delivered immediate performance improvements. Its ability to work around Python's Global Interpreter Lock and leverage modern CPU architectures made previously impossible calculations feasible.
Prefect provided a reliable orchestration layer, ensuring that their increasingly complex data pipelines ran reliably and on schedule. This was crucial for a trading operation where timing can make the difference between profit and loss.
Coiled eliminated their infrastructure constraints and gave them access to virtually unlimited computational resources on demand, completing their modern data stack.
Why Coiled Was the Right Solution#
After evaluating several options for scaling their compute resources, Double River chose Coiled for its unique combination of flexibility, ease of use, and seamless integration with their existing Python workflow.
We gave Coiled a try and it just worked out of the box.
The hardware flexibility Coiled provided was particularly valuable. Instead of being limited to Cloud Run's fixed offerings, they could now access any machine type in Google Cloud's portfolio with a simple code change.
We can always use the fastest machines. We don't have to worry about infrastructure.
The PRs to make a change to switch to a faster machine is so easy. Google just released their C4A ARM machines, which are way faster than their T2A1s were. And we just made a bunch of PRs to switch them over. And it was like a 30 second thing because all we had to do was change the VM type in our code.
For the research team, many of whom weren't cloud experts, Coiled made accessing powerful computing resources trivially simple:
We threw one command in their repo and said if you ever want to run what you're running, but you want to run it on a big machine, type this command before you run it and then you're done. And so they were just like, cool, don't think about it, it works.
Coiled's package synchronization capability was another game-changer. It eliminated the need to build and push Docker images for every code change, dramatically accelerating their development cycle.
Business Impact: Everyone Does Better Work#
The implementation of this modern stack, centered around Coiled, transformed Double River's research and trading capabilities. Complex simulations that previously took days now complete in hours:
Sometimes we need to run tons of simulations and our choices are either go manually spin up VMs and do them in batches or run them on my computer in the background.
Now it's like throw a Coiled decorator on something and all 20 will run at the same time.
This acceleration has directly impacted their ability to develop and refine trading strategies. The team can now test more hypotheses, explore more parameters, and iterate more quickly on their models. This leads to more robust strategies and better trading decisions.
The engineering team also reclaimed valuable time previously spent on infrastructure management:
I never open the Google Cloud VM Compute tab anymore. I used to make sure people had spun down their big machines. And when people needed something I had to spun up a new machine and ensure that the environment worked.
This freed capacity allows the engineering team to focus on higher-value activities, including their own research contributions. The result is a more productive team delivering better results across the board.
Looking Forward: Scaling with Confidence#
With their computational infrastructure now able to scale on demand, Double River is well-positioned for growth. They're preparing to raise outside capital for the first time, confident that their technical foundation can handle the increased scale that will come with managing more assets.
We are really big fans of both Polars and Prefect. We've been Polars fans from the get-go, and as Prefect has built out, we've found more and more that we've liked about it. When you add Coiled to that mix, you've got a tech stack that could build a lot of companies.
Their modern Python stack—Polars for data processing, Prefect for orchestration, and Coiled for flexible compute—provides a solid foundation that can scale with their business without requiring proportional increases in engineering resources or infrastructure overhead.
For Double River, computational efficiency isn't just a technical concern—it's a competitive advantage. In the fast-moving world of quantitative trading, the ability to process more data, test more strategies, and execute decisions faster translates directly to better performance.
Cost is nice, speed is important. Being able to execute and hit the times that you have to hit so that you can trade things when you want to trade them is critical.
Nelson Griffiths
Head of Engineering, Double River Investments
With Coiled as a core part of their technical foundation, Double River has the computational power they need to compete with much larger firms while maintaining the agility and innovation that drives their success.