Scale By the Bay Scale By the Bay

Funnel Rocket: A Serverless Query Engine


Elad Rosenheim
Stackbit, Developer Growth

20+ years experience in software development, all around the software stack. One of the first employees at Dynamic Yield, where he has built teams, managed the R&D org and now working with the CTO on new technology initiatives.

Funnel Rocket is a newly released open-source engine for running serverless big data queries at scale - quickly and cheaply.

It builds on robust foundations: AWS Lambda, Pandas, Apache Arrow and Redis, which are used in a variety of interesting ways to facilitate fast, highly distributed job with minimal orchestration logic.

Funnel Rocket is currently purpose-built for a specific type of query: from a sea of raw data, find users who've matched multiple conditions - optionally in a specific sequence. However, I believe there's a big potential for cloud-native big data tools: composing existing building blocks by cloud providers + the OSS community to create lightweight, fast-scaling and cost-efficient solutions.