Scale By the Bay Scale By the Bay

Thursday, October 28, 2021

Funnel Rocket: A Serverless Query Engine
Elad Rosenheim
Elad Rosenheim
Stackbit, Developer Growth

Funnel Rocket is a newly released open-source engine for running serverless big data queries at scale - quickly and cheaply.

It builds on robust foundations: AWS Lambda, Pandas, Apache Arrow and Redis, which are used in a variety of interesting ways to facilitate fast, highly distributed job with minimal orchestration logic.

Funnel Rocket is currently purpose-built for a specific type of query: from a sea of raw data, find users who've matched multiple conditions - optionally in a specific sequence. However, I believe there's a big potential for cloud-native big data tools: composing existing building blocks by cloud providers + the OSS community to create lightweight, fast-scaling and cost-efficient solutions.