Thursday, October 28, 2021
Funnel Rocket is a newly released open-source engine for running serverless big data queries at scale - quickly and cheaply.
It builds on robust foundations: AWS Lambda, Pandas, Apache Arrow and Redis, which are used in a variety of interesting ways to facilitate fast, highly distributed job with minimal orchestration logic.
Funnel Rocket is currently purpose-built for a specific type of query: from a sea of raw data, find users who've matched multiple conditions - optionally in a specific sequence. However, I believe there's a big potential for cloud-native big data tools: composing existing building blocks by cloud providers + the OSS community to create lightweight, fast-scaling and cost-efficient solutions.
Download these images to your phone and post using the Instagram app.