Scale By the Bay Scale By the Bay

Thursday, October 28, 2021

Apache Spark Performance Tuning Session with Delight
Jean-Yves Stephan
Jean-Yves Stephan
Data Mechanics, Co-Founder & CEO

Delight is an open-source monitoring dashboard for Apache Spark displaying system metrics (Memory, CPU, ...) aligned on the same timeline as your Spark jobs and stages. It's a great complement to the Spark UI to help you troubleshoot the performance and stability of your applications. Delight works for free on top of all commercial and open-source Spark platforms.

In this talk, JY will use Delight and the Spark UI to go through real-world performance tuning sessions for Apache Spark. Parallelism, shuffle, memory, instance type.. he will show you how Delight can help make your applications stable and cost-effective at scale.