DeveloperWeek New York 2020 DeveloperWeek New York 2020

Thursday, December 10, 2020

Observability at Brex: Leveraging Logs to Fight Fraud and More
Join on Hopin
Thomas Cesare-Herriau
Thomas Cesare-Herriau
Brex, Lead of Observability
Sherwood Callaway
Sherwood Callaway
Brex, Software Engineer

Logs are ubiquitous and indispensable. They are arguably the most important tool used by software engineers in our day-to-day work. In development, we use them to mark key points in our code, so we can peek into its execution (without using a debugger). In production, they serve as a way of understanding software execution in the real-world, providing immense value at both the macro and micro levels; they can be used to diagnose systemic problems, which affect your entire user base, but also to trace the journey of a single user, thus allowing you to provide individualized support.

Unfortunately, most companies are not taking full advantage of logs, because they fail to see what logs really are: a source of rich, real-time event data. In this session, we will explain how the Observability team at Brex unlocked the power of logs by allowing other teams to access and build on top of structured log streams. This was accomplished by (1) designing and implementing a super-powered logging infrastructure, which delivers log messages to multiple consumers in near-real-time, and (2) creating a strict log schema as well as a schema-adherent logging library to make the underlying data useful and coherent.

We will also deep dive into a particular use case: powering the alerts used by our Fraud team to identify suspicious behavior.