DeveloperWeek Europe 2021
Wednesday, April 28, 2021
VIP Networking Lounge: Fireside Chat
Join DevNetwork Co-founder Jon Pasky in this fireside chat on DEV TEAMS & SKILLS.
OPEN TALK: Refining Systems Data without Losing Fidelity
Join on HopinIt is not feasible to run an observability infrastructure that is the same size as your production infrastructure. Past a certain scale, the cost to collect, process, and save every log entry, every event, and every trace that your systems generate dramatically outweigh the benefits. If your SLO is 99.95%, then you'll be naively collecting 2,000 times as much data about requests that satisfied your SLI as those that burnt error budget. The question is, how to scale back the flood of data without losing the crucial information your engineering team needs to troubleshoot and understand your system's production behaviors?
Statistics can come to our rescue, enabling us to gather accurate, specific, and error-bounded data on our services' top-level performance and inner workings. This talk advocates a three-R approach to data retention: Reducing junk data, statistically Reusing data points as samples, and Recycling data into counters. We can keep the context of the anomalous data flows and cases in our supported services while not allowing the volume of ordinary data to drown it out.
Download these images to your phone and post using the Instagram app.