Choosing Architecture: Microservices, Containers, Kubernetes
Tuesday, May 10, 2022
In the last decade, the development of modern horizontally scalable open-source Big Data technologies such as Apache Cassandra (for data storage), and Apache Kafka (for data streaming) enabled cost-effective, highly scalable, reliable, low-latency applications, and made these technologies increasingly ubiquitous. To enable reliable horizontal scalability, both Cassandra and Kafka utilize partitioning (for concurrency) and replication (for reliability and availability) across clustered servers. But building scalable applications isn’t as easy as just throwing more servers at the clusters, and unexpected speed humps are common. Consequently, you also need to understand the performance impact of partitions, replication, and clusters; monitor the correct metrics to have an end-to-end view of applications and clusters; conduct careful benchmarking, and scale and tune iteratively to take into account performance insights and optimizations. In this presentation, I will explore some of the performance goals, challenges, solutions, and results I discovered over the last 5 years building multiple realistic demonstration applications. The examples will include trade-offs with elastic Cassandra auto-scaling, scaling a Cassandra and Kafka anomaly detection application to 19 Billion checks per day, and building low-latency streaming data pipelines using Kafka Connect for multiple heterogeneous source and sink systems.
Wednesday, May 11, 2022
DevSecOps is an inclusive term yet most security teams argue they remain left out of the effort to automate deployment processes. While DevOps teams and their tools are progressing along the automation curve, security teams are left to manual hand-offs from DevOps. As a result, DevOps and security teams struggle to collaborate as their organizations modernize and adopt cloud-native technologies.
Using the resources we already possess, is it possible for cybersecurity to gather speed and collaborate with DevOps? In this session, I will cover the current automation challenges faced by cybersecurity, namely a lack in collaborative when it comes to automation with DevOps. I will show how security tools can be used in DevSecOps pipelines to automate routine security tasks including: adding infrastructure-as-code security scanning before deploying Kubernetes clusters, scanning Kubernetes deployed on cloud infrastructure against the CIS K8s benchmark , and even performing remediation with Kubernetes in real-world DevSecvOps pipelines.
Cybersecurity is a growing field but the gap between DevOps and cybersecurity is widening. The faster we make DevSecOps a reality, the quicker we can enable cybersecurity teams to truly contribute in a collaborative approach with DevOps with cloud-native technologies.
Development teams are increasingly choosing a microservices architecture in order to boost apps' agility, scalability, and maintainability. With this decision to switch to the modular software architecture — in which each service is a separate unit with its own logic and database that communicates with other units through APIs — comes the need for new testing strategies and new testing tools.
This presentation will provide an overview of the key microservices testing strategies, types of tests, & the best testing tools to support dev teams in accelerating application development.