Wednesday, February 17, 2021
Talk to executives in IT divisions of large enterprises about security and invariably the conversation will hover around
Is DevSecOps the only thing you need to do for security in your IT division or is there more?
What impact does bringing in secure culture in an engineering context mean?
What handshake is needed between the IT function and the security / risk function for large enterprises?
How does this impact roles and responsibilities of a developer?
This talk is an attempt to answer questions such as these using a real world examples of transformations seen in Fortune 100 companies.
BUSINESS PROBLEM & CHALLENGE
Network automation was not well practiced or well understood inside our network engineering team, but was sorely needed. We needed to decrease effort and mistakes on daily management tasks by minimizing the direct human interaction with network devices. High on our priority list of goals, was improving network security by recognizing and fixing security vulnerabilities and increasing the network performance.
HOW WE OVERCAME THE CHALLENGE
We started by simplifying daily workflows, baselining our configurations and removing snowflakes. While this can be very labour-intensive at the outset when you’re working on a global scale in a highly critical customer environment, the long-term benefits far outweighed the labour.
Next, we created an inventory file which listed all network devices by type, model, location and IP address - this enabled us to retrieve info about devices and using network programming and automation, allowing us to deploy to all devices, or even a subset of devices (eg. only those in a specific area), depending on what was needed. The benefit to this is we avoided manual configuration and logging into hundreds of different devices to add configuration to each one.
Overcoming these two big challenges set us up for success and enabled us to deploy at a global scale. We lived by the mantra:
“If it’s not repeatable, it’s not automatable. And if it’s not automatable, it’s not scalable.”
LEARNINGS AND MEASURABLE OUTCOMES
So what did we learn? For starters, it can be hard to automate a use case or test in the same way you would if doing it manually. Testing that requires physical movement, for example losing service provider links or hardware failure is also a challenge, as automating something like that is very tricky. We also learned that code reviews are extremely important. Shared code ownership means the entire team can make changes anywhere, at any time.
And what we’re the measurable outcomes?
Faster deployment times - we were able to efficiently push changes to over 300 network devices and audit the configuration of our global network, taking the time to execute from days down to hours.
Removed the fear of large and complex network changes - the accuracy and efficiency with which we were able to deploy at scale, gave business and the leadership more confidence in subsequent large scale network changes and deployments.
Faster feedback on network changes - it allowed us to get reviews on network configuration changes with version control and peer review, treating infrastructure as code (IaC).
Helped with adhering to PSIRT/CSIRT challenging timeframes and security vulnerabilities.
We started by simplifying daily workflows, baselining our configurations and removing snowflakes. Next, we created an inventory file which listed all network devices by type, model, location and IP address.
Speed of deployment; speed of feedback on network changes; speed of adherence to PSIRT/CSIRT timeframes; confidence and buy-in from senior leadership on subsequent deployments!
Apache Kafka is a complex system with multiple parameters to configure for different use cases. As a Consulting Engineer at Confluent, I can see many clients who need to benchmark their production systems and understand its capacity. Benchmark details can be used to optimize the throughput and better utilize resources. Benchmarking and measuring aren’t just for finding a bottleneck; they’re about trying to better understand the loads you’re placing on the system. This talk will focus on methods and tools of Apache Kafka performance analysis and benchmarking. It will be helpful to anyone trying to operate a large Apache Kafka cluster and achieve the throughput and latency goals.
Most organizations engaged in transformation today are moving from left to right in digitally-driven maturity models. The objectives are well known: increase agility, boost productivity, and provide seamless digital experiences for consumers.
Architects play a pivotal role as the curators of this transformation. In this session, Asanka will share his experience on how architects can contribute and introduce a framework to follow on refactoring enterprises.
The purpose of this effort is to collect, organize, and analyze data on remote learning at a school, teacher, and student levels across multiple platforms and activities to monitor, track, and report on remote/blended learning across all schools in the New York City.
-Ingest data from all learning tools and DOE enterprise systems
Support all data formats
-Access to system-wide data across all platforms, resources, school types
-Ability to connect any additional data sources to central database
-Supplemental learning resources by grade level
-Access to individual-level data, in order to create aggregates and cuts as needed (including by school, by demographics, etc.)
Tracing can be very powerful. It gives the ability to connect the customer experience to the backend services several hops away. This comes down to what information is in your traces. There isn't one standard set of tags to add for EVERY application. It comes down to how traces are used and what matters to your organization. During this session we will discuss the need for tracing, dive into the why (and what) you would want to trace via distributed tracing, and delve into the OpenTelemetry specs and architecture on how we can tailor (or tag) our traces. You’ll leave with an understanding of the semantic context in OpenTelemetry and how it might help you understand your applications.
Thursday, February 18, 2021
Microservices are becoming mainstream now and the majority of new applications are developed using microservices architecture. But they throw some unique challenges around DevOps and Observability due to following:
- Rise of API ecosystem
- Application complexity is rising
- Applications are becoming more distributed
- Tech Stack is becoming broader and bigger everyday
- Hybrid and multi-cloud is becoming reality
- Monoliths to Microservices
Distributed Tracing, which is getting a lot of mindshare in the last few years is now becoming the new foundation of DevOps. Unlike traditional Application Performance Monitoring (APM), Distributed Tracing is intended to address the observability challenges and use-cases in the new microservices world.
eBPF advancement in Linux kernel brings whole new opportunities around kernel and application tracing. eBPF used in conjunction with distributed tracing unlock many interesting opportunities.
Eventually, Distributed Tracing will be a new APM, addressing MELT use-cases comprehensively (Metrics, Events, Logs and Traces). It is particularly well-suited to debugging and monitoring modern distributed software architectures, such as microservices. It helps pinpoint or isolate where failures occur and what causes sub-optimal performance.
It basically traces a network request as it travels through your services, as it would be in a microservices-based architecture. One of the main reasons you may want to do this is to troubleshoot or monitor the latency of a request as it travels through the different services.
In this talk, we will discuss Distributed Tracing and it’s various use cases in today’s cloud native world along with Demo of Distributed Tracing using some sample applications!
The growing demand for serverless computing runs parallel with the demand for businesses to rapidly scale up their software applications. Within the last five years, serverless has grown exponentially, however it still only represents a tiny fraction of all computing that takes place in the cloud.
There are important factors gating wider adoption, the most significant of these is the the absence of support for stateful workloads and applications. In this session, Rodric Rabbah, CTO and co-founder of Nimbella, will address those limitations and make suggestions on how the industry could realize the full potential of serverless architecture by leveraging native support for state in the serverless platform. Topics will cover how this approach works and why it is necessary to support state natively in the serverless platform.
This talk will leave attendees with the understanding of what stateful serverless offers developers, focusing on the ability to build complex workflows and stateful APIs with all the benefits of serverless. This includes no infrastructure or data stores to manage, no storage to provision explicitly, instant availability, and elastic scalability. In all, stateful serverless enables enterprises to rapidly scale their application layer in a cloud native environment.
Containers are the new ZIP format to distribute software" is a fitting description of today's development world. However, it's not always that easy and this talk highlights the development of Elastic's container strategy over time:
- Docker images: A new distribution model.
- Helm Chart: Going from demo to production.
- Kubernetes Operator: Day two operations with upgrades and scaling.
Besides the strategy, we'll discuss specific technical details and hurdles that appeared during the development. Or why the future is a combination of Helm Chart and Operator.
OPEN TALK (CloudWorld): Learn How to Manage Better Your Costs on AWS, Make Your Bill Less Scary or Your Credits Last LongerJoin on Hopin
Should you really always run your cluster in multiple availability zones? How can a transition rule to Glacier double your storage costs? I want to monitor and understand my data transfer costs, where should I start? Why are backups eating my database budget? What, one more storage class on S3? Following so-called “best practices” works only when you fully understand the implications, costs included. We will address a few cloud anti-patterns, making your bill smaller and your deployment better.
At the beginning of this year, I built Superpeer with a serverless architecture. We got a total of $10M investment in two different rounds.
Going serverless was one of the best decisions I made at the very beginning, and I am going to talk about the topics below:
- What did we learn from Cloud Functions, and why we switched to Cloud Run?
- How could I continue sleeping during our biggest launches?
- How serverless allowed me to scale my engineering team to 17 engineers?
- How about the costs?
Not too long ago, a reactive variant of the JDBC API was released, known as Reactive Relational Database Connectivity (R2DBC). While R2DBC started as an experiment to enable integration of SQL databases into systems that use reactive programming models, it now specifies a robust specification that can be implemented to manage data in a fully-reactive and completely non-blocking fashion.
In this session, we’ll briefly go over the fundamentals that make R2DBC so powerful. We'll keep light on the slides so that we can jump directly into application code to get a first-hand look at the recently released R2DBC client from MariaDB. From there we'll examine how you can take advantage of crucial concepts, like event-driven behavior and backpressure, that enable fully-reactive, non-blocking interactions with a relational database.
We went from a single monolith to a set of microservices that are small, lightweight, and easy to implement. Microservices enable reusability, make it easier to change and scale apps on demand but they also introduce new problems. How do microservices interact with each other toward a common goal? How do you figure out what went wrong when a business process composed of several microservices fails? Should there be a central orchestrator controlling all interactions between services or should each service work independently, in a loosely coupled way, and only interact through shared events? In this talk, we’ll explore the Choreography vs Orchestration question and see demos of some of the tools that can help.
OPEN TALK (CloudWorld): Pragmatic App Migration to the Cloud: Quarkus, Kotlin, Hazelcast and GraaIVM in ActionJoin on Hopin
At a point in the past, it was forecast that Java would die, but the JVM platform would be its legacy. And in fact, for a long time, the JVM has been tremendously successful. Wikipedia itself lists a bunch of languages that run on it, some of them close to Java e.g. Kotlin, some of them very remote e.g. Clojure.
But nowadays, the Cloud is becoming ubiquitous. Containerization is the way to go to alleviate some of the vendor lock-in issues. Kubernetes is a de facto platform. If a container needs to be killed for whatever reason (resource consumption, unhealthy, etc.), a new one needs to replace it as fast as possible. In that context, the JVM seems to be a dead-end: its startup time is huge in comparison to a native process. Likewise, it consumes a lot of memory that just increase the monthly bill.
What does that mean for us developers? Has all the time spent in learning the JVM ecosystem been invested with no hope of return over investment? Shall we need to invest even more time in new languages, frameworks, libraries, etc.? That is one possibility for sure. But we can also leverage our existing knowledge, and embrace the Cloud and containers ways with the help of some tools.
In this talk, I’ll create a simple URL shortener with a “standard” stack: Kotlin, JAX-RS and Hazelcast. Then, with the help of Quarkus and GraalVM, I’ll turn this application into a native executable with all Cloud/Container related work has been moved to the build process.
For the past few years the terms cloud-native and microservice architecture have been used interchangeably. However recently people have questioned whether microservices are the go-to solution for all problems, reflecting a growing maturity in our understanding of how to best take advantage of the benefits of cloud computing.
While microservices have many benefits, they also bring their own challenges and better understanding (and some failed microservices projects) have led to a more critical evaluation of the suitability of microservices to all problems. This evolved understanding has resulted in a new interest in the place for monoliths and other architecture styles in the cloud. Whether referred to as macroservices, or cloud-native monoliths there is an apparent loosening of the association between the terms cloud-native and microservices.
What does this industry evolution mean? Can monoliths truly be cloud-native? If it's not microservices, what makes something cloud-native? Join me as we take a look at microservices and monoliths in the context of cloud-native.
OPEN TALK: Accelerating to Escape Velocity: Migrating Applications to the Cloud with Programmable Data InfrastructureJoin on Hopin
Building on its rich history of innovation, Choice Hotels used a cloud-first approach to accelerate innovation and battle digital-native competitors.
To escape the gravitational pull of data and make the cloud promise a reality, Choice Hotels brought on an API-first programmable data infrastructure platform to automate data compliance and data delivery. By doing so, the organization broke down data silos, allowing software teams to significantly improve the quality and speed of application testing and development.
In this fireside chat with Jason Simpson, VP of Engineering at Choice Hotels, you’ll discover:
Learnings from a migration of hundreds of applications from legacy systems to Amazon Web Services (AWS)
Insights on leveraging data to minimize the risk of outages and delivering a frictionless experience for customers and franchisees across platforms
Reaping the scalability of cloud to manage unexpected business shifts including impacts from COVID-19
Perspectives on how Programmable Data Infrastructure can be leveraged for Migration, CI/CD Acceleration and even Service Restoration with APM Integration
Speakers: Jason Simpson, VP of Engineering at Choice Hotels & Alex Hesterberg, Chief Customer Officer at Delphix
If you are building applications today, you are probably using either cloud or Kubernetes ... or both! As a result, we are entering an era that we don’t have to make complex architecture decisions by weighing tradeoffs on scale, uptime, and usability. Patrick McFadin has been building and supporting scale applications for a long time and has seen all the evolution that has brought us to today. Engineer to engineer, Patrick wants to show you his journey into this world and what he’s been doing at DataStax and the Apache Cassandra project to help make it a reality. Here’s what he’ll cover.
-How you can shorten application development time and ship code fast
-The role of open source in this next wave of modern application development
-Ways to participate in this fast-moving community of data services
-How you can futureproof your code and be ready for the next big thing
Learn about a cloud service which provides not only app testing but also AI/ML driven app insights that can drive lower TCO by offering targeted testing recommendations
In this session, we’ll explore how to secure your delivery pipelines, from development to deployment with key learnings including:
- Combining continuous packaging with integration & delivery.
- Applying holistic security principles across the whole value stream.
- Using infrastructure-as-code techniques to build, stage, and deploy.
Many enterprises want to become multi-cloud or have the ability to switch clouds. However, cloud providers try very hard to lock in customers with proprietary core services. Cloud Sidecar provides a simple way to take existing software and deploy it to different clouds without complex rewrites. Learn about Cloud Sidecar and how it can unlock your software from only working on one cloud.
All SaaS products have reports, but most don't truly embrace analytics as a differentiator. You'll learn how to use analytics within your product to increase customer satisfaction, decrease churn and expand revenue opportunities using data that is already being captured by your product.
Friday, February 19, 2021
KEYNOTE (CloudWorld): Ro -- Serverless - How to Speed up Tests over 300 Times and Achieve Continuous Feedback?Join on Hopin
Modern software development methodologies and approaches help to increase delivery speed while still caring about software quality. However, if we think about manual regression tests, they are not only time consuming and repeatable, but also feedback time is quite high. These are the reasons why regression tests are often automated.
Automated tests can provide results faster and it’s possible to execute them more frequently than manual ones. They can decrease the overall time needed for tests, but what's probably even more important, they allow to test and receive feedback earlier in the development process. It's essential if we want to release and deliver business value faster and more frequently.
But what if we have more and more tests and even automated execution of them takes too much time - 10 minutes... 30 minutes... maybe even hours? Should we consider the ability to execute full tests set just a few times a day as something normal? Is adding more compute resources the only option to reduce the execution time? Or maybe there are too many high-level tests and some of them should be replaced by low-level ones according to the Test Pyramid? Is the Test Pyramid still valid in the cloud world?
During the presentation, you will see how the serverless cloud services like AWS Lambda may be used to run tests in the highly parallelized environment that can speed up test execution even hundreds of times.
Benchmarking cloud providers is difficult and time-consuming. In 2017 our cloud provider decisions were based on tests and informed assumptions. But when we started to see results that contradicted our assumptions, we dug further. Were our assumptions leading us astray?
This led us to develop the Cloud Report, a benchmark comparison of Amazon Web Services (AWS),Google Cloud Platform (GCP), and Microsoft Azure. Every year since, cloud infrastructure improves, so we have developed a cadence of benchmarking to keep our own engineering team, as well as our customers, at the cutting edge. In this talk, John breaks down how we benchmark each of the clouds, how we select the benchmarks, and the difficulties that we encounter -- no matter how variables we control for.
OPEN TALK: WordPress as a Service: Get It Done in Less Than 30 Minutes with Terraform & K8s on IONOS CloudJoin on Hopin
Let’s assume that we’d like to become the next big internet tycoon by offering an awesome, high end managed WordPress service to the world, including monitoring, completely dedicated database, backups, restores and the whole nine yards. In addition to our great idea, let’s also set some…less realistic goals:
– We need a Proof of Concept up and running in less than 30 minutes
– We need to reach our goal(s) without having to deep-dive into WordPress specifics
– A new dedicated wordpress site needs to be deployable with a single command or API call.
– Upon deployment, each WordPress website needs to be secured through HTTPS
This speech will give you an overview on how to start a project like this by leveraging the power of Kubernetes Operators running on the IONOS Cloud infrastructure.
Cloud deployments offer the potential for almost infinite resources and flexible scalability. But there are so many options! It can be overwhelming to know which services are best for your use case. Building distributed systems which take advantage of in-memory computing only adds to the complexity. During this session we will introduce the Apache Ignite in-memory computing platform and identify key metrics that can help you maximize application performance on your existing cloud infrastructure. We will provide best practices on how best to structure and deploy in-memory applications on both public and hybrid clouds.
Manage Databases with MySQL
Create, Deploy, and Manage Containers with Docker
Build and Manage Websites with Plesk
Over the past 17 years, the Linode community has grown to serve millions of developers and small businesses around the world. Tap into a diverse and growing community to accelerate your app's awareness and adoption.
Simplify your sales cycle, discover new customers and leverage co-marketing support from the Linode Partner Team. We're here to guide you through listing on the Linode Marketplace, and see what opportunities come next.
Once your listing is created, developers, teams, and companies can discover your solution in our library and quickly deploy it from our Cloud Manager, API, or CLI.
We’ve all heard the buzz around pushing application security into the hands of developers, but if you’re like most companies, it has been hard to actually make this a reality. You aren’t alone - putting the culture, processes, and tooling into place to make this happen is tough. Join StackHawk CSO Scott Gerlach as he shares his triumphs and failures while building DevSecOps practices and tools at companies such as GoDaddy, SendGrid, and Twilio. Dig into specific reasons why developers struggle with AppSec and what you can do to make it work better. Whether you’re a seasoned DevSecOps pro or just starting out, this will be an entertaining (and judgement-free!) talk you won’t want to miss!
AI is at the peak of its hype cycle. Too often, ‘AI-capable’ refers to marketing claims instead of practical value add. For this reason, developers tend to be skeptical about AI-driven development. Slapdash application of AI ends up diminishing developer’s creativity and effectiveness.
When implemented in inventive, unique ways, AI dramatically improves the productivity of developers and opens up new opportunities for creativity – especially when applied to cloud app development. Furthermore, beyond the initial development process, AI has the potential to completely transform the entire application lifecycle. Pairing AI with visual, model-driven development enables guidance to be both more powerful and less obtrusive and can compress CI/CD pipelines into days or even hours, instead of weeks.
Come join us as we discuss the three most fundamental design decisions regarding integrating AI into an application platform, our experience analyzing models based on 10s of millions of application graphs and flows, and explore the implications for improving your cloud development productivity by 100x.
OPEN TALK (CloudWorld): The Good, the Bad and the Ugly: Keeping Your Service Reliable with SLIs and SLOsJoin on Hopin
With increasing service traffic and services scaling, the need to ensure reliability and customer satisfaction has never been higher. How can we ensure that a service is reliable and the needs of customers are met?
Through defining and monitoring SLIs and SLOs! This talk will cover why strategically defining SLIs, SLOs and SLAs and monitoring SLIs can help improve the reliability of your service and ensure customer satisfaction in the long term. We follow this by walking through the process of defining these critical metrics, and go through some case studies and industry practices.
After a long evolution, the browser has become a programmable client that lives in a globally connected world of APIs. This combination of a ubiquitous client with a sea of serverless APIs and the emergence of APIs with advanced security features have enabled the new, client-serverless application model. In such a model, we slowly move away from three-tier applications. In three-tier applications, APIs were typically guarded by the backend. In client-serverless, clients are rapidly taking on a more central role, where clients become responsible for gathering their data services directly from the data source. Needless to say, this reduces complexity, but also brings an entirely different security model which SaaS providers will need to prepare for.
You have successfully stepped on the Kubernetes bandwagon and joined the multitude of companies who think Kubernetes will solve all their problems. But no one told you how. What kind of DevOps processes need to be build on top of Kubernetes to reap the benefits of all that was promised ? How do you do CI/CD, monitoring, On calls, upgrades, maintenance and more. More, so how do you do all this when your Enterprise runs on bare metal and machines go down and never come up for weeks ? You will leave this talk with a recipe for doing DevOps within your Enterprise using Kubernetes. You will become more confident of the exact steps you need to follow to deliver a good experience for your internal developers while still maintaining their trust and zero downtime for their applications. You will also learn about delivering software using staged manner across your data centers. All of this will be shared from a real experience of running Kubernetes clusters at Salesforce.
The networked software systems we build are increasing in complexity every moment. From the abstractions of cloud hosting and inherited libraries to container scheduling and third-party vendors, the turtles go all the way around!
Today the most successful builders and operators are embracing complexity through CI/CD, Chaos Engineering, and innovation in Incident Response. They realize that the adaptive world around us is advancing at such a breakneck speed, it is leaving our capacity to understand it in the dust. That humans and technology must race a gauntlet of automation surprises and collaboration challenges as a team, learning and improving along the way.
This session showcases methods of deploying, running, and navigating complexity. It offers a practical view of how software systems can scale and remain robust to failure (like fallbacks or high-availability), achieve highly reliable socio-technical operations (via runbooks and game-days), and adapt to surprise through techniques of resilience engineering (graceful extensibility and building for adaptation).
Are you ready to say goodbye to your application backlog, and hello to developing apps at lightning speed? Look no further than progressive web apps.
Progressive web apps (PWAs) are web apps that provide end-users with a frictionless experience while still offering the features of the native-like applications they know and love. Some of the key benefits of PWAs include cross-platform, independent distribution, no installation, and offline capabilities.
In this session, Rui Barbosa, Developer Advocate at OutSystems, will demonstrate how to quickly build a mobile application from scratch and immediately distribute it using PWA technology. See how the OutSystems platform can empower you to become a world-class problem solver!
Amazon and AWS have a long history with DevOps, averaging 1 production release a minute. While other companies may not release at that scale, it has become increasingly clear that DevOps is both a unique opportunity and a key challenge in the SDLC. In this session, we will go over DevOps at AWS and how our tooling supports our customer’s journey to DevOps Nirvana.
Provisioning AWS resources one by hand is tedious and cumbersome, especially in an environment where you have multiple similar set ups. Learn how you can use AWS CloudFormation to simplify setting your virtual machine up and how it can be used to even install and configure your software for you.
Through this live coding session, you will learn how to setup a Kotlin Ktor project with some routes using the framework tools and features (Routing, DataConversion, ContentNegotiation...) and communicate with a sample Android app.
Let see how we can easily setup a Postgres database connection with Exposed, an ORM library for Kotlin and see how clean architecture can be a good choice for developping your brand new API.
We can even go further and see that we can actully share code from your backend application and your Android application (say hi to Swagger!).
OPEN TALK (CloudWorld): Performance & Latency Prediction Using AI/ML on Streaming and Eventing PlatformsJoin on Hopin
Streaming and Eventing platforms like RabbitMQ, Kafka and IBM MQ are key enablers of Messages-Oriented-Middleware & Event Driven Architectures which are foundational to Digital Transformation.
Performance management for interactions with these platforms is always challenging.
Latency could have wide ranging impact on business
Creating Machine Learning based models to predict latency related anomalies would help avoid business outages before they happen.
Serverless cloud technologies have been around for some time now and most of us know the benefits include simpler management and pay-per-use billing. There are additional benefits on top of these, however, adoption of this technology is not as straightforward as other cloud transition strategies. In this talk, we explore the advantages of serverless technologies, highlight the architectures that support them, and discuss the challenges of adoption.
As we go through these points you will hear from an organization that has embraced Serverless technology and are well on their journey towards full adoption. You will hear about the reasons they selected serverless, the challenges they faced while adopting the serverless mindset, and how they overcome these challenges.