Ein Release alle 2 Wochen, ein Deployment alle paar Minuten

Lessons Learned aus einer Digital Transformation

Roman Spitzbart

EMEA Solution Director

Dynatrace

confidential

Dynatrace - Our global enterprise app performance expertise

Fortune 100

8,000+ customers across 6 continents

72/100

Cloud providers

9/10

Banks

8/10

Retailers

8/10

Tech brands

9/10

Manufacturers

9/10

8 years

2x

Market

share

MQLeader

confidential

J2EEClient ServerMainframe

SOA

Microservices

Containers

Container OrchestrationServerlessIoTSelf-Healing

1998 2006 2008 2014 - 2016

Static ApplicationsStatic Infrastructure

Complex ApplicationsStatic Infrastructure

Complex ApplicationsComplex Infrastructure

1st Gen. APM 2nd Gen. APM3nd Gen.

Monitoring Redefined

Cloud

A historical perspective

2 major releases/year

customers deploy & operate on-prem

26 major releases/year

500 prod deployments/daySaaS & Managed

2011 2018

DevOps

Continuous Delivery

is a culture that emphasizes the collaboration of various teams involved in software delivery

is an approach about building, testing and releasing software reliable, faster and more frequent

Utmost goal: minimize cycle time

timefeature cycle time

minimize Users

This is where youcreate value!

feature cycle time time

minimize Users

Utmost goal: minimize cycle time

Deploy faster!!

Failfaster??

Right focus?!

The “War Room”

Facebook – December 2012

Sure glad the hole isn’t at our end.

Make also architects responsible for production

iPhone 6

failing early improves quality

late feedbacksucks

Developer will never do that!

Operator’s job

call our DevOps

NoOps

believe in the mission impossible

6 monthsmajor/minor release+ intermediate fix-packs

+ weeks to months

rollout delay

sprint releases (continuous-delivery)

1h : code to production

confidential

It changed product, quality and culture

releasesper year

production bugs reported by customers

awsinstances

daily deployments

7%

100%26

2

450

5 0

500

2011 2017 2011 2017 2011 2017 2011 2017

confidential

4 Key Initiatives

Enterprise cloud: enable innovation w/ break, shift, re-platform!

Shift-left: engage Dev with earlier & automated feedback

Shift-right: empower Ops to deliver to users with lower risk

Self-heal: transform from traditional-to-zero dashboard NOC

confidential

The foundation and enabler of DevOps: Enterprise cloud

Enterprise cloud: enable innovation w/ break, shift, re-platform!

Shift-left: engage Dev with earlier & automated feedback

Shift-right: empower Ops to deliver to users with lower risk

Self-heal: transform from traditional-to-zero dashboard NOC

confidential

• Where to start?• What to break out?• Dependencies?

• Works as expected?• Users happy?• Does it scale?• Does it perform?

• Works as expected?• Users happy?• How to optimize?• How to automate?

?? ?

?

?

?

??

?

?

?

?

?

?

?

?

?

? ?

??

?

? ? ?

?

?

?

?

From On-Premise Monolith

Via Containerized Services

To Functions & Cloud Services

Enterprise cloud: enable innovation w/ break, shift, re-platform!

confidential

• Where to start?• What to break out?• Dependencies?

• Works as expected?• Users happy?• Does it scale?• Does it perform?

• Works as expected?• Users happy?• How to optimize?• How to automate?

Enterprise cloud: Dynatrace enables break, shift, re-platform!

Dynatrace SmartscapeWHAT to migrate and WHERE to

break the monolith

Dynatrace AIAutomatic End-User, service and infrastructure root-cause analysis

Dynatrace Automation APIAuto rollout to containers & functions,

auto Shift-left and Self-healing

confidential

In minutes:• Auto-dependency mapping of internal & external services• Auto-load detection and baselining• Auto-resource consumption detection

Dynatrace helps to decide WHAT to migrate

confidential

Dynatrace helps to decide WHERE to break the monolith

Tightly coupled! Shall we really distribute/extract?

Dynatrace “virtually” breaks your monolith!

confidential

Dynatrace validates & optimizes scalability

Low Load Peak LoadRegular Load

How does each service scale? Does it correctly fail over??

confidential

Dynatrace validates architecture

Planned Architectural Service Flow

Actual Architectural Service Flow

Identify / optimize architectural patternsRecursive Calls, N+1 Call Pattern, Chatty Interfaces, No Caching Layer …

confidential

Dynatrace automates log analytics

confidential

Shift-Left: Faster & Self-Service Feedback in the Delivery Pipeline

Enterprise cloud: enable innovation w/ break, shift, re-platform!

Shift-left: engage Dev with earlier & automated feedback

Shift-right: empower Ops to deliver to users with lower risk

Self-heal: transform from traditional-to-zero dashboard NOC

confidential

Shift-left: engage Dev with earlier & automated feedback

Injecting speed &

quality: Automatic

gates at test &

performance

• Continuous Performance Validation for daily builds

• Root Cause details automatically pushed to JIRA

• Decisions made to compare, break, or good-to-goThomas Steinmaurer, Dynatrace

confidential

pushDynatraceDeploymentEvent

Pushes Deployment Info to Dynatrace Entities

validateBuildDynatraceWorker

Compares Builds and Approves/Rejects Pipeline

pushDynatraceDeploymentEvent

Pushes Deployment Info to Dynatrace Entities

validateBuildDynatraceWorker

Validates Production and Approves/Rejects Pipeline

handleDynatraceProblemNotification

Executes Auto-Remediating Actions, e.g: Rollback

Build 6

Build 7

Production

Production

Auto-Approve!Auto-Reject!

Auto-Approve!Auto-Reject!

Shift-left: automating quality gates in CI / CD

confidential

Shift-left: Dynatrace provides RUM session replay feedback

Visual Session ReplayOptimize based on what your users really do!

confidential

Shift-Right: Bringing it faster to your customer in a safer way!

Enterprise cloud: enable innovation w/ break, shift, re-platform!

Shift-left: engage Dev with earlier & automated feedback

Shift-right: empower Ops to deliver to users with lower risk

Self-heal: transform from traditional-to-zero dashboard NOC

confidential

docker run –e DT_TAGS=BLUE

dtcli tag srv CartServicev2 GREEN

dtcli evt push host .*demo

version=123 source={code_deploy}

dtcli evt push pg tomcat1

desc=JVMMemIncr hint=+100MB

Dynatrace SmartscapeRelease

AutomationDynatrace Automation

API, CLI, Auto-Detection

Shift-right: empower Ops to deliver to users with lower risk

confidential

Shift-right: empower Ops with more context to react faster

confidential

Self-healing: Smart auto-remediation vs reactive War Rooms

Enterprise cloud: enable innovation w/ break, shift, re-platform!

Shift-left: engage Dev with earlier & automated feedback

Shift-right: empower Ops to deliver to users with lower risk

Self-heal: transform from traditional to zero-dashboard NOCs

confidential

Self-heal: path to NoOps with smart auto-remediation

Auto Mitigate!

1 CPU Exhausted? Add a new service instance!

3 Issue with BLUE only? Switch back to GREEN!

?Escalate at 2AM?

2 High Garbage Collection? Adjust/Revert Memory Settings!

4 Hung threads? Restart Service!

5 Still ongoing? Initiate Rollback!

Escalate? Still ongoing?5

1

2

3

4

Mark Bad Commits

Update Dev Tickets

…

…

Impact Mitigated??

confidential

Self-heal: path to NoOps with smart auto-remediation

confidential

4 Key Initiatives

Enterprise cloud: enable innovation w/ break, shift, re-platform!

Shift-left: engage Dev with earlier & automated feedback

Shift-right: empower Ops to deliver to users with lower risk

Self-heal: transform from traditional-to-zero dashboard NOC

Thank you for your attention