JA - Proposed Few-optical Cycle Laser-driven Particle Accelerate Structure
Accelerate User Driven Innovation [Webinar]
-
Upload
dynatrace -
Category
Technology
-
view
180 -
download
0
Transcript of Accelerate User Driven Innovation [Webinar]
Andreas GrabnerDevOps Activist@grabnerndi
Accelerate User Driven InnovationFebruary 2017
Mark TomlinsonPerformance Sherpa and CEO @mark_on_task
James PulleyPerformance Evangelist and CTONewCOE LLC, @perfbytes
Drag picture to placeholder or click icon to add
Drag picture to placeholder or click icon to add
confidential
User Driven Innovation?
confidential
24 “Features in a Box” Ship the whole box!
Very late feedback
F r u s t r a t i o n !
confidential
Accelerated User Driven Innovation1 “Feature at a Time”
Optimize Before DeployImmediate Customer Feedback
confidential
DevOps: Faster & Efficient Innovation
Smaller Apps, Micro-Services More Deployments
App-, Service- & End-User Feedback LoopsHappy Users
Lower Costs
+
-
Total Number of Users per User Experience
Conversion Rate
Use Case #1: User Feedback Driven Decisions
New Features + Day # 1 of Mkt Push
Overall increase of Users!
Jump in Conversion Rate!
Use Case #1: User Feedback Driven Decisions
Users keep growing
Increase # of “tolerating” users!
Lower Conversion as Day #1
Day #2 of Marketing Campaign
Use Case #1: User Feedback Driven Decisions
Drop in Conversion Rate
Spikes in FRUSTRATED Users!
Hotfix Deployment was rolled out
Use Case #1: User Feedback Driven Decisions
User Experience Back to Normal
Jump in Conversion Rate!
Fix of the Hotfix was rolled out
Use Case #1: User Feedback Driven Decisions
4x $$$ to IaaS
Use Case #2: Resource / Cost Driven Decisions
Use Case #3: User Behavior Driven Decisions
confidential
Use Case: Accelerated User Driven Innovation
in Finance
confidential
Defining Performance Engineering You must model the system for
performance, before any code is written
You must test the model or idea for performance risk
You must understand the big picture to write small components of code
You must collaborate with others: developers and system architects
You must adapt your thinking and your relationships
“If you can impact the [performance of the] code before it is written, then and only then, can you consider your work to be performance engineering. Otherwise, you are just doing testing.”
- Jim Duggan, Gartner
confidential
Purpose-BuiltJobs
Capture-Playback
LoadRunner
LoadRunner
+ others
Open Sourc
e
Integrated COE Suites
Open Sourc
e
Cloud Tools
Open Source
confidential
DEV:DIY Scala Tool
Yourkit, JProfilerLocal log parsing
TEST:SmartBear/SoapUIDIY Selenium FX
Stand-alone JMeter on Laptop
LT01:Manual Setup and
ExecutionDIY Scala Tool
Curl & PerlShared NFS log parsing
PROD:Solaris tools - prstat
Dynatrace 4.2TraverseNagios
AWR & Ignite DB Monitoring
Tools, Teams and Environments: 2012
confidential
DEV:Dynatrace 4.2
Local log parsingJProfiler
Some JMeter + Plugins
TEST:Functional Test
Automation
LT01:Dynatrace 4.2
Local log parsing JMeter + Plugins in the labAutomated prstat captureAutomated Loadosophia
UploadIgnite DB MonitoringIntegrated Scenarios
PROD:Dynatrace 4.2
Solaris tools - prstatTraverseNagiosIgnite
Tools, Teams and Environments: 2013-14
confidential
DEV:Dynatrace 6.3
SplunkJProfiler
Rundeck + JMeter + PluginsReal-time InfluxDB &
GrafanaJMeter + Plugins
TEST:Functional Test
AutomationExploratory Testing
LT01:Dynatrace 6.3
One-off Logs to Splunk Rundeck + JMeter + Plugins
Real-time InfluxDB & Grafana
Ignite DB MonitoringIntegrated Scenarios
Notification to Wiki/Jira/Slack
PROD:Dynatrace 6.3
SplunkSolaris tools - prstat
TraverseNagiosIgnite
Tools, Teams and Environments: 2015-16
confidential
DEV:Self-Service
MeasurementsDynatrace 7
SplunkPTaaS & ECD
K8 & Prometheus App Dynamics
TEST:Functional Test
AutomationExploratory Testing
LT01:Continuous Execution
Dynatrace 7Splunk
PTaaS & ECDK8 & Prometheus
Ignite DB Monitoring
PROD:Dynatrace 7
SplunkSolaris tools - prstat
TraverseNagiosIgnite
Tools, Teams and Environments: 2017+
confidential
Performance Practice(production-like)
Production Performance
(“the Real World”)
Development(very unreal world)
Fast Performance FlowsPROMOTIONAL FLOW
FEEDBACK FLOW
operateinvent
confidential
Performance Practice(production-like)
Production Performance
(“the Real World”)
Development(very unreal world)
Fast Performance Feedback
6
5
4
1
2
3
CI CD
PROMOTIONAL FLOW
FEEDBACK FLOW
unit perf
essential perf
new releasestrategic tests
triage
operateinvent
design
confidential
Make Performance ContinuousIN DEVELOPMENT
Enable Self-Service Measurements – on demand execution of a small testEvery engineer on the team can build, submit and execute a testTest environments may be built on-the-flyConnected to CI system for unattended executions after nightly builds on on-demand pushes
IN LOADTEST Enable Non-stop schedule for
performance test executions Trending test results on measurement
thresholds, compared to baseline Use a split-schedule for high-priority
test execution - nightly continuous execution, self-service during the day
Connected to CD system to automate release feedback
In all environments use common tooling for monitoring, profiling and logging – on by default, automated and accessible to all.
confidential
Use Case: Accelerated User Driven Innovation
at Dynatrace
confidential
2011: APM about to be disrupted!
Migrate from On-Prem to VM, Cloud, Containers and PaaSArchitectures include micro-services, on-demand scaling, self-healing”Cloud Natives“ demand SaaS based solutionsDigital Transformers demand Analytics for Biz, Dev, Ops & SecMany new players on the market
confidential
Challenges to master!
Bridging the gap between ”New Stack“ and “Enterprise Stack“Deploying the same way our customers do: Continuously!Not disrupting current operations and slower moving customersAligning 300+ engineers across 3 different geosSolution: Innovation through Incubation!
2 major releases/yearcustomers deploy & operate on-prem
26 major releases/year170 prod deployments/dayself-service online sales SaaS & Managed
2011 2016
29 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
believe in the mission impossible
6 monthsmajor/minor release+ intermediate fix-packs+ weeks to months rollout delay
sprint releases (continuous-delivery)1h : code to production
NOC lessons learnt
Shift-Left QualityQuality/Performance matters in Dev/Staging as well!Make Dev/CSA/PM dependent from Quality in trunk!
DevOps = start thinking like an Ops before Commit
Shift-Right Metricsenable DEVs defining quality metrics make DEVs to the primary consumers of their metrics
acting asEngineers
Role of Dynatrace DevOps Team
Dynatrace Managed/SaaSOrchestration Layer
Dynatrace Pipeline Visualization
Deployment Timeline
Log Overview using Dynatrace Log APIJIRA Integrations
&Product Managers
confidentialhttps://github.com/Dynatrace/ufo
Raising Awareness of Pipeline Quality
ruxit theme 2014.05.15
Be proud of your feature!DevOps NoOps
Dynatrace Transformation by the numbers
26
170
Releases / Year
Deployments / Day
31000 60hUnit & Int Tests / hour
UI Tests per Build
More Quality
~200 340Code commits / day
Stories per sprint
More Agile
93%Production bugs found by Dev
More Stability 450 99.998%Global EC2
InstancesGlobal Availability
confidential
Use Case: Accelerated User Driven Observations at
eCommerce
confidential
Trends across eCommerce
▪Development taking ownership of all quality issues, except Security
▪ Orchestration and deployment is moving to the public cloud (Jenkins, AWS, CloudAzure)
▪ CI is the norm▪ Serverless computing is becoming second nature▪ Rules based analytics are replacing first tier diagnostics,
moving human performance analysis to tougher issues
confidential
Cautionary Notes
▪Developer Architecture Myopia ▪ Loose Gates on Quality and Performance▪ Developer ownership on quality, but the need for
performance training▪ Broken links between the business and Dev/Ops -
particularly marketing▪ Pattern feedback loop to the code analysis level remains
immature ▪ With rules based analysis, the feeder tier for new
performance engineers is compromised.
confidential
Final Thoughts …
confidential
80%20%organization & culture technology
Confidential, Dynatrace, LLC
Develop
Ship
Deploy
Run
Scale
Compute
nodejs mongo db netty cassandra redis
ansible jenkins puppet chef
docker cloudfoundry rh openshift rh atomic rocket
core os rancher kvm busybox
mesos marathon kubernetes swarm
amazon azure openstack mesosphere calico weave
eureka/hystrix
A whole new technology stack & polyglot development
confidential
Ecosystem …
confidential
Dev/Test: Bad Code Changes
Ops/Biz: Unhappy End Users
Performance: Overloaded COEs
confidential
Dev/Test: Check-In Better Code
Ops/Biz: Use Real-Time End User Feedback
CI/CD/Performance: Stop Bad Builds Faster
confidential
confidential
AI-Driven Digital Operation Ecosystem
DAVIS / REST
Andreas GrabnerDevOps Activist@grabnerndi
Accelerate User Driven InnovationFebruary 2017
Mark TomlinsonPerformance Sherpa and CEO @mark_on_task
James PulleyPerformance Evangelist and CTONewCOE LLC, @perfbytes
Drag picture to placeholder or click icon to add
Drag picture to placeholder or click icon to add