Java Forum Stuttgart 2016 Christoph Heger, …2016/07/07 · FROM APM EXPERTS FOR EVERYONE:...

FROM APM EXPERTS FOR EVERYONE: Application Performance Management With Open Source Tooling

Java Forum Stuttgart 2016 Christoph Heger, Alexander Wert

Why is performance important?

07.07.2016 From APM Experts for Everyone: Application Performance Management With Open Source Tooling 2

Response Time Limit Perception

0.1 second Users feel their actions are directly causing something to happen

1.0 second Users feel they are navigating freely and stay focused on their current train of thought

10.0 seconds Users feel a break of the flow, get impatient while waiting and leave the site

Jacob Nielsen, Power of 10: Time Scales in User Experience, 2009

Business Impact Brand perception, conversions, revenue, shopping card abandonment, page views,

and search engine rankings

User Experience

Reports 49% of online shoppers expect load times of <=2 seconds 50% of online shoppers abandon page if load time is > 2 seconds 42% of men and 35% of women don‘t give a company a second chance after experiencing slow load times

The HealthCare.gov Fiasco

3

Summary [Kevin Surace, fastcompany.com; Byron Wolf, CNN]

Among the worst software launches in history

Hundreds of millions of dollars were spent

Outages and slow response times were the norm

U.S. government had to apologize

Companies like Google and Red Hat rushed

to the rescue

Never actually tested for scalability and performance before its launch

[Image: healthcare.gov]

125k plus concurrent users peak

Load times between 6 and 22 seconds depending on business transaction

07.07.2016 From APM Experts for Everyone: Application Performance Management With Open Source Tooling

Ellen DeGeneres‘ Oscars selfie crashes Twitter

4

[Image: Ellen DeGeneres]

Summary 2.6 million retweets in about 2 hours

20 minutes service disruption

Disruptive change in user behavior

[Image: Twitter.com]


Take Away

5

[Image source: http://content.hollywire.com/sites/default/files/Success-Kid.jpg]


APPLICATION PERFORMANCE MANAGEMENT

Requirements

Design

Development Testing

Operations

Application Performance Management … is a Cross-cutting Concern!

7

Network

App

System

Clients

Development

Operations Business

Application Performance Management …

… spans the entire software lifecycle … covers all system layers … requires collaboration of different business units


APM in the Software Lifecycle

8

Requirements

Design

Development Testing

Operations

Elicit Requirements for Performance

Design for Performance

Create Code for Performance

Test for Performance

Monitor Application


APM accross System Levels

9

Network - Internet -

Server-side Applications

System Resources

Clients

http://www.vidalcasey.com/wp-content/uploads/2014/07/speed-bottleneck.jpg

latencies, bandwidth, CDNs, …

http://www.it4fans.ro/wp-content/uploads/2011/09/PC-burning.jpg

system monitoring, availability,

scaling, … hotspots, bottlenecks,

component communication, request flow, …

page rendering times, resource loading times, ...


APM Requires Collaboration

10

Development

Operations Business

How does performance and user experience affect the business success?

How do business metrics correlate with application performance metrics?

Are there currently any performance / availability issues with the application?

How to handle peak loads and anticipated future loads?

What is the root cause of a performance problem? How to resolve problems?

DevOpsBiz


Requirements

Design

Development Testing

Operations

APM is a Versatile and Complex Topic

11

Network

App

System

Clients

Development

Operations Business

Performance Testing

Web Perf. Testing Performance

Modelling

Root Cause Diagnosis Synthetic

Monitoring

Real User Monitoring

Application Perf. Monitoring

Regression Testing

APM


Commercial APM Tools

12

Leaders Dynatrace AppDynamics New Relic

Challengers CA Technologies IBM HP Enterprise Dell Riverbed Technology BMC Software

Niche Players AppNeta ManageEngine Microsoft Tingyun Nastel Technologies JenniferSoft

Gartner‘s Magic Quadrant for Application

Performance Monitoring Suites (12/2015),

Cameron Haight, Will Cappelli, Federico De Silva

http://www.dynatrace.com/en/gartner-magic -quadrant-application-performance-monitoring -2015.html


Open Source Tools for APM

13

Performance Modeling

Web Performance Analysis

System & Resources Monitoring


JRat JMemProf

Low-Level Performance Profiling

Monitoring & Application Deep Dive

Load Testing


APPLICATION PERFORMANCE MONITORING

Application Performance Monitoring

What is Performance Monitoring About?

15

Estimate Business Impact Monitor Application Health

Goals Identify anomalies and

performance issues

Prod. Environment Real Users

Request

Dashboards Anomaly Detection

Monitoring Tool


Problem Diagnosis

16


Gathering and Visualizing Timeseries Data

Time Series Databases

Graphing Tools Data Collectors

Custom Code …

Persist data Query & Visualize


17


Dashboards


18


Dashboards



The Flaw of Averages

19

The Flaw of Averages: Why We Underestimate Risk in the Face of Uncertainty Sam L. Savage, with illustrations by Jeff Danziger – http://flawofaverages.com


Gil Tene – http://latencytipoftheday.blogspot.de/

http://latencytipoftheday.blogspot.de/

20


Dashboards


21


Anomaly Detection & Alerting



Missing Business Context

22

CPU utilization over time

How critical is this peak in CPU utilization?

Asking the right questions: How are the users affected? Which business transactions are affected? Are any time-critical batch jobs affected?

11pm 0am 1am 2am 3am 4am 5am 6am

100 %

50 %

0%



Defining Business Context

23 07.07.2016 From APM Experts for Everyone: Application Performance Management With Open Source Tooling


Defining Business Context


Hawkular BTM


Problem Diagnosis

25

Deep Dive into request execution flow necessary

Level of Detail vs. Measurement Overhead Do not instrument frequently called methods (e.g. Getters, Setters, String class, etc.) Instrument system and component boundaries (entry and exit points)


Application Performance Testing & Diagnosis

Configuring Instrumentation


APPLICATION PERFORMANCE TESTING & DIAGNOSIS


What is Performance Testing About?

28

Identify & Diagnose Performance Problems

Evaluate Application Behaviour under Load

Goals

Test Environment Monitoring Tool

Test Scalability of Application

load e-2-e response times throughput

detailed invocations system resource data deep dive information

fLoad Generator

Agent

Agent

Agent

Request

Response


29

0

1

2

3

1 2 3 4 5 6 7

Res

pons

e Ti

me

[s]

Execution Nr.

e-2-e response times

load data

detailed measurement

data

System Under Test Load Driver


Regression Tests


WEB PERFORMANCE ANALYSIS

31

End User Experience?

[Image | http://cdn2.hubspot.net/hub/2224117/file-4141070752-jpg/blog-files/three-e1409846588737.jpg]

Satisfied Tolerating Frustrated

Performance User Actions Users

Performance issues Functional Errors Network (Local ISPs, Mobile network carriers) Third party content providers

First user action Last user action Did the visit convert? Did the visit bounce?

Device Resolution Browser Versions Geolocation



What is Web Performance Analysis about?



Measuring End-User Experience

33

Real User Testing

Synthetic Monitoring

Continuous external testing Known testing nodes Different Locations (ISP, Networks) No baseline traffic required Competitor Benchmark Availability testing (incl. 3rd Party) e.g., every 30 mins from different nodes


Real End Users Real Interaction Real traffic

External testing with all major browsers, operating systems, mobile devices and real world data

QoS Validation

QoS Expectation QoS Optimization

Anomaly



Testing and Monitoring Web Performance

34 [http://www.webpagetest.org/performance_optimization.php?test=160623_DF_MV7&run=1&cached=0]



webpagetest.org

35

https://dashboard.sitespeed.io/dashboard/db/compare-multiple-sites



Synthetic Testing

Conclusion

36

Monitoring & Application Deep Dive

Load Testing

Performance Modeling


System & Resources Monitoring


JRat JMemProf

Low-Level Performance Profiling

Open-Source tools sufficiently

address APM needs!

There are lots of them for different purposes!

Combine them!


Java Forum Stuttgart 2016 Christoph Heger, …2016/07/07 · FROM APM EXPERTS FOR EVERYONE:...

Documents

Transcript of Java Forum Stuttgart 2016 Christoph Heger, …2016/07/07 · FROM APM EXPERTS FOR EVERYONE:...