Intro to Perf Test

8/8/2019 Intro to Perf Test

1/31

Agenda

Why Performance Test?

What is Performance testing and How do we doPerformance testing?

Intro to Performance Engineering Methodology

Agile S crum methodology - Testing


2/31

Why P erformance Test?

Speed - Does the applicationrespond quickly enough for theintended users?

Scalabilit y Will the applicationhandle the expected user loadand beyond? (Capacity)

Stabilit y Is the applicationstable under expected and

unexpected user loads?(Robustness)

C onfidence Are you sure thatusers will have a positive

experience on go-live day?


3/31


4/31

Scalabilit y

How many users

before it gets slow ? before it stops working?

will it sustain?

do I expect today? do I expect before the nextupgrade?

How much data can it hold?

Database capacity

File S erver capacity

Back-up S erver capacity

Data growth rates


5/31

Stabilit y

What happens if

there are more users than we expect?

all the users do the same thing? a user gets disconnected?

there is a Denial of S ervice Attack?

the web server goes down?


6/31


7/31

Wh at is P erformance Related Testing?

Performance Validation

Performance Testing

Performance Engineering

Detect

Resolve


8/31

P erformance Validation Performance validation is the process by whichsoftware is tested with the intent of determining if thesoftware meets pre-existing performancerequirements. This process aims to evaluate

compliance.

Primarily used for

determining SL A compliance.

Independent Validation and Verification validating subsequent builds/releases.


9/31

P erformance Testing Performance testing is the process by which software is

tested to determine the current system performance. This

process aims to gather information about current

performance, but places no value judgments on thefindings.

Primarily used for

determining capacity of existing systems. creating benchmarks for future systems.

evaluating degradation with various loads and/or configurations.


10/31

P erformance Engineering Performance engineering is the process by whichsoftware is tested and tuned with the intent of realizing the required

performance. This process aims to optimize the most

important application performance trait, user experience.

Primarily used for

new systems with pre-determined requirements. extending the capacity of old systems.

fixing systems that are not meetingrequirements/ SL As.


11/31


12/31

C ompare and C ontrast

Engineering:

Is iterative.

Has clear goals, but fuzzy end points.

Includes the effort of tuning the application.

Focuses on multiple scenarios with pre-determined

standards.

Heavily involves the development team.

Occurs concurrently with software development.


13/31

Intro to PE M et h odolog y

Evaluate S ystem

Develop Test Assets

Baselines and Benchmarks

Analyze Results

Tune

Identify Exploratory Tests

Execute S cheduled Tests

Complete Engagement


14/31


15/31

Ev aluate S ystem

Determine performance requirements.

Identify expected and unexpected user activity.

Determine test and/or production architecture.

Identify non-user-initiated (batch) processes.

Identify potential user environments.

Define expected behavior during unexpectedcircumstances.


16/31


17/31

D e velop Test Assets

Create S trategy Document.

Develop Risk Mitigation Plan.

Develop Test Data.

Automated test scripts:

Plan

Create

Validate


18/31

B aseline and B enc h marks

Most important for iterative testing.

Baseline (single user) for initial basis of comparison andbest

case .

Benchmark (15-25% of expected user load) determinesactual

state at loads expected to meet requirements.


19/31


20/31

Anal yz e Results

Most important.

Most difficult.

Focuses on:

Have the performance criteria been met?

What are the bottlenecks?

Who is responsible to fix those bottlenecks?

Decisions.


21/31


22/31

Tune

Engineering only.

Highly collaborative with development team.

Highly iterative.

U sually, performance engineer supports andvalidates while

developers/admins tune .


23/31


24/31


25/31


26/31

Ex ecute Sc h eduled Tests

Only after Baseline and/or Benchmark tests.

These tests evaluate compliance with documented

requirements.Often are conducted on multiplehardware/configuration

variations.


27/31


28/31

C omplete Engagement

Document:

Actual Results

Tuning S ummary

Known bottlenecks not tuned

Other supporting information

Recommendation

Package Test Assets:

S cripts

Documents

Test data


29/31


30/31

Summar y

We test performance to: Evaluate Risk.

Determine system capabilities.

Determine compliance.

Performance Engineering Methodology:

Ensures goals are accomplished.

Defines tasks.

Identifies critical decision points.

S hortens testing lifecycle.


31/31

Th ank You!!

Intro to Perf Test

Documents

Transcript of Intro to Perf Test