by Jan Verner

18.5.2016 CzechTest 2016

PERFORMANCE TESTING DONE BETTER AND FASTER

“Application performance is an integral part of user experience. Take responsibility for it.”

Agenda

Introductory presentationShort breakWorkshop

Installing the toolPreparing test scriptsCoffee breakTests executionResult analysis

Why do performance testing

What performance tests offerEveryday use casesBenefits

Predict System Behaviour Under Load

To be able to simulate and measure system’s behaviour under load.

Anticipate the future.

Verify Configuration Settings

Super-performing environments can be limited by bad configurations.

Verify and fix configurations.

Discover Concurrency Issues

Even small load with unlucky timing can cause great troubles.

Discover concurrency issues.

Everyday Use Cases

Marketing campaigns (newsletters, TV, AdWords, ..)Events (sales, sports, ..)Trends (mobile, virtualization)

Changes in configurationPerformance tuning

Parallelism errors (multithreading)

Everyday Use Cases

Marketing campaigns (newsletters, TV, AdWords, ..)Events (sales, sports, ..)Trends (mobile, virtualization)

Changes in configurationPerformance tuning

Parallelism errors (multithreading)

Everyday Use Cases

Marketing campaigns (newsletters, TV, AdWords, ..)Events (sales, sports, ..)Trends (mobile, virtualization)

Changes in configurationPerformance tuning

Parallelism errors (multithreading)

Benefits

For customers• Improved user experience (latency, availability)

Benefits

For customers• Improved user experience (latency, availability)

For business owners• Planning (expansion, campaigns)• Keeping SLAs• Managing suppliers

Benefits

For customers• Improved user experience (latency, availability)

For business owners• Planning (expansion, campaigns)• Keeping SLAs• Managing suppliers

For maintainers• Knowing of system limits• Calm nights and weekends

Predict System Behaviour Under LoadVerify Configuration SettingsDiscover Concurrency IssuesEveryday Use CasesBenefits

Q&A

Summary “Why”

How do performance testing

ToolsMethodology

Tool

Record & edit test scenarios

Easy to useAccurateDistributed

Run tests

Analyze results

Create reportsArchive results

Distributed Load

Methodology

Workshop focus

Weak Spots

Infrastructure ConfigurationApplications

Performance test - verify criteriaStress test - discover system limitsSoak test - resource managementFailover test - verify configurationsVolume test - test for future

Finding Weak Spots

Cooperation!Tests result (data) → Test report (information)Need to understand the tested systemLog analysisMonitoring

Pinpointing Weak Spots

ToolsMethodologyWeak SpotsFinding Weak Spots

Q&A

Summary “How”

Workshop Walkthrough

StoryApplying the methodologyAssignment

Story - The Smart University

Assignments for students● questions published at 8pm ● submit answers in 24 hours

Assignment Details

500 students bit.ly/workshop-demo Contact Mr. John Doe

Design and execute performance tests that will verify University’s readiness.

Identify Test Environment

Tested system• URL bit.ly/workshop-demo • Architecture - Single server, Linux based, Apache Tomcat,

MySQL, proprietary CMS built on Java platformTool

• SmartMeter.io (www.smartmeter.io) Test environment

• Your laptop, Wi-Fi connectivity

Identify Acceptance Criteria

Assignment download• Throughput - 200 downloads in one minute• Response time - 90% percentile under 5 secs

Assignment submit• Throughput - 100 submits in one minute• Response time - 90% percentile under 3 secs

Plan and Design Tests

Identify key scenarios• homepage → assignment page → download assignment• homepage → submit page → submit assignment

Determine variability• students names• CSRF token in submit form

Test data• students names (CSV)

Configure Test Environment

Prepare test environmentGet familiar with resource monitoring (CPU, RAM, ..)Install SmartMeter.io tool (requires 64-bit OS)Make sure you can reach the tested system

Implement and Verify Tests

Record test scenariosGather test dataModify test scenarios (variability)Run smoke testsSet up load profilesBe sure you are ready to go!

Execute Tests

Be vigilant!Monitor system resourcesCommunicate in real time with system ownersExecute various test types (stress, soak, ..)

Analyze, Report, and Retest

Gather necessary data (test results, logs, monitoring data, ..)Make notesTransform data into meaningful reportRetest

Identify Acceptance criteriaPrepare test environmentCreate test scriptsExecute testsReporting

Q&A

Summary “Workshop Walkthrough”

Presentation Summary

We learnt “why”Areas, use cases,

benefitsWe learnt “how”

Tool & MethodologyWorkshop Walkthrough

Get Ready for Workshop

● After break ..○ Step by step solution○ Best practices○ Hands on tool and

methodology● Get your laptops ready!

Thank you for your attention!

Share your feedback

bit.ly/performance-tutorial

Jan Verner

Product DeveloperPerformance tester

jan.verner@smartmeter.io

@SmartMeterIO

#SmartMeterIO