Automated Performance Testing for Desktop Applications by Ciprian Balea

2

THE SCARY QUESTION OF APPLICATION DEVELOPMENT

© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

How does it perform in real conditions?Never as first expected!

3

BENCHMARKING• What impact does the app have on the system running it? (is the

system still reliable)• How does the app perform when running on the host system?

(does it keep all functionality)


Ideally,• consuming as little resources as possible without loosing

functionality• the app should not break /slow the system down

4

HOW TO PREDICT THE FUTURE?


Run it multiple times and average out.

A single run doesn’t accurately predict real life usage.

5

HOW TO GET A CREDIBLE RESULT?


Avoid inconsistencies induced by the outside world => no Internet connection, own

webserver & assets

6

WHAT’S OUR APPLICATION ABOUT?


A Custom Web Browser

7

HARDWARE SETUP


8

BIG PROBLEM #1: HOW TO KNOW HOW MUCH TIME THE PAGE TOOK TO LOAD


Based on the HTML content, loading time cannot be computed (element loading order is not consistent across

multiple runs).

Small solution:

9

BIG PROBLEM #2: HOW TO READ SYSTEM RESOURCE USAGE WITHOUT 3RD PARTY TOOLS


Small solution (a):

10

BIG PROBLEM #2: HOW TO READ SYSTEM RESOURCE USAGE WITHOUT 3RD PARTY TOOLS


Small solution (b):

11

BIG PROBLEM #3: HOW TO MANIPULATE THE APPLICATION’S GUI


Small solution:

12

WHAT DATA WE COLLECT?• Test 1

• Webpage load time• CPU usage• Memory usage• Disk I/O

• Test 2• Timing a “complex” operation


13

TEST 1: READING SYSTEM

STATISTICS


14

HOW IS THE DATA COLLECTED?


Not at the start of the game

Not at the end of the game

Throughout the entire

game

15

HOW IS THE DATA COLLECTED?


16

RUNNING THE TEST• One test sequence:

• clear cache• open the browser• start reading system data• load webpage• close the browser• save result in an Excel spreadsheet

• Full test:• Restore the “clean” snapshot of the test

environment• Automatically runs the test sequence for

1000 times on the native browser• Automatically runs the test sequence for

1000 times on the custom browser


17

RUNNING THE TEST


18

TEST 2: TIMING A

“COMPLEX” OPERATION


19

“COMPLEX” OPERATION


Open the browser

Load a webpage

Bookmark page

Go to homepage

Open bookmark

Wait for page to load

20

RESULTS


21

RESULTS


22

CONCLUSIONSPerformance testing is very important as it can reveal performance flaws.

It is very time and effort consuming to do it manually.

It can be automated with some initial effort, but the returned value is huge.

Offers very conclusive results on what needs to be improved in the application to boost performance.


23

Q&A


Automated Performance Testing for Desktop Applications by Ciprian Balea

Business

Transcript of Automated Performance Testing for Desktop Applications by Ciprian Balea