AtlasCamp 2013: Show Me Number! Automated Browser Performance Testing

#atlascamp@atlassian

Automated Browser Performance Testing

Chris Mountford Senior Software Developer, JIRA

Atlassian@chromosundrift

Show Me the Numbers

2

Chris Mountford

3

20053.4.118516660

Chris Mountford

4

Chris Mountford

4

Inline Edit

Chris Mountford

4

Inline Edit

Search

Chris Mountford

4

Inline Edit

Search

Detail View

Browser Performance Testing

Welcome!

”“


Measuring the perceived performance of specific interactions for a single user

6

”“


Measuring the perceived performance of specific interactions for a single user

7

perceived

• Intro• JavaScript Renaissance

Engineering• Performance Factors• JIRA Perf Tests

• Optimisation Techniques• Perception• Gotchas• Take Aways

Outline

8

JavaScript Timeline

A history lesson

10

Classical JavaScript

1995

10

Browser Wars

10

Browser Wars Dark Ages

10

Dark Ages Firebug

10

Firebug

V8

10

Firebug

V8 Node.js

10

Node.js

Hipsters

13

“Why do we split everything into so many files?”

13

“Why do we split everything into so many files?”

“Performance.”

14

“Why do we merge all the files into large batches?”

14

“Why do we merge all the files into large batches?”

“Performance.”

15

“Why...”

15

“Why...”

“Performance!”

Show me the numbers!

– Not Jacky Chan

“ ”

16

Hey it’s Jacky

If we have data, let’s look at data. If all we have are opinions, let’s go with mine.

– Jim Barksdale

“”

17

Netscape CEO

Factors That Influence Browser Performance

sad panda

• Everything is the same but worse• network / radio power mode steps, sleep, etc.

• cpu / battery impact

• memory

• standards compliance

• tooling

Mobile Web

19

20

NetworkTime

21

ParseTime

• JS Engines• native bind(), map() etc

• DOM Manipulation• events

• JQuery• Sizzle selectors

• micro-optimisation ☠

• Memory leaks

JavaScript Execution

22

23

Blocking

23

Blocking

<script src=”wtf.js”>

Top Tools

24

Chrome DevToolsFirebug Lite dynaTrace

Reflow

27inspired by @stoyanstefanov

27

Will It Reflow?inspired by @stoyanstefanov

27

Will It Reflow?

Yes!

inspired by @stoyanstefanov

• CSS Transforms• Hardware acceleration?

• Data weight• Concurrent request count• Inline Base64 Encoded resources

But wait, there’s more!

28

Designing the JIRA Browser Performance

Test SuiteSweet!

JIRA Browser Perf Tests

30


30

Page Objects


30

Page Objects

Soke Framework


30

Page Objects

Soke Framework Bamboo


30

Page Objects


Plugin


30

Page Objects


DB

Plugin


30

Page Objects


Hacking

DB

Plugin


30

Page Objects


Hacking

Koto Charts

DB

Plugin


30

Page Objects


Hacking

Koto Charts

Psycho Events

DB

Plugin


30

Page Objects


Hacking

Koto Charts

Psycho Events

OnDemandAnalytics

DB

Plugin


30

Page Objects


Hacking

Koto Charts

Psycho Events

Baselines

OnDemandAnalytics

DB

Plugin


30

Page Objects


Hacking

Koto Charts

Psycho Events

Baselines

OnDemandAnalytics

Vigilance

DB

Plugin

• Define Timed Sections• Number of Repetitions• Personas

Soke Interactions

31

• Define Timed Sections• Number of Repetitions• Personas

Soke Interactions

31

Big ups to Atlassian PerfEng Team

Soke Interactions in Scala

32

• Batching• Atlassian Plugins, No Worries

• Predictive Fetching• Guided by Analytics

• Caching• Watch for memory leaks

Optimisation Techniques

35

I don’t know... it just feels slow.

– a JIRA Product Manager

“ ”

Show me the numbers!

36

Perception is Reality

No, it only seems that way

• Ready for Action• When does the User believe the app is waiting for her?

• Manual “psycho” event placement• and maintenance!

• Yardstick tuning against a real system• Optimisation Example: Execution Reordering

Psycho Latency

39

Gotchas

I’m gonna

click( )

Selenium / Webdriver

42

• Chromedriver implemented by Chromium team


42

• Chromedriver implemented by Chromium team• mouse click on geometric centre of target


42


• W3C Webdriver wire protocol draft


42


• W3C Webdriver wire protocol draft • OK Browsers, follow the standard!


42


43

• Windows and IE process control


43

• Windows and IE process control• Browser auto-upgrades


43

• Windows and IE process control• Browser auto-upgrades• Changing what you measure


43

• Windows and IE process control• Browser auto-upgrades• Changing what you measure

• Prefetch cache benefit in JIRA 6.0


43

• Lots of factors• Measure perceived reality• Be holistic, quantitative, beware micro-optimisation trap• Monitor and Maintain your performance suite• Use Chrome DevTools etc.

Take Aways

44

Thank you!

For resources:#atlascamp

@chromosundrift

AtlasCamp 2013: Show Me Number! Automated Browser Performance Testing

Technology

Transcript of AtlasCamp 2013: Show Me Number! Automated Browser Performance Testing