Post on 06-May-2015
#atlascamp@atlassian
Automated Browser Performance Testing
Chris Mountford Senior Software Developer, JIRA
Atlassian@chromosundrift
Show Me the Numbers
2
Chris Mountford
3
20053.4.118516660
Chris Mountford
4
Chris Mountford
4
Chris Mountford
4
Inline Edit
Chris Mountford
4
Inline Edit
Chris Mountford
4
Inline Edit
Search
Chris Mountford
4
Inline Edit
Search
Chris Mountford
4
Inline Edit
Search
Detail View
Chris Mountford
4
Inline Edit
Search
Detail View
Browser Performance Testing
Welcome!
”“
Browser Performance Testing
Measuring the perceived performance of specific interactions for a single user
6
”“
Browser Performance Testing
Measuring the perceived performance of specific interactions for a single user
7
perceived
• Intro• JavaScript Renaissance
Engineering• Performance Factors• JIRA Perf Tests
• Optimisation Techniques• Perception• Gotchas• Take Aways
Outline
8
• Intro• JavaScript Renaissance
Engineering• Performance Factors• JIRA Perf Tests
• Optimisation Techniques• Perception• Gotchas• Take Aways
Outline
8
JavaScript Timeline
A history lesson
10
Classical JavaScript
1995
10
Browser Wars
10
Browser Wars Dark Ages
10
Dark Ages Firebug
10
Firebug
V8
10
Firebug
V8 Node.js
10
Node.js
Hipsters
11
12
13
13
“Why do we split everything into so many files?”
13
“Why do we split everything into so many files?”
“Performance.”
14
14
“Why do we merge all the files into large batches?”
14
“Why do we merge all the files into large batches?”
“Performance.”
15
15
“Why...”
15
“Why...”
“Performance!”
15
“Why...”
“Performance!”
Show me the numbers!
– Not Jacky Chan
“ ”
16
Hey it’s Jacky
If we have data, let’s look at data. If all we have are opinions, let’s go with mine.
– Jim Barksdale
“”
17
Netscape CEO
Factors That Influence Browser Performance
sad panda
• Everything is the same but worse• network / radio power mode steps, sleep, etc.
• cpu / battery impact
• memory
• standards compliance
• tooling
Mobile Web
19
20
NetworkTime
21
ParseTime
• JS Engines• native bind(), map() etc
• DOM Manipulation• events
• JQuery• Sizzle selectors
• micro-optimisation ☠
• Memory leaks
JavaScript Execution
22
23
Blocking
23
Blocking
<script src=”wtf.js”>
Top Tools
24
Chrome DevToolsFirebug Lite dynaTrace
Reflow
26
27inspired by @stoyanstefanov
27
Will It Reflow?inspired by @stoyanstefanov
27
Will It Reflow?inspired by @stoyanstefanov
27
Will It Reflow?
Yes!
inspired by @stoyanstefanov
• CSS Transforms• Hardware acceleration?
• Data weight• Concurrent request count• Inline Base64 Encoded resources
But wait, there’s more!
28
• CSS Transforms• Hardware acceleration?
• Data weight• Concurrent request count• Inline Base64 Encoded resources
But wait, there’s more!
28
Designing the JIRA Browser Performance
Test SuiteSweet!
JIRA Browser Perf Tests
30
JIRA Browser Perf Tests
30
Page Objects
JIRA Browser Perf Tests
30
Page Objects
Soke Framework
JIRA Browser Perf Tests
30
Page Objects
Soke Framework Bamboo
JIRA Browser Perf Tests
30
Page Objects
Soke Framework Bamboo
Plugin
JIRA Browser Perf Tests
30
Page Objects
Soke Framework Bamboo
DB
Plugin
JIRA Browser Perf Tests
30
Page Objects
Soke Framework Bamboo
Hacking
DB
Plugin
JIRA Browser Perf Tests
30
Page Objects
Soke Framework Bamboo
Hacking
Koto Charts
DB
Plugin
JIRA Browser Perf Tests
30
Page Objects
Soke Framework Bamboo
Hacking
Koto Charts
Psycho Events
DB
Plugin
JIRA Browser Perf Tests
30
Page Objects
Soke Framework Bamboo
Hacking
Koto Charts
Psycho Events
OnDemandAnalytics
DB
Plugin
JIRA Browser Perf Tests
30
Page Objects
Soke Framework Bamboo
Hacking
Koto Charts
Psycho Events
Baselines
OnDemandAnalytics
DB
Plugin
JIRA Browser Perf Tests
30
Page Objects
Soke Framework Bamboo
Hacking
Koto Charts
Psycho Events
Baselines
OnDemandAnalytics
Vigilance
DB
Plugin
• Define Timed Sections• Number of Repetitions• Personas
Soke Interactions
31
• Define Timed Sections• Number of Repetitions• Personas
Soke Interactions
31
Big ups to Atlassian PerfEng Team
Soke Interactions in Scala
32
33
33
34
• Batching• Atlassian Plugins, No Worries
• Predictive Fetching• Guided by Analytics
• Caching• Watch for memory leaks
Optimisation Techniques
35
I don’t know... it just feels slow.
– a JIRA Product Manager
“ ”
Show me the numbers!
36
Perception is Reality
No, it only seems that way
38
• Ready for Action• When does the User believe the app is waiting for her?
• Manual “psycho” event placement• and maintenance!
• Yardstick tuning against a real system• Optimisation Example: Execution Reordering
Psycho Latency
39
Gotchas
I’m gonna
click( )
Selenium / Webdriver
42
• Chromedriver implemented by Chromium team
Selenium / Webdriver
42
• Chromedriver implemented by Chromium team• mouse click on geometric centre of target
Selenium / Webdriver
42
• Chromedriver implemented by Chromium team• mouse click on geometric centre of target
• W3C Webdriver wire protocol draft
Selenium / Webdriver
42
• Chromedriver implemented by Chromium team• mouse click on geometric centre of target
• W3C Webdriver wire protocol draft • OK Browsers, follow the standard!
Selenium / Webdriver
42
• Chromedriver implemented by Chromium team• mouse click on geometric centre of target
• W3C Webdriver wire protocol draft • OK Browsers, follow the standard!
Selenium / Webdriver
42
Selenium / Webdriver
43
• Windows and IE process control
Selenium / Webdriver
43
• Windows and IE process control• Browser auto-upgrades
Selenium / Webdriver
43
• Windows and IE process control• Browser auto-upgrades• Changing what you measure
Selenium / Webdriver
43
• Windows and IE process control• Browser auto-upgrades• Changing what you measure
• Prefetch cache benefit in JIRA 6.0
Selenium / Webdriver
43
• Lots of factors• Measure perceived reality• Be holistic, quantitative, beware micro-optimisation trap• Monitor and Maintain your performance suite• Use Chrome DevTools etc.
Take Aways
44
Thank you!
For resources:#atlascamp
@chromosundrift