Webinar: How to Load Test for Your Mobile Peak
-
Upload
jennifer-finney -
Category
Technology
-
view
155 -
download
2
Transcript of Webinar: How to Load Test for Your Mobile Peak
©2 016 S OA S TA , A l l r i gh t s r es e rv ed .
Tom Chavez @tomchavezMike Ostenberg @mikeostenberg
How to Load Test for your Mobile Peak
©2 016 S OA S TA , A l l r i gh t s r es e rv ed .
• How important is Mobile?• Mobile Web and Apps• The need for load testing• Measure everything• What to test?• Test early and often – shift left and right• How to test?
Importanceof Mobile
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 4
Mobile is growing!
• In 2015, mobile commerce
accounted for $115B worldwide
• In 2016, mobile commerce
is project to reach $142B
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 5
How is your site now?
• mCommerce is globalo US is not the benchmark
• 39% of mobile users
are unhappy with their online experienceso Top reasons: slow pages and site freezes/crashes
• 57% have experienced problems when using mobile
• 3 out of 4 consider mobile slower than desktop
• 46% will not return to a site that performs poorly
• 27% of smartphone users exceed data cap every month
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 6
When is your Peak Season?
• Peak is year-roundo Super Bowl Sunday – 110 million viewers
o Bi-weekly or monthly payroll processing
o Valentine’s Day, President’s Day, Mother’s Day, Father’s Day
o Quarterly processing and filing
o U.S. Tax filing
o Spring, Summer, Fall & Winter season sales
o July 4th, Christmas in July
o Back to School
o Olympics
o Halloween We are here
o Election Day
o Singles Day – world’s biggest shopping day, $9B in 12 hours
o Black Friday, Cyber Monday and Holidays
Back to top to repeat again next year!
Mobile Weband Mobile Apps
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 8
Mobile Web
Responsive web site WebSite
• Desktop Users
• Mobile Users
• Same or similar contento Adapts to different
screen sizes
Mobile optimized
• “m-dot” site for mobile
• Custom built for mobile devices
All have html, images, JavaScript, css, plus third-party content
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 9
Native App
• Android – Java
• iOS – Objective C/Swift
• Mobile app frameworko Same source code
Hybrid App
• Mix of native code + html
All have backend calls for login/authentication, account
settings, database calls, transactions, plus third-party SDK
calls
Mobile Apps
WebSite/MobileBackend
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 10
The Need for Load Testing
• Everyone runs load testing on their website, right?o Load and Performance testing is often not done or too late
o Black Friday, Cyber Monday, Holiday failures
• What’s the need for mobile web load testing?o Mobile is growing faster than desktop
o Responsive sites and mobile sites deliver different content to
mobile devices over mobile network
• What about mobile app load testing?o Mobile app back ends serve different content to mobile apps
o Remember Pokemon Go? It wasn’t the app that failed.
Measure Everything
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 12
Measure Real and Virtual Users
• Real Users (RUM) provide deep data about your visitorso No other way to know everything about your users on your site
and in your app
SOASTA mPulse collects and analyzes RUM data from your web
and mobile app users
• Virtual Users tell you how your site or app will do at scaleo You can’t get a million friends to visit your site or app at the same
time!
SOASTA CloudTest lets you easily build and execute load tests at
scales from hundreds to millions of virtual users
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 13
Other Things to Measure with RUM
• Session pathso How are your users going through your site
• Mobile App Performanceo How are your mobile apps performing?
o What are users doing in your mobile apps?
• Revenueo How much money are you making from each user and where?
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 14
Average Performance
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 15
Performance by Device
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 16
Creating a Test Plan – Peak Load
What was the busiest Day, Hour, Minute, Second?
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 17
Creating a Web Test Plan – Page Flows
What were most popular page flows with think times? Which APIs?
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 18
Network Emulation
• Testing to match realistic situationso Users are not all from one location on a fast network
o Packets get delayed, lost, re-ordered
• Apps perform differently as network conditions worseno Reduced throughput
o Timeouts occur, connections are lost
o Transactions take longer, longer sessions
Adding real network modeling to load and performance testing
Test Earlyand Often
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 20
Don’t Wait Til the Last Minute to Test!
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 21
Test Left and Test Right, even into Production
• Performance Testing in developmento Set performance SLAs for new code and test in development,
same as unit and functional tests which code must pass
• Performance Testing in stagingo Higher loads in near-production systems
• Performance Testing in Productiono Full scale load tests to expected load and beyond
o Find weak and stress points and prioritize fixes into project
Performance Testing is Continuous
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 22
Test in Production
Some issues will only be found at scale load on production systems
Don’t forget your third party providers: CDNs, external scripts
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 23
Measure Mobile App Performance Under Load
All mobile apps perform well when there is no load
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 24
Automate Your Testing
Use Continuous Integration tools to automate testing
Architectureand Demo
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 26
Now That the Test is Over
• Analyze results for future areas to addresso Weak or stress areas that aren’t likely this time but …
• Identify third parties that were affected by loado Coordinate with suppliers to address issues
o Align schedules or at least be aware of their changes
Even if the test passed, you’re not done
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 27
Issues Will Happen
• “I am 100% confident that everything we tested will work
just fine.” – Nordstrom Performance Leadero When loads are different than what you modeled, may have
problems
o E.g. At Nordstrom, during a recent anniversary sale, mobile
sign-ins were 9x greater than anticipated
o Used RUM to get to the root cause of the issue
Realize that Issues Will Happen Where You Can’t Predict Them
Q & AResourcesNext Steps
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 29
CloudTest – Load Testing for mobile web, apps
• Load testing from development to productiono Continuous testing at any scale, on-prem or in the cloud
• Real-time analytics with customizable dashboardso Safely test in production with full visibility
• Run tests live or automaticallyo Full hands-on control or hands-off automation
• Full end-to-end viewo Front-end performance to back-end systems
Fast, scalable, affordable load testing for performance optimization
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 30
SOASTA Digital Performance Management Platform
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 31
Questions, Resources & Next Steps
• Holiday Readinesso soasta.com/holiday-readiness
• The Performance Beacono soasta.com/blogo soasta.com/blog/2015-popular-posts-web-performance
• Webinarso soasta.com/webinars
o Load Testing for Peak; RUM-Based Testing
• CloudTest on-Demando soasta.com/cloudtest: SOASTA will build & run tests for you
Resources
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 32
“The leader in performance analytics.”
www.soasta.com
SOASTA