Post on 28-Jul-2015
1
Mobile Apps that WinAchieving High Performance with Real-time Analytics
Steve Fox Foglight Mobile APM Product Manager
2
Response Times: 3 Important Limits
• 100 ms – Limit user will feel as instantaneously• 1.0 sec – Limit to not interrupt user flow of thought• 10 sec – Limit of keeping the user’s attention on the task
• “Usability Engineering”, Jakob Nielsen, 1993• http://www.nngroup.com/articles/response-times-3-important-limits/
Application Performance Monitoring
3
Web Response Times Studies
• Google: +500 ms page load time = 25% fewer searches• Amazon: +100 ms of page load time = 1% lost sales• Facebook: +500 ms of page load time = 3% less traffic
Application Performance Monitoring
4
Walmart Conversion Rate Analysis
Application Performance Monitoring
• Multiple Retailer Experiment: • 1 second delay = 7% drop in conversion
Source: http://minus.com/msM8y8nyh#2e
5
Mobile Expectations Compared to Web
• Best Smartphones compared to average desktop/laptop• Less CPU, Memory• Smaller Screen Size• Often used on slower network
• 85% of mobile users surveyed expect experience on mobile device to be “as good or better than online via a computer”
http://portal.sliderocket.com/AHJJM/Tealeaf-Mobile-Transactions-Survey
• Frame Rates: 30 fps?• Less than 50-60 fps and users report as “jerky”• Early Android scrolling was 30 fps, now 60 fps
Application Performance Monitoring
6
Measuring Mobile Frustration
• EEG Study of tasks using iPhone with variable delays introduced
• 500 ms delay resulted in• 26% increase in frustration• 8% decrease in engagement
• http://www.webperformancetoday.com/2013/12/11/slower-web-pages-user-frustration/
Application Performance Monitoring
7
Responses to Mobile Frustration
• http://www.webperformancetoday.com/2011/10/21/phone-rage-how-people-react-to-slow-mobile-sites/
Application Performance Monitoring
9
Mobile App Launch!
• Code complete!• QA Passed!• App Store Approval!• User Reviews!!
App is Unusable! by rsl345
Cool idea, but way too slow to use. Save yourself the frustration and use … instead.
• Followed by 30 more• Phone rings• What do you do?
Application Performance Monitoring
10
Launch Post-Mortem
• Review the QA tests• Every performance test passed• What tests were done?
• 25 comprehensive tests on 5 devices• Using what network?
• Wifi in lab
• What about our beta testing?• 10 employees using their iPhone 5/6 on LTE in
the Bay Area• What’s different about the real user usage?• More often than not, it’s the network
Application Performance Monitoring
11
But 4G networks are supposed to fix this, right?
Application Performance Monitoring
-Max Firtman, Velocity, Extreme Mobile Performance
12
But 4G networks are supposed to fix this, right?
Application Performance Monitoring
-Max Firtman, Velocity, Extreme Mobile Performance
14
Is the Answer More QA Performance Testing?
• Yes, and No• Effective Mobile Performance
Testing is Hard & Expensive• Permutations grow
exponentially• Device Types• Network Type• Signal Strength• Carrier• Etc
• Reality is your app users are going to be exercising many untested permutations
• So you need to monitor real end user performance
Application Performance Monitoring
15
How Get Real End User Performance
• Have the app measure performance• Add some timing code around network requests• Time-intensive, so try to wrap NSURLConnection class• Still time-intensive, and realize library uses AFNetworking• Once data is collected, where do I send it, when?• I have millions of users, how do I store it all?• How do I make sense of the data?
Application Performance Monitoring
16
Mobile Application Performance Management
• Use a solution designed to solve this problem: Mobile APM• Link MobileAPM library into your app• Only code change required is couple lines of initialization• Less than 5 minutes to add instrumentation• What it does:
• Automatically times all network requests• Records which ViewControllers were active
during which network requests• Caches and sends results to cloud collection
point in the background
Application Performance Monitoring
17
Dimensions Automatically Tracked for Each Network Request
• Network Type• Signal Strength• Location• Carrier• URL• Start time• Completion Time• Response Time• + Other metrics
Application Performance Monitoring
20
App Screen Grouping Paradigm
• One app often makes many http requests• Often one user action or one screen load requires multiple
http requests• Closer to user experience• Auto-Detect App Screen changes and group http requests by
App Screen
Application Performance Monitoring
22
Advanced Options
• Crash detection and reporting• Exclude URL’s from tracking• Customize your App Screen start/stop measurement points• Add app specific timings• Record errors / messages delivered to user• Record handled exceptions
Application Performance Monitoring
24
Follow Http(s) Requests through back-end
• If it is a back-end issue causing performance problem, you want to be able to easily go from user perspective to back end execution details for that user’s request
Application Performance Monitoring
26
But What about Android?
• Android Supported Too• For Android 2.3+, automatically detects any requests made
through• java.net.HttpURLConnection• javax.net.ssl.HttpsURLConnection• android.net.AndroidHttpClient• org.apache.http.impl.client.DefaultHttpClient
• For iOS 7+, automatically detects any requests made through• Foundation Framework• AFNetworking 2.0
Application Performance Monitoring
27
Mobile App Performance Monitoring
• Deep visibility into performance of app for real users
Application Performance Monitoring
28
Mobile Performance Monitoring Options / Trade-Offs
Method Pros Cons
User Reviews No additional work Business Impact
More QA Perf Testing Catch more issues Expensive, incomplete
Build Own Mobile APM
Customize collection and analysis
Expensive, large effort, compete with feature development
Use 3rd Party Mobile APM
Small quick effortHigh VisibilityPre-built analytics
Some additional operational costs / SaaS fees
Application Performance Monitoring
29
App Performance Design Tips
• Design with the network in mind, especially latency• Asynchronous requests• Cache• Pre-fetch content likely needed soon• Make fewer larger requests vs many smaller ones• Be cognizant of 3rd party requests / dependencies• Be network aware, degrade gracefully on slower networks
and/or poor signal strength• Revert to lower resolution images• Limit non-essential content
Application Performance Monitoring
30
App Performance Design Tips
• Perception is Reality• Prioritize content to paint screen first, even if partial• Give some feedback in 100 ms time limit• Spinners?
• Can actually increase perceived time!• Use judiciously, and do not display immediately
• Use animations to buy you more time and provide feedback
Application Performance Monitoring
http://blog.teamtreehouse.com/perceived-performance
31
Use Optimistic Actions When Possible
• Assume interaction with server will be successful• Update UI for instantaneous feedback• Examples• Like in Facebook or Instagram• Polar
Application Performance Monitoring
http://www.lukew.com/ff/entry.asp?1759
http://www.mobify.com/blog/beginners-guide-to-perceived-performance/