Mobile apps that win

1

Mobile Apps that WinAchieving High Performance with Real-time Analytics

Steve Fox Foglight Mobile APM Product Manager

2

Response Times: 3 Important Limits

• 100 ms – Limit user will feel as instantaneously• 1.0 sec – Limit to not interrupt user flow of thought• 10 sec – Limit of keeping the user’s attention on the task

• “Usability Engineering”, Jakob Nielsen, 1993• http://www.nngroup.com/articles/response-times-3-important-limits/

Application Performance Monitoring

3

Web Response Times Studies

• Google: +500 ms page load time = 25% fewer searches• Amazon: +100 ms of page load time = 1% lost sales• Facebook: +500 ms of page load time = 3% less traffic


4

Walmart Conversion Rate Analysis


• Multiple Retailer Experiment: • 1 second delay = 7% drop in conversion

Source: http://minus.com/msM8y8nyh#2e

5

Mobile Expectations Compared to Web

• Best Smartphones compared to average desktop/laptop• Less CPU, Memory• Smaller Screen Size• Often used on slower network

• 85% of mobile users surveyed expect experience on mobile device to be “as good or better than online via a computer”

http://portal.sliderocket.com/AHJJM/Tealeaf-Mobile-Transactions-Survey

• Frame Rates: 30 fps?• Less than 50-60 fps and users report as “jerky”• Early Android scrolling was 30 fps, now 60 fps


6

Measuring Mobile Frustration

• EEG Study of tasks using iPhone with variable delays introduced

• 500 ms delay resulted in• 26% increase in frustration• 8% decrease in engagement

• http://www.webperformancetoday.com/2013/12/11/slower-web-pages-user-frustration/


7

Responses to Mobile Frustration

• http://www.webperformancetoday.com/2011/10/21/phone-rage-how-people-react-to-slow-mobile-sites/


8

Speed Matters on Mobile


9

Mobile App Launch!

• Code complete!• QA Passed!• App Store Approval!• User Reviews!!

App is Unusable! by rsl345

Cool idea, but way too slow to use. Save yourself the frustration and use … instead.

• Followed by 30 more• Phone rings• What do you do?


10

Launch Post-Mortem

• Review the QA tests• Every performance test passed• What tests were done?

• 25 comprehensive tests on 5 devices• Using what network?

• Wifi in lab

• What about our beta testing?• 10 employees using their iPhone 5/6 on LTE in

the Bay Area• What’s different about the real user usage?• More often than not, it’s the network


11

But 4G networks are supposed to fix this, right?


-Max Firtman, Velocity, Extreme Mobile Performance

12

But 4G networks are supposed to fix this, right?


-Max Firtman, Velocity, Extreme Mobile Performance

13

Even on 4G, Latency is still a big issue


14

Is the Answer More QA Performance Testing?

• Yes, and No• Effective Mobile Performance

Testing is Hard & Expensive• Permutations grow

exponentially• Device Types• Network Type• Signal Strength• Carrier• Etc

• Reality is your app users are going to be exercising many untested permutations

• So you need to monitor real end user performance


15

How Get Real End User Performance

• Have the app measure performance• Add some timing code around network requests• Time-intensive, so try to wrap NSURLConnection class• Still time-intensive, and realize library uses AFNetworking• Once data is collected, where do I send it, when?• I have millions of users, how do I store it all?• How do I make sense of the data?


16

Mobile Application Performance Management

• Use a solution designed to solve this problem: Mobile APM• Link MobileAPM library into your app• Only code change required is couple lines of initialization• Less than 5 minutes to add instrumentation• What it does:

• Automatically times all network requests• Records which ViewControllers were active

during which network requests• Caches and sends results to cloud collection

point in the background


17

Dimensions Automatically Tracked for Each Network Request

• Network Type• Signal Strength• Location• Carrier• URL• Start time• Completion Time• Response Time• + Other metrics


18

Login to Your Account for Detailed Analytics


19 Application Performance Monitoring

20

App Screen Grouping Paradigm

• One app often makes many http requests• Often one user action or one screen load requires multiple

http requests• Closer to user experience• Auto-Detect App Screen changes and group http requests by

App Screen


22

Advanced Options

• Crash detection and reporting• Exclude URL’s from tracking• Customize your App Screen start/stop measurement points• Add app specific timings• Record errors / messages delivered to user• Record handled exceptions


23

Is Back-End the Problem?


24

Follow Http(s) Requests through back-end

• If it is a back-end issue causing performance problem, you want to be able to easily go from user perspective to back end execution details for that user’s request


26

But What about Android?

• Android Supported Too• For Android 2.3+, automatically detects any requests made

through• java.net.HttpURLConnection• javax.net.ssl.HttpsURLConnection• android.net.AndroidHttpClient• org.apache.http.impl.client.DefaultHttpClient

• For iOS 7+, automatically detects any requests made through• Foundation Framework• AFNetworking 2.0


27

Mobile App Performance Monitoring

• Deep visibility into performance of app for real users


28

Mobile Performance Monitoring Options / Trade-Offs

Method Pros Cons

User Reviews No additional work Business Impact

More QA Perf Testing Catch more issues Expensive, incomplete

Build Own Mobile APM

Customize collection and analysis

Expensive, large effort, compete with feature development

Use 3rd Party Mobile APM

Small quick effortHigh VisibilityPre-built analytics

Some additional operational costs / SaaS fees


29

App Performance Design Tips

• Design with the network in mind, especially latency• Asynchronous requests• Cache• Pre-fetch content likely needed soon• Make fewer larger requests vs many smaller ones• Be cognizant of 3rd party requests / dependencies• Be network aware, degrade gracefully on slower networks

and/or poor signal strength• Revert to lower resolution images• Limit non-essential content


30

App Performance Design Tips

• Perception is Reality• Prioritize content to paint screen first, even if partial• Give some feedback in 100 ms time limit• Spinners?

• Can actually increase perceived time!• Use judiciously, and do not display immediately

• Use animations to buy you more time and provide feedback


http://blog.teamtreehouse.com/perceived-performance

31

Use Optimistic Actions When Possible

• Assume interaction with server will be successful• Update UI for instantaneous feedback• Examples• Like in Facebook or Instagram• Polar


http://www.lukew.com/ff/entry.asp?1759

http://www.mobify.com/blog/beginners-guide-to-perceived-performance/

Q&A

Steve [email protected]@MrSteveFox

Sign up for Free Trial! www.foglight.com

Mobile apps that win

Mobile

Transcript of Mobile apps that win