Real User Measurement Expert Panel by SOASTA

1© 2012 SOASTA. All rights reserved.

RUM Roundtable

Webinar

Presents

Real User Measurement Past, Present & Future

2© 2012 SOASTA. All rights reserved. October 30, 2012

Real user measurement – then and now…and….In This Webinar

TODAY’S PRESENTERS

Aaron Kulick: – Speed Freak @WalmartLabs - @GoFastWebBuddy Brewer: VP Engineering, SOASTA - @bbrewerPhillip Tellis: Chief Architect, SOASTA - @bluesmoonCliff Crocker: VP Product Management, SOASTA mPulse -

@cliffcrocker

Agenda: • Roundtable discussion• Moderated QA with Panelists

Questions: Submit in the question box during event

• VP Product SOASTA

• Real User Measurement (mPulse)

• @cliffcrocker

Cliff Crocker

• VP Engineering SOASTA

• Co-founder of LogNormal

• @bbrewer

Buddy Brewer

• Chief Architect SOASTA

• Co-founder of LogNormal

• @bluesmoon

Philip Tellis

• Guerilla Engineer @WalmartLabs

• Founder SF Web Perf Meetup and Web Perf Days

• @GoFastWeb

Aaron Kulick


2001 – 2005 (Five Years)This monitoring stuff is easy

Internet Explorer 6


2006 – 2009Browser diversity, the mobile web catches on

Internet Explorer 7, 8

Firefox 2, 3

iPhone 1st Gen, 3G

Chrome 1, 2, 3


2010 – 2012This isn’t going to scale

Internet Explorer 9

Chrome 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17,18, 19

Firefox 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16

Android Cupcake, Donut, Éclair, Froyo, Gingerbread, Honeycomb, Jelly Bean

iPhone 3GS, 4, 4S, 5

iPad 1, 2, 3, 4, Mini


• When we speak about performance, we're really talking about latency

• Humans have measured latency for as long as we've known large distances

Measuring Performance


• “How long does it take to get a package from NY to SF?”

• “How fast can a man run 100 metres?”

• “What's the fastest laptime at the Indy 500?”



In general, fast things excite us and slow things frustrate us.



So what should we measure, and how should we measure it?



• The traditional way is to use robots

• This is good to tell us about infrastructure - but what we really care about is user experience



That's akin to asking a Robot to drive Route 66 and tell you if it wasexciting or boring

… and you have to do this with 374 different cars!



So let's bring out the RUM!



• Measure the real experience of a real user

• How fast was their experience?• Were they engaged, did they take out their credit cards, do

they stick around?

Real User Measurement


A Brief History of RUM …

Real User Measurement


At a Yahoo! internal hack day, Steve Souders built aJavaScript tool to measure page load time within the browser.

RUM Timeline 2005 – 2006


RUM Timeline 2006 – 2010

Several Improvements were made: • Bandwidth measurements added• Creation of Yslow• 100s of billions of datapoints were analysed (a tiny sample of

Yahoo!’s traffic at the time)


RUM Timeline 2008

Steve Souders open sources Episodes• http://stevesouders.com/episodes/paper.php

WhitePages.com published jiffy• http://code.google.com/p/jiffy-web/

http://stevesouders.com/episodes/paper.php

http://stevesouders.com/episodes/paper.php

http://code.google.com/p/jiffy-web/


RUM Timeline 2010

Yahoo! open sourced the JavaScript library, calledboomerang, under the BSD license• http://lognormal.github.com/boomerang/doc/

At the same time (on the same day actually), Internet Explorer 9beta announced support for the Navigation Timing API.• http://www.w3.org/TR/navigation-timing/

http://lognormal.github.com/boomerang/doc/



http://www.w3.org/TR/navigation-timing/


Navigation Timing API

Let's talk a bit about NavTiming

• Exposes network level timing details to JavaScript

• DNS lookup, TCP connect, SSL handshake, First Byte time, and more

• For a given web request, you can tell exactly where the time was spent

• Supported by IE >= 9, Chrome >=6, Firefox >=7

• No exposure into rendering times


Navigation Timing API

The level of detail available in NavTiming made RUM performance data comparable to what you'd get from a bot.

boomerang integrates well with it, and works around browser bugs when needed


RUM Timeline 2011

Buddy and I started LogNormal to pull in data from boomerang,analyze it, and present it back to site owners in a useful way.


RUM Timeline 2012

In 2012, the area has exploded with many new companies getting in tothe playing field

LogNormal is now part of SOASTA and the product is called mPulse


RUM Timeline 2012

At SOASTA, we'd like mPulse to look deeper into the human aspect ofweb and mobile performance


RUM at Walmart

Operational RUM

Real User Measurement and the ‘Tribe’

http://sfwebperf.org@SFWebPerf

http://webperfdays.org@WebPerfDays

http://sfwebperf.org/

http://sfwebperf.org/


Panelist Question

Thinking beyond the current horizon, where do you see real user measurement over the next 10 years?

Listen to Your UsersSource: Flickr – jamescridland

Q&A


Thank You!

Contact SOASTA:[email protected] us:

twitter.com/cloudtest

facebook.com/cloudtest

Next Webinar: Nov. 15, 2010 - 10 a.m. PST“Automated Testing & Continuous Integration in the

Cloud”* Kohsuke Kawaguchi – Jenkins CI creator & SOASTA TouchTest mobile CI

demo *

Register at www.soasta.com/knowledge-center/webinars

Contact Us: [email protected]

Real User Measurement Expert Panel by SOASTA

Documents

Transcript of Real User Measurement Expert Panel by SOASTA