Measuring the Mobile Experience

download Measuring the Mobile Experience

of 26

Transcript of Measuring the Mobile Experience

  • 7/29/2019 Measuring the Mobile Experience

    1/26

    Measuring the

    Mobile Sports ExperienceHow to win at the mobile game

  • 7/29/2019 Measuring the Mobile Experience

    2/26

  • 7/29/2019 Measuring the Mobile Experience

    3/26

    How ESPN defines UX

    3

    The customers perception of

    usefulness,

    usability,and desirability

    based upon the sum of

    all direct and indirect interactions

    with our products and services

  • 7/29/2019 Measuring the Mobile Experience

    4/26

    The customers perception of

    usefulness,

    usability,and desirability

    based upon the sum of

    all direct and indirect interactions

    with our products and services

    How ESPN defines UX

    4

    usefulness,

    usability,desirability

    all direct and indirect interactionsHow well does it integratewith our other touchpoints?Consider all of the different ways users

    consume content and interact with usacross our multiple products platforms

    Can fans achievetheir goals?If it doesnt satisfy theirneeds, they have noreason to use it.

    Do fans want to use it?What is the emotional impact?How is it attractive? Is it fun,exciting, or pleasurable?

    How easily can fansachieve their goals?

    People want the greatestreward with the least amountof effort.

  • 7/29/2019 Measuring the Mobile Experience

    5/26

    The customers perception of

    usefulness,

    usability,and desirability

    based upon the sum of

    all direct and indirect interactions

    with our products and services

    How ESPN defines UX

    5

    usefulness,

    usability,desirability

    all direct and indirect interactionsHow well does it integratewith our other touchpoints?Consider all of the different ways users

    consume content and interact with usacross our multiple products platforms

    Can fans achievetheir goals?If it doesnt satisfy theirneeds, they have noreason to use it.

    Do fans want to use it?What is the emotional impact?How is it attractive? Is it fun,exciting, or pleasurable?

    How easily can fansachieve their goals?

    People want the greatestreward with the least amountof effort.

    When a fan engages with any ESPNproduct or service,

    they have an experience.

    We aim to shape it.

  • 7/29/2019 Measuring the Mobile Experience

    6/26

    Fifty-six TV networks acrossseven continents

    Two domestic radio networks,syndicates in eleven countries

    ESPN.com,ESPNDeportes.com, anddozens of internationalmarket-specific sites

    ESPN The Magazine

    ESPN Live on XBox

    ESPN Mobile

    Millions of fans

    6

  • 7/29/2019 Measuring the Mobile Experience

    7/26

    Industry-leading collection ofmobile web sites, apps, games,alerts, mobile TV and video ondemand clips

    Mobile Web:70% share of the Sports category

    Mobile Apps:63 apps across iOS, Android, Blackberry,and Windows Mobile with 1.8m uniquevisitors per day

    Mobile Alerts:7.5 billion alerts, over 9m subscribers

    WatchESPN:Streaming domestic cable networks andESPN3 to mobile tablets and handsets

    Mobile: Rapid growth

    7

  • 7/29/2019 Measuring the Mobile Experience

    8/26

    8

    Measuring the Sports Experience

  • 7/29/2019 Measuring the Mobile Experience

    9/26

    Helps us fully meet theneeds of our fans andbusiness partners.

    How do fans behave today?

    What are their needs(stated and unstated)?

    What products or servicesmay meet their needs?

    What characteristics should

    it have?

    If this is an existing product, isit working as intended?

    Fan and market research

  • 7/29/2019 Measuring the Mobile Experience

    10/26

    MarketplaceCompetitors and trends

    TechnologyEmerging tech / innovation opportunities

    Field ResearchObservation and interviews

    AnalyticsQuantitative data on usage behavior: numberof users, time spent, conversions, minutesviewed, location, and much more

    Multivariate TestingHow different combinations of possibledesign solutions impact usage behavior

    What we look forFAN AND MARKET RESEARCH

  • 7/29/2019 Measuring the Mobile Experience

    11/26

    Apps vs. WebsitesHandsets vs. TabletsHome vs. Office vs. Away

    Nearly 1/3 downloaded apps areused just once

    About half of mobile users deleteapps within the same month theybuy the app; up to 90% areeventually deleted

    Most (68%) handset owners use fiveor less apps on a regular basis

    The top 10 Android apps (of 250,000total apps) account for 43% of all appusage (Aqueant)

    Some of what we knowFAN AND MARKET RESEARCH

  • 7/29/2019 Measuring the Mobile Experience

    12/26

    12

    Measuring the Sports Experience

  • 7/29/2019 Measuring the Mobile Experience

    13/26

    The mobile landscape is rapidlychanging, and so are fanexpectations. This typicallyresults in a degradation ofperformance over time.

    We use a variety of techniques tobenchmark performance anddiagnose problems:

    Usability testing

    Walkthroughs

    Expert reviews

    Usability reviews and testing

  • 7/29/2019 Measuring the Mobile Experience

    14/26

    Experimental Design We cant move events or seasons

    Fixed timeframes (dates driven by event schedules)

    Synchronization between live events and test sessions

    Prototypes and offending ones fandom

    Accounting for environmental/situational effects

    At home, work, waiting rooms, trains, bars, bathrooms Alone or with friends

    Edge, 3g, 4g, WiFi availability and quality of connections

    Test devices and the fans personal device may differ Hardware, operating systems, and personal settings

    Technology

    Screen capture is challenging (or impossible);not directly supported by products like Morae

    Capturing hand movement, gestures

    Many versions of devices (especially Android):how do we account for issues specific to each?

    Challenges of mobile and sportsUSABILITY REVIEWS AND TESTING

    14

  • 7/29/2019 Measuring the Mobile Experience

    15/26

    Recruiting Participants unassociated with ESPN (difficult in CT)

    Attitudes toward ESPN brand (not everyone loves us)

    Moderation Building rapport is critical; sports fandom is about

    identity and community the moderator must come offas authentic Understanding of the sport

    Language (terminology, colloquialisms)

    Accent

    Gender

    Fandom

    Presentation of Results Familiarity with different devices; some may not

    understand platform-specific issues

    Challenges of mobile and sportsUSABILITY REVIEWS AND TESTING

    15

  • 7/29/2019 Measuring the Mobile Experience

    16/26

    Screen Capture Requires unlocking or rooting; can

    result in system instability, abnormalbehavior, and crashes even damage

    Does not capture issues resulting fromphysical and/or screen interaction

    (gestures, size, sensitivity of touch points)

    Videographer

    Inconsistent capture; participant andvideographer movement

    Awkward for participants; may impact their

    task performance

    Additional resource coordination and cost

    CapturingUSABILITY REVIEWS AND TESTING

    16

  • 7/29/2019 Measuring the Mobile Experience

    17/26

    17

    The mobile sledUSABILITY REVIEWS AND TESTING

  • 7/29/2019 Measuring the Mobile Experience

    18/26

    Plastic cut and formed to fitdevice and camera so theentire screen is in-view(without obstructing screen visibilityfor the participant)

    Lightweight Manual-focus HD webcam

    Device-specific mounts

    Anti-glare screen protectors

    The mobile sledUSABILITY REVIEWS AND TESTING

    18

  • 7/29/2019 Measuring the Mobile Experience

    19/26

    Sharing functional prototypesfrom developmentenvironments

    Simulators available in bothiOS and Android SDKs

    Remote testingUSABILITY REVIEWS AND TESTING

    19

  • 7/29/2019 Measuring the Mobile Experience

    20/26

    20

    Measuring the Sports Experience

  • 7/29/2019 Measuring the Mobile Experience

    21/26

    Key Measures Content

    Navigation

    Advertising

    Video

    Scores

    Future Intent

    Longitudinal Measurement SUS (System Usability Scale)

    NPS (Net Promoter Score)

    ACSI (American Customer Sat. Index)

    Spot Surveys Surveys specific to individual products

    and topics to explore other dimensions ofuser experience

    Surveys

  • 7/29/2019 Measuring the Mobile Experience

    22/26

    The value of survey data

    Important indicator satisfaction downstream of usability

    Helps us understand expectations and primary tasks (are we

    meeting/supporting them)

    Helps validate other evaluation methods

    Compare mobile to other platforms

    Insight into mobile adoption and behavior, by device and OS

    How things out of our direct control impact satisfaction

    FAN SURVEYS

  • 7/29/2019 Measuring the Mobile Experience

    23/26

  • 7/29/2019 Measuring the Mobile Experience

    24/26

    Challenges of mobile

    Limitations on screen size; questions with multiple possibleanswers (like Likert scales) can be hard to display

    Usage contexts, distractions, and connection quality;less tolerance for long surveys

    FAN SURVEYS

  • 7/29/2019 Measuring the Mobile Experience

    25/26

    Take-Aways

  • 7/29/2019 Measuring the Mobile Experience

    26/26

    WORKING TOGETHER

    Take-Aways

    Look for trends across the entire ecosystem;the user experience is not contained to one

    device, platform, app, or website

    Involve the entire team

    Deliver actionable results quickly