!!!

Table of Contents

Page 2-4: Mobile Testing Requires a

Difference Approach

Page 5-7: A Mobile Testing Case Study—

Platform for Humana

Page 8-10: A Mobile Testing Case Study—Platform for Starwood Hotels &

Resorts

1

WHITE PAPER

MOBILE TESTINGIN THE CLOUD

As the pace of mobile device use among workers and consumers accelerates, enterprises are challenged to develop applications and websites for devices beyond PCs. Unfortunately, as mobile gains in importance for the enterprise, companies sometimes cut corners in a key area in order to get their apps and websites to market quickly: testing.

Mobile Testing RequiresA Di! erent Approach

Why you should choose the cloud

TESTING, 1, 2, …

Consider the cautionary tale of the Romney campaign’s supposed “killer” mobile app, Orca, which was intended to deliver data and help with get-out-the-vote efforts in battleground states. The app was never tested and thus suffered crashes throughout Election Day. Compare that to the approach of healthcare company Humana. To ensure its 22 million customers have reliable access to information when they need it while doing so cost effectively, companies like Humana now insist on testing its mobile apps and websites using a proven cloud-based platform.

5 STARS APPS

Some companies may quickly build apps or websites without testing and deploy them through an app store, risking low app ratings and treating fi rst users as testers. This is a sure way to get negative feedback for your app. With readily available mobile testing and spot-checking tools available, there is virtually no excuse today for such high-risk practices. Taking the risk of skimping on testing can be detrimental to a mobile app, and internal IT organizations have less time and resources to manage the complexities of the “Bring Your Own Device” (BYOD) culture. Testing is crucial.

2

Mobile apps multiply the complexity because of device diversity, platform diversity (iOS, Android, BlackBerry or Windows) and network challenges, among others.

To meet the relatively complex testing demands of a growing mobile market, companies are looking for alternatives to maximize their testing effi ciency while keeping budgets in check. Some companies are outsourcing testing overseas to reduce labor costs. Reliance on overseas assets for mobile testing introduces its own set of challenges. Hardware and network access become much more diffi cult to manage. Rather than sending test cases overseas, enterprises are now reaping the benefi ts of sending them to the cloud. Cloud testing is now growing in favor as it offers both lowered costs and increased resource productivity.

ELIMINATING COMPLEXITY

It’s a simple idea. Real devices connected to live mobile operator networks are electrically integrated into specialized hardware hosted in a centralized location. Because the screen output is also captured, a remote user can see the actual rendering in real time. Users can therefore tap directly into real devices to test mobile functionality on different devices and platforms.

Rather than hours or days of in-house testing (and monitoring), testing becomes an on-demand activity. Developers can access mobile devices from any desktop in the world to test a mobile website or service while simultaneously testing multiple devices for functionality. Leading cloud testing platforms provide additional testing tools and advanced metrics, and they stay abreast of the latest devices to hit the market.

A VIRTUAL WORKBENCH

Cloud testing assures that companies have continual access to the right tools for the job, greatly reducing the amount of time needed to get to the root of most issues (and solve them) by having real-time access via the cloud to a broad set of devices and networks. In addition, testing can begin immediately, with no limitations in terms of geography or time zones. Companies can fi nd out precisely how their mobile apps, websites and content perform on actual browsers, networks and mobile devices using on-demand test and measurement products and services. Even better, testing can ramp up and scale back quickly, which would be very diffi cult to replicate outside of the cloud.

3

KEEPING PACE WITH MARKET CHANGES

Beyond cost savings and productivity, cloud testing also provides fl exibility. New devices, platforms, fi rmware releases and technologies are constantly being released, but companies still need to be assured that their apps and websites function consistently and correctly, not putting any user data at risk of leaks or unauthorized access. Meanwhile, images and text must display correctly on the many different mobile devices.

For example, the iPhone 5 introduced a new screen size and iOS 6 introduced new features, thus prescribing testing for a new form factor and updated operating system. Using cloud-based testing assures companies will have access to the most popular devices on the market, accommodating every upgrade and patch. Leading cloud testing platforms will also provide access to older versions of each device model, enabling functional and regression testing across a representative sample of devices out in the market.

ADAPTABLE TO YOUR NEEDS

While cloud testing can provide simplifi cation and effi ciency, one shouldn’t walk away assuming that that a platform is a one-size-fi ts-all solution. Every company will have a different set of application, performance, security and compliance requirements, sometimes giving them reservations about trusting the cloud. Security, in particular, is often a concern. Fortunately, top cloud testing vendors can build a test system behind a customer’s fi rewall (for security) while still delivering the cost savings that companies need.

Companies should investigate the cloud testing capabilities of any vendor they are considering. The best vendors will make sure that when customers are using a device, no one is able to see that device screen except those who have been invited to do so. Good cloud testing solutions execute automated cleanup scripts that wipe the device at the end of each session. Such scripts clean the device, deleting all downloaded or installed applications, clearing the browser cache, and removing other usage footprints like text or picture messages.

OS?

4

Keynote Systems, Inc.777 Mariners Island Blvd.San Mateo, CA 94404

T 650-403-2400F 650-403-5500www.keynote.com

The trademarks or registered trademarks of Keynote Systems, Inc. in the United States and other countries include Keynote®, DataPulse®, Keynote Customer Experience Rankings®, Perspective®, Keynote Red Alert®, Keynote WebEffective®, The Internet Performance Authority®, MyKeynote®, SIGOS®, SITE®, keynote® The Mobile & Internet Performance Authority™, Keynote FlexUse®, Keynote DeviceAnywhere®, Keynote DeviceAnywhere Test Center®, Keynote DemoAnywhere® and Keynote MonitorAnywhere®. All related trademarks, trade names, logos, characters, design and trade dress are trademarks or registered trademarks of Keynote Systems, Inc. in the United States and other countries and may not be used without written permission. All other trademarks are the property of their respective owners.

GROW WITH AUTOMATION

One fi nal consideration for moving your testing to the cloud is scale. As your mobile presence grows, so will the demands of your testing platform. A good cloud testing platform will grow with you. When you’re ready for automated testing, it will let you conduct more tests on more devices in signifi cantly less time. Automation also reduces human error, resulting in better quality.

Look for an automation solution with an intuitive scripting interface. A comfortable, usable interface will increase the ease of creating automation ultimately resulting in a higher percentage of automated test cases. Also, look for a solution that can scale beyond the needs of the individual tester. The same cloud testing solution may also serve as a website or app-monitoring platform. Within your organization, those with responsibility for performance and availability may also fi nd great value. Sharing a resource could further reduce overall costs and provide additional value to the enterprise.

NOT IF, BUT WHEN.

Companies that are not testing to ensure the user experience of mobile users should seriously consider adopting a mobile-testing strategy. What’s more, companies reliant on in-house solutions should evaluate the benefi ts of moving their testing to the cloud. Cloud-based testing of mobile apps and websites is a practical and proven solution that can improve both the cost and quality. After all, it’s not the smart devices that drive business innovation, but the apps, websites and services that run on them. In this demanding but hugely promising era, mobile functionality is only as good as how well it’s been tested.

1

We are dedicated to making

business decisions that refl ect our

commitment to improving the health

and well-being of our members,

our associates, the communities

we serve, and our planet.

Humana’s health benefi ts strategy

is changing our industry. Our focus

on people, choice, engagement,

and innovation guides our business

practices and decision-making.

CASE STUDY

THE CHALLENGE:

As more and more consumers rely on accessing information using mobile devices, it’s becoming increasingly imperative for businesses to provide mobile-based customer support. This is especially true for health care providers, for whom fast, reliable access to account information is a crucial aspect of members care.

Humana, a Fortune-100 health and well-being company with more than 40,000 associates provides consumers and businesses with access to information via mobile channels across their various health care operations in all 50 U.S. states. Several of Humana’s mobile solutions are delivered through HTML5- based web apps, in addition to mobile websites and native mobile apps, so that the largest possible number of members could access their account information across the largest number of mobile devices.

Given the importance of the information to its members, Humana needs to ensure that the mobile apps, websites and web apps worked consistently with as little downtime as possible. That means that the company needs a cost-effective way to constantly test and monitor all its mobile channels from a member’s perspective, in order to ensure the effi cacy of their mobile solutions.

Humana needed a cost-e! ective way to constantly test and monitor all its mobile channels from a members perspective.

DeviceAnywhereMobile Testing Platform for Humana

2

THE SOLUTION: KEYNOTE DEVICEANYWHERE

To guarantee that customers enjoyed continual access to information on mobile channels, Humana turned to Keynote.

With the DeviceAnywhere mobile testing platform, Humana has set up a local, dedicated testing environment to cover all of their mobile channels. The environment is designed to test their native apps, mobile websites, and hybrid HTML5 apps. With this test environment the healthcare provider can run manual and automated tests on 50 live devices, including iOS, Android, and Blackberry platforms.

By using DeviceAnywhere Humana can write scripts just once and test their HTML5 web apps across all HTML5-enabled browsers and devices.

IMPACT HIGHLIGHTS

• Humana required a solution for testing mobile apps and websites, to ensure their members’ continued, reliable access to healthcare information.

• Humana wanted a testing environment that would enable them to test native, hybrid and HTML5 based web apps and websites.

• Keynote’s DeviceAnywhere mobile testing platform allows Humana to implement over 150 test cases across many of the most popular mobile devices on the market today, allowing the health and well-being company to ensure their members have fast, easy access to their health records and account information, regardless of location or time.

3

ABOUT KEYNOTE

Keynote (NASDAQ: KEYN) is the global leader in Internet and mobile cloud monitoring. We provide companies with solutions for continuously improving the online experience.

As an independent and trusted third-party, Keynote provides IT and marketing executives with an unbiased view into their Internet services from around the world. For over a decade, we have been providing measurement data and testing capabilities that allow companies to understand and improve their customer’s online and mobile experience.

Keynote Systems, Inc.777 Mariners Island Blvd.San Mateo, CA 94404

T 650-403-2400F 650-403-5500www.keynote.com

All rights reserved. The trademarks of Keynote Systems, Inc. include Keynote®, DataPulse®, CustomerScope®, Keynote CE Rankings®, Keynote Customer Experience Rankings®, Perspective®, Keynote Red Alert®, Keynote Traffic Perspective®, Keynote WebEffective®, The Internet Performance Authority®, MyKeynote®, SIGOS®, SITE®, keynote™, The Mobile & Internet Performance Authority™ and all related trademarks, trade names, logos, characters, design and trade dress are trademarks or registered trademarks of Keynote Systems, Inc. in the United States and other countries and may not be used without written permission. CS1060

THE RESULTS: RELIABLE MOBILE SOLUTIONS FOR HUMANA MEMBERS

When members cannot reliably access their health care information, it can become more challenging than just another customer support issue in other industries. The DeviceAnywhere mobile testing platform acts as an early-warning system so that Humana can react to issues before their members encounter them.

Members increasingly depend on their mobile devices, which is why Antonio Melo, technology manager for enterprise mobile engineering at Humana, noted, “we have focused on ensuring that information is available on the devices that our members use, and we are committed to ensuring the quality and uptime so that our customers know they will be able to access the information wherever and whenever they need it.”

A leader in healthcare, Humana leverages Keynote’s DeviceAnywhere platform to deliver leading, dependable mobile solutions for members throughout the United States.

1

CASE STUDY

THE CHALLENGE: ACCESSING MOBILE APPS WITHOUT PURCHASING MOBILE DEVICES

DeviceAnywhereMobile Testing Platform for Starwood Hotels & Resorts

With Starwood’s mobile presence expanding, the global online compliance team needed to ensure that the content and functionality of its mobile apps and websites met corporate brand standards.

Individual Starwood brands and properties are constantly developing and improving their websites and mobile apps to communicate with their customers. Customers use these channels to find hotels, learn about services, view pictures of properties and rooms, book reservations, and more.

To ensure that these sites and apps properly represent the Starwood brands and are compliant with corporate policies, the global online compliance team conducts regular audits and reviews. As the hotels have increasingly moved from providing information on websites to also supporting various mobile platforms, such as Android and iOS the global online compliance group needed to keep pace with reviews and audits of these new apps.

Normally, the global online compliance group would have needed to purchase mobile devices to evaluate device-specific mobile apps. The group, however, wanted to save the time that would have been necessary to go through budget request and approval process and the cost of the devices themselves. It began looking for an alternative.

Starwood Hotels & Resorts

Worldwide, Inc. is the most global

high-end hotel company in the

world. It boasts nine distinct brands

with a total of 1,134 properties in

nearly 100 countries and employs

171,000 associates worldwide.

2

KEYNOTE DELIVERS THE RIGHT CAPABILITIES

Janay McWhorter, senior manager for the global online compliance group spearheaded the search. She began by seeking references, researching online communities, and searching the web, ultimately learning about the Keynote’s DeviceAnywhere platform and requesting a demo of the product.

After seeing the product go through its paces, McWhorter was sold. “We chose DeviceAnywhere because it allowed us to do exactly what we needed. Rather than requiring us to rent or purchase the actual devices, DeviceAnywhere allows us to go online and pick from any number of Smartphone devices and OSs to use for our testing.”

In contrast, alternative solutions were “complicated and convoluted,” McWhorter said. Moreover, they focused exclusively on testing app performance, which was outside the scope of the compliance group’s requirements.

“We chose DeviceAnywhere because it allowed us to do exactly what we needed. Rather than requiring us to rent or purchase the actual devices, DeviceAnywhere allows us to go online and pick from any number of Smart-phone devices and OSs to use for our testing.”

Janay McWhorter, Senior Manager, Global Online Compliance, Starwood Hotels & Resorts

THE SOLUTION: CLOUD-BASED REAL DEVICE TESTING OVER THE INTERNET

The online compliance organization used DeviceAnywhere as it embarked on a four-week auditing project. Five testers performed manual testing of their mobile websites and apps for Android and iOS platforms utilizing the largest real device library in the world from DeviceAnywhere. DeviceAnywhere allows site reviewers to remotely interact and control every aspect of the mobile device including pressing handset buttons, viewing LCD displays, listening to ringers and tones, playing videos, taping and swiping touch screens, muting, powering on/off, downloading apps, sending/receiving messages and anything else they could do if they held the physical device in their own hands. Using DeviceAnywhere the group was able to perform a general review and audit of all the company’s mobile apps worldwide, looking at their content and functionality.

3

ABOUT KEYNOTE

BENEFITS: THE RIGHT SOLUTION

Keynote (NASDAQ: KEYN) is the global leaderin Internet and mobile cloud monitoring. Weprovide companies with solutions for continuouslyimproving the online experience.

As an independent and trusted third-party, Keynote provides IT and marketing executives withan unbiased view into their Internet services fromaround the world. For over a decade, we havebeen providing measurement data and testingcapabilities that allow companies to understandand improve their customer’s online and mobileexperience.

Keynote Systems, Inc.777 Mariners Island Blvd.San Mateo, CA 94404

T 650-403-2400F 650-403-5500www.keynote.com

The trademarks or registered trademarks of Keynote Systems, Inc. in the United States and other countries include Keynote®, DataPulse®, Keynote Customer Experience Rankings®, Perspective®, Keynote Red Alert®, Keynote WebEffective®, The Internet Performance Authority®, MyKeynote®, SIGOS®, SITE®, keynote® The Mobile & Internet Performance Authority™, Keynote FlexUse®, Keynote DeviceAnywhere®, Keynote DeviceAnywhere Test Center®, Keynote DemoAnywhere® and Keynote MonitorAnywhere®. All related trademarks, trade names, logos, characters, design and trade dress are trademarks or registered trademarks of Keynote Systems, Inc. in the United States and other countries and may not be used without written permission. All other trademarks are the property of their respective owners. CS1063

McWhorter has been very pleased with the results. “Mobile apps are the direction Starwood’s business is moving. Increasingly, we need to monitor more than just websites. DeviceAnywhere really proved its worth.”

The group found that it was able to ramp up quickly on DeviceAnywhere. Said McWhorter, “DeviceAnywhere was very easy to use and the support was great. Our testers were able to use their devices in exactly the same way they’d use the real devices. This meant that testers were able to start working immediately with no learning curve required for the device itself.”

DeviceAnywhere does exactly what McWhorter intended. “If an app is device specific, we can now instantly go to the Internet, access the device we need to put the app through its paces.”

If one of the testers found something that didn’t look right on a particular device, he or she could easily select from DeviceAnywhere’s many choices to look at the app on a different device or a different version of the same device to determine whether the problem was with the app or with the way it was rendered on the particular device.