Presentation GuideThursday, October 24, 2013

Agenda

8:30 AM Breakfast - Pearson Room

9:00 AM Welcome & IntroductionJim Spillson (AbelsonTaylor)

9:10 AM Setting Context for Quality in Healthcare MarketingJoe Shields (AstraZeneca)

9:40 AM Shift LeftJim Spillson (AbelsonTaylor)

10:00 AM Mind the GapBen Currie and Elizabeth Estes (GA Communication Group)

10:30 AM Break

10:40 AM “Roundtable” Discussion

12:00 PM Lunch - Buckingham Room

1:00 PM Keynote PresentationTesting is not Assurance: QA in Digital MarketingMichael Morowitz (R/GA)

1:30 PM Client Panel Discussion

2:40 PM Shift to MobileShachar Schiff (Bad Testing)

3:00 PM Break / Afternoon Snack

3:20 PM Best Practices Document Working Session

4:50 PM Closing - Jim Spillson

Welcome

Introductions

Jim Spillson – HostManager of Quality Assurance, AbelsonTaylor

Dale Taylor – Co-HostPresident, CEO, AbelsonTaylor

Mark Goble – Co-HostPresident, COO, GA Communication Group

Ben Currie – Co-HostGroup Director Digital Solutions, GA Communication Group

Joe Shields – ModeratorDigital Marketing Leader, AstraZeneca

2

QA: We Hit a Nerve

3

We All Face the Same Questions

iOS management How much testing is enough?

4

QA team responsibility

Jailbreaking (iOS)Simulators

Automation

3rd party vendors

PC operating systems

MDM deployments

Are some bugs not worth fixing?

What about mobile testing?

Bug tracking systems

Test devices

What about Android?

Xcode

Crowd source testing

Constant release of browsers

client ITBrowsers too old?

Response has been Overwhelming

5

Agenda

Morning Session

9:00 – Welcome & introductions [Jim Spillson]

9:10 – Setting Context for Quality in Healthcare Marketing [Joe Shields]

9:40 – Shift Left [Jim Spillson]10:00 – Mind the Gap [Ben Currie &

Elizabeth Estes]10:30 – Break10:40 – “Roundtable” Discussion

6

Afternoon Session

12:00 – Lunch1:00 – Testing is not Assurance:

QA in Digital Marketing [Michael Morowitz]

1:30 – Client Panel Discussion2:40 – Shift in Mobile

[Shachar Schiff]3:00 – Break3:20 – Best Practices Working

Session4:50 – Closing

Our Goal

Best Practices Document• Gather all information today• Review, refine, participant approval/

endorsement• Publish and distribute

7

Next Stop

8

Discussion Format and Miscellaneous

• Participation– Ask questions: working session with real outcomes– Q&A for panel discussion, keynote and presentations will be at the end– Best Practices template in front of you

• Considerations– Place phones on vibrate– Respectful of others– Microphones for everyone

• Miscellaneous– Breaks and lunch in Buckingham Room– Charging station– Special requests (hosts/hotel)

9

Future Events

10

Information

• Publish Best Practices

• YouTube channel• White paper• Publications• LinkedIn group• StateofQA.com

2014 QA SummitSXSW 2014• QA Best Practices

in Today’s Digital Agencies

Setting Context for Quality in Healthcare Marketing

Joe ShieldsDigital Marketing Leader

Disclaimers

• The views expressed are my own & not necessarily those of my employer

• Please note that all images in this presentation are borrowed from the Internet, and cited where possible

• There may be a Google Glass reference today. Sorry

2

Everything I know about quality is from film shoots

3

ALMOST

“We’ll fix it in post” never really happened

4

Managing creativity is hard

“When the job is to conjure the next brilliant idea out of thin air, against

deadline, via a combination of inspiration, hard work, experience,

intuition, and confidence, getting the best work out of creative people on a consistent and efficient basis can be

tricky business.”

5

Source: Rae Ann Fera, “10 Tips for Managing Creative People,” Fast Company, September 17, 2013.

We struggle with being subjective vs. objective

6

We struggle with being subjective vs. objective

7

The world has changed….

Customers live in a multiscreen world

• Screens are more than just objects we look at to consume content. We can also:– Use voice+gestures to instruct them– Shoot pictures+video– Play+write music– Get+give directions– Pay for lattes+shoes– Message friends+foes– Control apps+devices

9

Connectivity is now a basic human need

10

SELF-ACTUALIZATION

SELF-ESTEEMBELONGING-LOVESAFETYPHYSIOLOGICAL

WIFI

Technology is fundamentally changing human behavior

• “One in six Americans with recent dating experience have broken up with someone—or had someone break up with them—by text message, email, or sending a message online”

• “People high in FOMO … feel less competent, less autonomous & less connected with others than people who don't worry about being left out”

11

Sources: Pew Internet & American Life Project: Online Dating & Relationships, October 21, 2013, and “No More FOMO: Fear of Missing Out Linked to Dissatisfaction,” Stephanie Pappas, LiveScience.com, May 14, 2013.Image credit: Hubspot.

These human behaviors are also healthcare behaviors

•Consume•Create•Connect•Collaborate•Contextualize•Care

12

Image credit: Boyle-Associates.com.

HCPs are close to the digital tipping point

• In 2014, the majority of EU HCPs will be digitally native

13

Source: Manhattan Research, TTP, ePharma HCP 2012.Image credit: LBi Digitas.

Clinicians are now ‘digital omnivores’

• Digital omnivores routinely use tablet, smartphone & laptop/desktop for professional purposes

• Omnivores are becoming the dominant type– 2012: 28% of clinicians– 2014: 82% of clinicians

• Mobile devices extend the digital day– Computer use drops at 5 PM, but mobile use

for professional reasons remains high until 9 PM

14

Source: “Mobile Trends Report,” ePocrates, 2013.

Changing healthcare context has made ‘Healthcare Anywhere’ a reality for many

Ubiquitous Connectivity

HEALTHCARE ANYWHERE

Empowered Patients

Technology-enabled

Outcomes -driven

Cost Containment

Investments in healthcare must yield better quality outcomes

• “Quality measures are tools that help us measure or quantify healthcare processes, outcomes, patient perceptions, and organizational structure and/or systems that are associated with the ability to provide high-quality health care and/or that relate to one or more quality goals for health care

• These goals include: effective, safe, efficient, patient-centered, equitable & timely care”

16

Source: Centers for Medicare & Medicaid Services, cms.gov.

We have been slow to respond to these dramatic changes….

We are no longer in the advertising business

18

It was fun while it lasted.

We need more content for more devices, and faster

19

RESPONSIVE ADAPTIVE

C D E F

A B

C

D E F

A B ABCD

C D E F

A B

C D

A G

E

G

HB C

< optimal viewing experience

optimal user experience >

So we must methodically plan content

20

Operating System (OS)

Screen Size

Platform or Channel

Although the possibilities seem endless

21

Operating System (OS)

Screen Size

Platform or Channel+ MILLIONS OF CUSTOMER JOURNEYS+ SEVERAL BRANDS

+ MANY FILE FORMATS+ PUSH VS. PULL+

BU

SIN

ES

S R

ULE

S

We must be better at technology

22

visualizationconnectivity

memoryprocessingbattery lifeintegration

Image credit: Samsung.

Fitness or medical device

We can’t do it ourselves

23

“The Marketing department in my company spends

more on IT services than the IT department.”-- Overhead at a recent

pharma industry conference

Yet we’re confused about the current & future role of medical communications agencies

24

Source: “Omnipotent, or omnishambles?” The Economist, August 3, 2013.

And stuff like this isn’t helping

"Go back 70 years, go ahead 50 years, a human is a human. There are primal things that will always drive us: Will this product be better, will it help me succeed, will it make me more attractive? So the technology and the way we to speak to people will change, but those fundamentals will never change.“

David Lubars, North American chairman & chief creative officer, BBDO

25

Source: “The Future of Advertising: Three Agencies' Visions,” Fast Company, November 19, 2010.

Or this

26

• Email on Friday from one of my agency friends, who is a VP >

Quality is more important than ever….

We are all in the patient care & healthy behaviors business

28

Image credit: Regrounding.Wordpress.com.

Together we build complex, technology-driven systems

29

Image credit: MasterNewMedia.org.

Quality matters in everything we do, not just in software

• Except when it doesn’t

30

Source: Twitter post from Forrester Ideas conference, October 3, 2013.

The lifecycle management of these products & systems happens over years, not months

31

Image credits: PSDgraphics.com, Apple Corporation.

forever.

Finally, we must still manage the tension among quality, speed & cost

• “The ‘cost of quality’ isn't the price of creating a quality product or service. It's the cost of NOT creating a quality product or service. Every time work is redone, the cost of quality increases.”

32

Source: American Society for Quality, ASQ.org. Image credit: Jokeroo.com.

So let’s talk.

Shift Left

Why Shift QA to the Left?

2

QA starts too late in the process – like at the end.

Build things that aren’t broken in the first place.

The longer a bug hangs around, the more it costs to fix.

Typical Agency Process

3

Job start Requirements Documentation Concept/design Routing

Client review Revisions Development Routing Revisions

Testing Programming revisions Testing Regulatory

review Repeat (2x)

Release

Where Defects Come From

56%

27%

10%

7%

Where Defects Originate

Requirements

Code

Other

Design

4

The Journal of Defense Software Engineering

Shift Left

5

Requirements ReleaseTestDevelopmentDesign

Phase View of Quality

Project Life Cycle

Num

ber o

f Def

ects

Job

Ope

ns

‘Shift Left’Early Detection of Defects

Project Life CycleShift Left Life Cycle

Financial Impact of Shifting Left

6

Cost to fix a defectTime detected

Requirements Doc/Creative Development Testing Post-release

Time introduced

Requirements 1x 3x 5-10x 10x 10-100x

Doc/Creative – 1x 10x 15x 25-100x

Development – – 1x 10x 10-25x

Quality Assurance Really Isn’t A Luxury

7

“It can be tempting to see quality assurance as an overhead. When schedules start to slip then it can be tempting to cut down on quality assurance, but this is a

false economy. A project that is beset by quality assurance difficulties is a grueling experience. It’s also an expensive one as you end up pouring resources into fixing bugs that could and should have been caught earlier in the development process. It blows a hole in profitability,

damages the reputation of your business, undermines user confidence and demoralizes development teams.

Quality assurance really isn’t a luxury.”

10 Myths About Quality Assurance in Software Development – Ben Morris

It’s an Ongoing Process

8

Shift in mindset

• Support from the top

• Client involvement

• Collaboration at all levels

Enable teams

• Training• Proper Integration

• Empowerment

Continuous improvement

• Keeping up with tech

• Tracking defect origination

Process Changes

• Bug tracking throughout

• Automation tools• Best practices document

Shift Left – What it Means to Your Business

9

Improved overall quality

Reduction in overall Software Development LifeCycle (SDLC) cost

Transparency and awareness of quality in SDLC

Increased client satisfaction

Improved adherence to timing and scopes of work

Who Are We?

2

Name: Ben CurrieTitle: Director of Marketing

Technology for GA Communication Group

Fav QA Quote: “A foolish consistency is the hobgoblin of little minds.” -

Emerson

Name: Elizabeth EstesTitle: EVP/Chief Strategy Officer

for GA Communication Group

Fav QA Quote: “If you don’t have time to do it right, you must have time to do it over.” - Unknown

Yin/Yang

3

Goal: Get the Conversation Going

4

• Let’s Look At It From Inside Out

• Part Presentation/Part Discussion

• Debate vs. Answers

• Catalyst for Best Practices Doc

Our General Thinking

5

• Quality Requires More Than a Technical Discipline at the End of a Project

• Determining “Critical to Quality” Factors at the Beginning Is Vital

• There Is a Language Barrier Between the Disparate Groups

Why Is It Hard?

6

“I need an automated email that tells people what the weather will be on the day of their check up”

“I need their email address, location and date of their appointment”

Let’s Understand the Players

7

http://blog.mindjet.com/wp-content/uploads/2012/05/Between-Minds-An-Ongoing-Taxonomy-of-Team-Dynamics1.jpg

We Are Different Now

8

http://www.mdgadvertising.com/blog/the-modern-marketer-infographic/

Team Dynamics Are Complex(er?)

9

http://esdandassociates.com/2012/06/the-dynamics-of-marketing-success/

So Are the Options and Opportunities

10

http://www.mergesocialmedia.com/

So Are Expectations

11

http://www.strangeloopnetworks.com/resources/infographics/mobile-infographics/poster-mobile-infographics/

• Consumer Tolerance Limits

• On-Demand World

• Personalization Expected

• User Experience Vital

Under Pressure (Da-Da-Da-DaDa-Da-Da)

12

• Expected to Know More and Do More in Less Time

• While Coordinating More Specialists

• Without Sacrificing Quality

Kinda Like...

13

So What Are The Components of Shift Left?

14

How Do We…?

15

• Address the “Human Side” Systematically• Start at the Top• Involve Every Layer• Identify “Critical to Quality” Moments• Make the Formal Case • Create Ownership • Communicate the Message • Assess the Cultural Landscape• Address Culture Explicitly • Prepare for the Unexpected • Speak to the Individual

“Reach Right” to Help the Shift

16

Identifying Issues

17

Communication Process Strategy Time & Budget Functionality Maintenance

Outside of Functionality

18

Communication Process Strategy Time & Budget Functionality Maintenance

And Close the Gap

19

Communication Considerations

20

• Internal– Project Kickoffs– Ongoing Team

Reviews – Post Project

• External – Client Involvement– Client Communication

• Lack of Context– Language Barriers– Empathy Chips/Perspective

Process – Issues to Consider

21

• PPDS

• Identifying “Critical to Quality” Moments

• Documentation

• How Locked Down Should It Be?

• Acknowledged Shortcut Areas?

• Managing Change

Strategic Considerations

22

• Feasibility?

• Clear Direction?

• Correct Channel?

• Correct Technology?

• Pulled Through To End?

Time & Budget Considerations

23

• Enough Time?

• Enough Money?

• Is the Project Fully Understood To Determine Time & Budget?

Functionality Considerations

24

• Critical to Business Requirements?

• Critical to User?

• Critical to Success?

• How to Handle Distractions That Do Not Meet Critical Goals But Jeopardize Success?

Maintenance Considerations

25

• Opposite of Software Updates

• Where Are the Checks and Balances?

• Who Is Responsible?

• How Is Value Communicated and Understood?

What Is YOUR Single Biggest Issue?

26

It’s Your Turn Soon!

27

And Finally….

The Shift to Mobile

The Shift to Mobile.

• Shachar Schiff

• Work out of 1871, the technology center at Chicago's Merchandise Mart, designed to help launch and grow early-stage tech start-ups.

• BAD TESTING® offers an advanced and versatile skill set in software quality assurance, testing, and configuration to ensure effective and on time product releases.

Mobile | Overtakes Desktop

Mobile | QA

• Mobile-device testing involves a set of activities from monitoring and trouble shooting mobile application, content and services on real handsets.

• Testing includes verification and validation of hardware devices and software applications.

Mobile | Designs

• You need to know about responsive, adaptive, mobile, and native app design.

• Raises the bottom line on site's final cost.

• Review analytics data.

• Know what device your audience is on.

• Important to look at site traffic and target audience before determining.

Mobile | Estimation

• Project estimations for responsive/adaptive websites need to be treated as separate projects for each Break-point.

• Including development and QA.

Mobile | Responsive & Adaptive

Responsive

• "Fluid" layouts shrink and grow based on the browser size.– Content is sized using percentages, adjusts as the browser size changes.

Adaptive

• Content reconfigured at predetermined "breakpoints".

• Easier to control how content will flow

• 1,2,3 site redesign increases costs, time.

Mobile | Responsive

Mobile | Adaptive

Mobile | Specific & Native App

Mobile-specific

• Mobile-specific site separates your site's desktop experience from the mobile experience.

• Desktop URL on mobile redirects to separate URL (m.yourdomain.com)

Native App

• Device-specific features (GPS, camera, accelerometer, NFC)

• Difficult topic due to development costs

• iOS is different than Android:• Alienates 50% of potential users• 2x costs to develop (iOS, Android)

Mobile | Desktop Ownership Decreasing

Mobile | Pew Internet Project Highlights

As of May 2013:

• 91% of American adults have a cell phone

• 56% of American adults have a smartphone

• 28% of cell owners own an Android; 25% own an iPhone; 4% own a Blackberry

As of September 2013:

• 24% of Americans ages 16 and older own an e-reader

• 35% of Americans ages 16 and older own a tablet computer

Mobile | Device Shipments

Mobile | Device Shipment Highlights

• Today feature phones (non smartphones) outsell smartphones; and PCs and portable PCs (notebooks/laptops) both outsell tablets.

• In the future that is expected to change.

• Both analysts predict that marginally more smartphones will be sold in 2013 than feature phones.

Mobile | Desktop Limited to Working Hours

Mobile | Mobile Web Statistics

• 61% of people have a better opinion of brands when they offer a good mobile experience.

• 25.85% of all emails are opened on mobile phones, and 10.16% are opened on tablets.

• Almost half a billion tablets will ship in 2013 and 2014 alone.

• Mobile-based searches make up one quarter of all searches.

Mobile | Summary

• The reason shift left is so important to mobile is because customers and employees are making the mobile mind shift.

• They expect that any information or service they desire be available to them on any device, in context, at their moment of need.