Measuring Communication Success

SEARS, ROEBUCK AND CO.SEARS, ROEBUCK AND CO.

Measuring Success:Both externally and internally

Measuring Success:Both externally and internally

March 2003March 2003

AgendaAgenda

Measuring external communication Linking public relations to sales Focusing on ROI

Measuring internal change management communication

Scope of the Conversion Measuring associate understanding and attitudes Customer and financial measures Conclusions

Measuring external communication Linking public relations to sales Focusing on ROI

Measuring internal change management communication

Scope of the Conversion Measuring associate understanding and attitudes Customer and financial measures Conclusions

Keep it simpleKeep it simple

Revenue - Costs = ProfitRevenue - Costs = Profit

Simplifying further...Simplifying further...

Marketing Communication

Media Relations

Events, Publicity, etc.

Crisis Communication

Investor Relations

Etc.

Revenue Increases

Cost Control

Typically measurement...Typically measurement...

Stops at awareness Often stops short of awareness

Measures output. Not impact. Is often compared to advertising

Ad equivalencies are the wrong measure No measurement of impact, change in attitude

or behavior

Stops at awareness Often stops short of awareness

Measures output. Not impact. Is often compared to advertising

Ad equivalencies are the wrong measure No measurement of impact, change in attitude

or behavior

Do they believe? Do they believe?

Businesses want to measure behavior change -- Are they buying?

Are customers buying your products and services?

Are employees buying your messages?

Businesses want to measure behavior change -- Are they buying?

Are customers buying your products and services?

Are employees buying your messages?

Clipping and Message AnalysisClipping and Message Analysis

Can be a useful management tool Measures activity and output of PR group Measures whether key messages are being seen Helpful in gauging whether coverage is negative or positive Identifies positive and negative bias of media Can calculate cost per impression But, it is not a measure of ROI or behavior change

Can be a useful management tool Measures activity and output of PR group Measures whether key messages are being seen Helpful in gauging whether coverage is negative or positive Identifies positive and negative bias of media Can calculate cost per impression But, it is not a measure of ROI or behavior change

Taking clippings further...Taking clippings further...

Link to changes in awareness, preference or behavior (purchase)

What is the impact on revenue? What is impact on cost control, associate attitudes

or other intermediate measures?

Link to changes in awareness, preference or behavior (purchase)

What is the impact on revenue? What is impact on cost control, associate attitudes

or other intermediate measures?

News ReleasesEvents

Media calls, etc(Effort)

(Difficult to measure)

Publicity(semi-controllableand measurable)

Measure is circulationor viewers

(MediaLink)

Most people never see the article or

hear the broadcast(0 impact)

Effort that nevergets printed, broadcast

(0 impact)

Percentage of effort results in publicity

Percen

t of w

asted

effo

rt

Target AudienceConsumers who see or hear the publicity and are

part of the intended audience for message(difficult to measure, but assumed to be

a fraction of the circulation number)

No impact on consumers

“I still won’t shop.”

Consumers who have an attitudinal shift -- “I will shop as a

result of that story”(measurable-- Oprah and

BrandKeys)

Those who readthe article, intend to

shop, but neverfollow through

Consumers who takeaction as a result, and

visit a store with intent to purchase

(Traffic - no currentmeasurement device)

Those who don’tfind what theyneed or wantand make no

purchase

Customers make purchaseMeasure is revenue $

(SPRS report)

Budget funds PR activity

($)

Subract cost of all investments,leaving profit $ tied to PR program.

ROI is how much of this is generatedfor every dollar invested at beginning

Those not in targetaudience -

not likely to purchase.

(minimal impact)

Businesses keep score with $What is the relationship between a $ invested in PR and

company revenue or profit?

Measures existing

Subtract customerswho purchased withoutbenefit of the publicity(test market vs. controlor incremental above baseline) (AdEffect)

Does it work?Does it work?

Can you identify changes along the path? Is there a link between public relations events and

revenue? Two examples of how changes have been

measured at Sears and how you can design a model for your business.

Can you identify changes along the path? Is there a link between public relations events and

revenue? Two examples of how changes have been

measured at Sears and how you can design a model for your business.

BackgroundBackground

One placement on the Oprah show involved providing Christmas gifts for foster children in a small Texas community.

Sears provided all the gifts for 100 children Oprah exclaimed “We love Sears!” Can this be measured?

One placement on the Oprah show involved providing Christmas gifts for foster children in a small Texas community.

Sears provided all the gifts for 100 children Oprah exclaimed “We love Sears!” Can this be measured?

Sears is high quality companySears is high quality company

Slight positive shift following the broadcast

65% agree vs. 58% Shift came from all rating

levels

Slight positive shift following the broadcast

65% agree vs. 58% Shift came from all rating

levels

0

5

10

15

20

25

30

35

40

Ag

ree

Neutr

al

Dis

agre

e

Sears Pre Sears Post

0

5

10

15

20

25

30

35

40

Ag

ree

Neutr

al

Dis

agre

e


Sears does good things for communitySears does good things for community

Following broadcast saw a huge shift to positive on this question

Almost a five-fold increase in number of those agreeing completely

Following broadcast saw a huge shift to positive on this question

Almost a five-fold increase in number of those agreeing completely

0

10

20

30

40

50

60

Agr

eeC

omp

lete

ly

Neu

tral

Dis

agre

e


0

10

20

30

40

50

60

Agr

eeC

omp

lete

ly

Neu

tral

Dis

agre

e


Plan to shop Sears for HolidayPlan to shop Sears for Holiday

The show provided the incentive to move viewers from neutral to positive intent

Increased to 70% vs. 59% before show

The show provided the incentive to move viewers from neutral to positive intent

Increased to 70% vs. 59% before show

0

5

10

15

20

25

30

35

40

Agr

eeC

omp

lete

ly

Neu

tral

Dis

agre

e


0

5

10

15

20

25

30

35

40

Agr

eeC

omp

lete

ly

Neu

tral

Dis

agre

e


Percent

“I plan to shop at Sears for the holidays.”“I plan to shop at Sears for the holidays.”

The Oprah PR placement provided the additional incentive needed to move some viewers from a neutral position to a positive intent to shop at Sears.

Neutrals dropped from 73 respondents to 52, and somewhat negatives dropped from 29 to 18.

The strongest negatives remained the same.

The positive side of the intent to shop increased from 178 to 210 respondents.

The Oprah PR placement provided the additional incentive needed to move some viewers from a neutral position to a positive intent to shop at Sears.

Neutrals dropped from 73 respondents to 52, and somewhat negatives dropped from 29 to 18.

The strongest negatives remained the same.

The positive side of the intent to shop increased from 178 to 210 respondents.

24.3 24.3

9.66.7

34.3

17.3

6 6.7

35 35.6

05

10152025303540

Ag

ree

Co

mp

lete

ly

Ne

utr

al

Dis

ag

ree


24.3 24.3

9.66.7

34.3

17.3

6 6.7

35 35.6

05

10152025303540

Ag

ree

Co

mp

lete

ly

Ne

utr

al

Dis

ag

ree


1 2 3 4 5Sears Pre 35 24.3 24.3 9.6 6.6Sears Post 35.6 34.3 17.3 6 6.6

Percentage of Respondents

…estimate what you’ll spend at Sears during this holiday season.…estimate what you’ll spend at Sears during this holiday season.

We saw earlier there was a positive shift in the number of people saying they planned on shopping at Sears.

About 10% increase in positive intent.

If we assume the mid-point for each range, and $600 for the over $500 data, then these 300 respondents represent an average of $195 per respondent after seeing Oprah, versus $140 before.

An increase of +39% per shopper in the amount they estimate they’ll spend at Sears.

We saw earlier there was a positive shift in the number of people saying they planned on shopping at Sears.

About 10% increase in positive intent.

If we assume the mid-point for each range, and $600 for the over $500 data, then these 300 respondents represent an average of $195 per respondent after seeing Oprah, versus $140 before.

An increase of +39% per shopper in the amount they estimate they’ll spend at Sears.

48.6

40.3

5.62 3.3

35.6 34.3

17.3

6 6.6

0

10

20

30

40

50

60

Lessthan$100

$100to

$250

$251to

$350

$351to

$499

Over$500


48.6

40.3

5.62 3.3

35.6 34.3

17.3

6 6.6

0

10

20

30

40

50

60

Lessthan$100

$100to

$250

$251to

$350

$351to

$499

Over$500


What was the impact?What was the impact?

Shift in attitude led to planned spending increase of $40-million

Discounted by 2/3 to account for intentions that never come to fruition, the actual spending increase may be about $13-million

Subtract costs and calculate ROI

Shift in attitude led to planned spending increase of $40-million

Discounted by 2/3 to account for intentions that never come to fruition, the actual spending increase may be about $13-million

Subtract costs and calculate ROI

Fashion Footwear and NYTFashion Footwear and NYT

January feature placement in New York Times -- New York/New Jersey edition

What was the impact in that market vs. a control market and vs. national?

January feature placement in New York Times -- New York/New Jersey edition

What was the impact in that market vs. a control market and vs. national?

The New York TimesThe New York Times

Fashion Footwear ResultsFashion Footwear Results

Week following placement NY and NJ markets were up vs. year ago.

Chicago down National down

Week following placement NY and NJ markets were up vs. year ago.

Chicago down National down

-20-15-10

-505

10152025

Ne

w

Je

rse

y

Ne

w

Yo

rk

Ch

ica

go

Na

tio

na

l

Week 1 Week 2 Week 3

-20-15-10

-505

10152025

Ne

w

Je

rse

y

Ne

w

Yo

rk

Ch

ica

go

Na

tio

na

l

Week 1 Week 2 Week 3

Footwear ConclusionsFootwear Conclusions

A single placement in a credible publication made a dramatic difference in sales compared to both national and comparable market

The only variable that changed that week was the New York Times placement

Translated into measurable dollars Cost of placement vs. incremental revenue

increase gives ROI

A single placement in a credible publication made a dramatic difference in sales compared to both national and comparable market

The only variable that changed that week was the New York Times placement

Translated into measurable dollars Cost of placement vs. incremental revenue

increase gives ROI

Connecting the dots...Connecting the dots...

Two examples measure the impact of single “communication events” -- not the cumulative impact of entire program

The right events can have a significant positive impact

Activities and events which do not have positive impact are suspect

Impact can be either a shift in attitudes or actions

Two examples measure the impact of single “communication events” -- not the cumulative impact of entire program

The right events can have a significant positive impact

Activities and events which do not have positive impact are suspect

Impact can be either a shift in attitudes or actions

Measuring internallyMeasuring internally

Can the same principles be applied to an internal audience?

How do you define success in a large change management communication program?

Can the same principles be applied to an internal audience?

How do you define success in a large change management communication program?

BackgroundBackground

In late 2001 in Full Line Stores… Announced 4,000 layoffs Announced Productivity Improvements Sears. Where Else? introduced Morale is not at an all time high

In late 2001 in Full Line Stores… Announced 4,000 layoffs Announced Productivity Improvements Sears. Where Else? introduced Morale is not at an all time high

Measures going in wrong direction...Measures going in wrong direction...

50

55

60

65

70

75

80

1999 2000 2001

Associate Attitudes% positive

Understand

Strategy

Feel good

about future

Making

changes to

compete

Conversion ObjectivesConversion Objectives

Improved customer satisfaction Easier to shop Easier to operate and easier to support store model

Clear lines of accountability throughout the organization

Improved execution on in-stock, in-store marketing, customer service

Improved profitability

Improved customer satisfaction Easier to shop Easier to operate and easier to support store model

Clear lines of accountability throughout the organization

Improved execution on in-stock, in-store marketing, customer service

Improved profitability

Communication PrinciplesCommunication Principles

Open and honest Share information as soon as it’s available Focus on key audiences, while making information

available very broadly Credibility is key

No sugarcoating No promises we can’t keep Focus on the end result and how to get it

Use existing communication channels where possible; emphasize personal communication

Open and honest Share information as soon as it’s available Focus on key audiences, while making information

available very broadly Credibility is key

No sugarcoating No promises we can’t keep Focus on the end result and how to get it

Use existing communication channels where possible; emphasize personal communication

Communication ObjectivesCommunication Objectives

Gain alignment among senior management for the changes that will be necessary

Create high awareness and understanding for: Why change is needed What changes will be made and why

Win support for the dramatic changes, despite announcements of layoffs and new job descriptions and accountabilities

Focus on new model: easier to shop, easier to operate

Gain alignment among senior management for the changes that will be necessary

Create high awareness and understanding for: Why change is needed What changes will be made and why

Win support for the dramatic changes, despite announcements of layoffs and new job descriptions and accountabilities

Focus on new model: easier to shop, easier to operate

Communication TacticsCommunication Tactics

Identify and leverage credible channels Senior leadership meetings - FLST initiated One-on-one meetings with leadership Town Hall meetings (existing and new) PMO Website Regular features in S Journal, S Update Training meetings and support materials Detailed conversion manuals Communication materials provided to DGMs for

cascading through stores organization

Identify and leverage credible channels Senior leadership meetings - FLST initiated One-on-one meetings with leadership Town Hall meetings (existing and new) PMO Website Regular features in S Journal, S Update Training meetings and support materials Detailed conversion manuals Communication materials provided to DGMs for

cascading through stores organization

Communication Tactics (cont’d)Communication Tactics (cont’d)

Issues and questions fielded in multiple ways, with response ASAP

Video Updates to field midway through conversion In-store meeting materials for consistent

messaging Input into training materials to reinforce new store

model of excellent customer service Ongoing communication through newly defined

MAP and Weekly Updates

Issues and questions fielded in multiple ways, with response ASAP

Video Updates to field midway through conversion In-store meeting materials for consistent

messaging Input into training materials to reinforce new store

model of excellent customer service Ongoing communication through newly defined

MAP and Weekly Updates

MeasurementMeasurement

Baseline MOC results reviewed Key questions pulled for ongoing pulse surveys Surveys following major communication events Weekly review of Q&A and Issues raised to

identify gaps in communication program Informal discussions within stores organization

Didn’t overreact to anecdotal information Checked feedback against data

Baseline MOC results reviewed Key questions pulled for ongoing pulse surveys Surveys following major communication events Weekly review of Q&A and Issues raised to

identify gaps in communication program Informal discussions within stores organization

Didn’t overreact to anecdotal information Checked feedback against data

Performance Improvement ProgramPerformance Improvement Program

StrategyStrategyMarketingMarketing Supply Chain

Supply Chain

Store & Field

Store & Field

Store & Field Organization

Store Activity

In Store Marketing

Labor Management

Sign Creation

Flow Optimization

Integrated Order Management

Liability Inventory

Ad Production

Promotional Profitability

Lands’ End Initiative

Covington Roll out

Center Core

Home Fashions

Footwear

Appliances

HRHR

e-Learning

e-Recruiting

Associate Services

Projects that have a direct impact on the store were grouped together into a synchronized field “Conversion” calendar

Conversion ScopeConversion Scope

Store Structure District Structure Region Structure

Number of Stores

Center Aisle Cashwraps ALL Signage and Fixturing ALL Four Zone Merchandising ALL Appliances 101 Center Core 543 Home Accents 557 Mattresses 32 Men’s Big and Tall 349 Open Sell Shoes 216 Tool Territory 149

Store Structure District Structure Region Structure

Number of Stores

Center Aisle Cashwraps ALL Signage and Fixturing ALL Four Zone Merchandising ALL Appliances 101 Center Core 543 Home Accents 557 Mattresses 32 Men’s Big and Tall 349 Open Sell Shoes 216 Tool Territory 149

Merchandise and Customer Assist Consultative Selling Center Aisle Cashiering

POS Signing Ad Setup/Take-down Replenishment POG/ZOG Liability Merchandise Labor Management e-Learning e-Recruiting

Merchandise and Customer Assist Consultative Selling Center Aisle Cashiering

POS Signing Ad Setup/Take-down Replenishment POG/ZOG Liability Merchandise Labor Management e-Learning e-Recruiting

Note: initiatives will be implemented in all stores except where a specific number is given

Field Organization

Field Organization

Store Environment

Store Environment

Service LevelsService Levels

Activities and Processes

Activities and Processes

Communication OutputCommunication Output

PMO website has had more than 2-million hits during conversion; Of the 24,000 visitors 23,000 are from the field.

PMO base conversion has been successfully implemented, and has been widely supported

Articles every month in corporate publications Communication is becoming embedded as a core

leadership responsibility in the stores organization Web site is becoming an operational tool more than

information only tool

PMO website has had more than 2-million hits during conversion; Of the 24,000 visitors 23,000 are from the field.

PMO base conversion has been successfully implemented, and has been widely supported

Articles every month in corporate publications Communication is becoming embedded as a core

leadership responsibility in the stores organization Web site is becoming an operational tool more than

information only tool

Communication Results - AssociatesCommunication Results - Associates

Associate attitudes (all store associates) are more positive than they have been since 1998.

Understanding of strategy is higher than it has ever been.

Associate attitudes (all store associates) are more positive than they have been since 1998.

Understanding of strategy is higher than it has ever been.

50

55

60

65

70

75

80

19

99

20

00

20

01

Wa

ve

1

Wa

ve

2

Wa

ve

3

Assoc iate Attitudes% positive

Understand

Strategy

Feel good

about future

Making

changes to

compete

Store management is even more positive ... Store management is even more positive ...

Assistant Store Managers and hourly leads feel good about the future, believe we are making the changes necessary to compete, and understand our strategy.

Assistant Store Managers and hourly leads feel good about the future, believe we are making the changes necessary to compete, and understand our strategy.

50

60

70

80

90

100

Wave 1 Wave 2 Wave 3

UnderstandStrategy

Feel goodabout future

Makingchanges tocompete

Customer feedback is also encouragingCustomer feedback is also encouraging

Customers reacted favorably to the new store environment

Store revenues were on target for Holiday

Customers reacted favorably to the new store environment

Store revenues were on target for Holiday

Financial Benefits are on trackFinancial Benefits are on track

Conversion costs stayed on budget Field operating costs met planned savings Capital expenditures within budget Communication had no incremental budget increase Full Line Store profitability is up more than 30% Sears posted record earnings in 2002

Conversion costs stayed on budget Field operating costs met planned savings Capital expenditures within budget Communication had no incremental budget increase Full Line Store profitability is up more than 30% Sears posted record earnings in 2002

ProductivityProductivity

Objective is to create a culture of continuous productivity improvement

Ongoing process of continuous improvement in efficiency and effectiveness

Ultimately, time will tell Can we continue to improve both customer

satisfaction and productivity?

Objective is to create a culture of continuous productivity improvement

Ongoing process of continuous improvement in efficiency and effectiveness

Ultimately, time will tell Can we continue to improve both customer

satisfaction and productivity?

ConclusionsConclusions

Looking at measurement from the perspective of the CEO will help clarify what’s important

Establishing a logic model for communications and measurement of results is good start

Building evidence over time will help improve ability to make fact-based decisions about public relations and communication investments

In some areas (such as marketing public relations) we can remove the mystery of the craft and earn respect of management team on their terms

There is still much to do

Looking at measurement from the perspective of the CEO will help clarify what’s important

Establishing a logic model for communications and measurement of results is good start

Building evidence over time will help improve ability to make fact-based decisions about public relations and communication investments

In some areas (such as marketing public relations) we can remove the mystery of the craft and earn respect of management team on their terms

There is still much to do

SEARS, ROEBUCK AND CO.SEARS, ROEBUCK AND CO.

Questions? Questions?

Tom Nicholson

Sears, Roebuck and Co.

[email protected]

Tom Nicholson

Sears, Roebuck and Co.

[email protected]

Measuring Communication Success

Documents

Transcript of Measuring Communication Success