Evaluating the Effectiveness of your Media Relations Strategies Media Relations in the Digital Age ConferenceJon Wade – Head of Digital, Asia PacificWeber Shandwick

I’m just back from scuba diving in the Philippines…

Measurement is pretty crucial in scuba diving:• It tells you when you’re

about to run out of air• It tells you if you’re likely

to get nitrogen narcosis, oxygen toxicity or ‘the bends’

• It essentially stops you dying!!

Whilst, PR measurement isn’t quite in the same life-or-death category, nor the most fascinating subject matter on the planet…

…it’s still a vital component of our

communications activities because it

allows us to measure and optimise the

effectiveness of our activities

Contents• Introduction• Barcelona Principles• Measurement Framework• Prioritising Measurement• Measurement Tools• A bit of fun to end…• Q&A

“Barcelona Principles” for PR measurement

Association for Measurement & Evaluation of Communications

Institute for Public Relations

7 ‘Barcelona Principles’1. Importance of goal setting and measurement2. Measuring the effects on outcomes is

preferred to measuring outputs3. The effect on business results can and should

be measured where possible4. Media measurement requires quantity and

quality5. AVEs are not the value of Public Relations6. Social Media can and should be measured7. Transparency and replicability are paramount

to sound measurement

Principle #1: Importance of goal setting and measurement

Implication: If this is

true, we need to

invest both time and

budget in the goal

setting and

measurement aspects

of our programme

• Goal-setting and measurement are fundamental aspects of any public relations programme

• Goals should be as quantitative as possible and address who, what, when and how the PR program is intended perform

A couple of acronyms that can help us with Principle #1

S.M.A.R.T.

R.E.A.

Regarding our objectives:

Regarding our communication goals:

S.M.A.R.T.

pecificeasurablechievableealisticimed

Our objectives should be:

Are our objectives clearly defined?

Is it possible to measure the objectives?

Are the objectives even possible?

Do we have the resources / time to deliver?

What is the timescale?

Reach

Engagement

Action

Are our communications primarily concerned with generating…?

Are we trying to increase product / brand awareness?

Are we trying to increase product / brand consideration or willingness to buy

Are we trying to increase product sales?

A perfect client brief (from a measurement point of view)

We’re launching our new <insert product / brand>.The key business objective for the PR campaign is to raise awareness amongst <insert target audience> from the current 20% awareness level to 30% within 6 months as measured by an online survey.

S.M.A.R.T. R.E.A.☐ ☐

• Outcomes include shifts in awareness, comprehension, attitude and behavior related to purchase, donations, brand equity, corporate reputation, employee engagement, public policy, investment decisions with respect to a company, NGO, government or other entity.

• Practices for measuring the effect on outcomes should be tailored to the business objectives of the PR activities.

Quantitative measures preferred However, qualitative methods to

supplement

Principle #2: Measuring the effect on outcomes is preferred to measuring outputs

Business Objective

Campaign Objective

Example Measures

Tools

Increased product awareness

Reach • Exposure / Coverage

• Earned CPMs

• Social Listening Platforms

• Reach Monitoring Tools

Increasedproduct consideration

Engagement • Comments• Opinions• Sentiment• Shares / Likes• Searches• Site Traffic• Net Promoter

Score

• Engagement monitoring tools

• Search demand tools

• Opinion analysis platforms

• Web Analytics• Surveys• Quant NPS studies

Maximising product sales

Action • Site Traffic• Sales (online /

offline)• ROI

• Sales / CRM DB• Web Analytics• Econometric

Analysis

Measures & Tools

But of course it’s not always possible to measure everything…

…so we need a way of

priortising our measurement

programme.

Pick one primary business objective only

One method:

1

Improving Brand Reputation

Let’s take the example where the key business objective we are primarily trying to manage through our media relations strategy is:

And based on our earlier framework, to achieve this objective, our tactics and measurement are going to be primarily concerned with driving engagement with consumers:

1) Having them engage with the brand online and share / recommend the brand to people within their network

2) Having their opinion of the brand become more positive

3) Having them use positive language in their conversation about the brand online

If these are our objectives, we would consider the following measures:

1. The Brand’s Net Promoter Score2. Opinions of the brand expressed

online3. Sentiment of online

conversations involving our brand

The exact methods we select will depend on the associated costs and budgets available for

measurement

Reputation(NPS)

Long-term Profitability & Growth

Q: “How likely are you to recommend this brand to your friends and family?”

NPS = % Advocates - % Detractors(0 – 10 scale; 0 – 6 Detractors; 7 – 8 Neutral; 9 – 10

Advocates)

Net Promoter Study

Engagement – NPS Correlations

Engagement – NPS Trends & Benchmarks

Engagement – Reputation Survey

Engagement – Opinion, Crimson Hexagon

Sentiment Monitoring

• To measure business results from consumer or brand marketing, models that determine the effects of the quantity and quality of PR outputs on sales or other business metrics, while accounting for other variables, are a preferred choice. Related points are:

Principle #3: The effect on business results can and should be measured where possible

Clients are creating demand for market mix models

The PR industry has to understand the value and implications of market mix models for accurate evaluation of consumer marketing PR

The PR industry needs to develop PR measures that can provide reliable input into market mix models

Marketing Mix ModelsSALES = a + b[TV$] + c[PR$] + d[DM$] + e

Where: a = ‘Natural’ Salesb = Ratio of TV spend to incremental salesc = Ratio of PR spend to incremental salesd = Ratio of DM spend to incremental salese = margin of error

How can you split out the effects of PR for this type of data?

Control

Exposed

Cookie = 0 Cookie =

1

Online techniques provide tools to do this for digital PRExample: We can drop a custom cookie to the audience who visits our Facebook properties

By surveying a sample of those that have a cookie vs those that haven’t we can split out the effect of our Facebook efforts.

Overall clip counts and impressions are usually meaningless. Instead, media measurement, whether in traditional or online channels, should account for:• Impressions among the stakeholder or audience• Quality of the media coverage including:

Tone Credibility and relevance to the stakeholder or audience Inclusion of key messages Inclusion of a 3rd party or company spokesperson

• Quality can be negative, positive, or neutral

Principle #4: Media measurement requires quantity and quality

Verisign Example

Automated sentiment measurement allows us to to get monitor the quality of the coverage we are getting as well as early warning of potential issues and crises

Firebell is an online crisis simulation programme that allows Weber Shandwick client’s to practice responding to an online crisis in a safe offline environment

Reach – Klout

Reach – SNS

Reach – DoubleClick AdPlanner

• Advertising Value Equivalents (AVEs) do not measure the value of public relations and do not inform future activity; they measure the cost of media space and are rejected as a concept to value public relations

• Where a comparison has to be made between the cost of space from earned versus paid media, validated metrics should be used, stated for what they are, and reflect:

Principle #5: AVEs are not the value of Public Relations

negotiated advertising rates relevant to the client, as available

quality of the coverage, including negative results

physical space of the coverage, and the portion of the coverage that is relevant

• Multipliers intended to reflect a greater media cost for earned versus paid media should never be applied unless proven to exist in the specific case

• There is no research to back-up the claim that a news story and an advertisement have an ‘equivalent’ effect– News media credibility varies by story (positively &

negatively)– Has been dropping in past 2 decades

• Often the goal of PR is to reduce publicity…this cannot be measured using AVEs

• Ads are usually designed to work at frequency, news stories tend to be relatively unique

• There isn’t always an equivalent advertising rate – front covers, coverage during news bulletins with no ad breaks, coverage on sites that don’t accept advertising etc

• There is no advertising equivalent of a ‘bad’ or ‘unfavourable’ story – should we ignore those?

The problem with AVEs

Activities 47 Media, Blogger & Influencer Interviews

Reach 170 Earned & Social Media Placements

Relevance 64% Earned & Social Message Penetration

Outcomes 14% Increase in Brand Engagement

Worth $4.72 Earned CPM (Cost Per 1K Impressions)

94 Facebook, YouTube, Blog & Twitter Posts

3.9M Earned & Social Media Impressions

27% Earned & Social Media Share

27% Category Sales Share

$8.22 Social CPE (Cost Per Engagement)

Measuring Worth

Earned CPMs

Total Earned ImpressionsTotal Cost of Earned Coverage X 1000

Total EngagementsTotal Cost of Earned

Engagement

Cost Per Engagement

Engagement – PostRank

• Social media measurement is a discipline, not a tool; but there is no “single metric”

• Media content analysis should be supplemented by web and search analytics, sales and CRM data, survey data and other methods

• Evaluating quality and quantity is critical, just as it is with conventional media

• Measurement must focus on “conversation” and “communities” not just “coverage”

Principle #6: Social Media can and should be measured

• Understanding reach and influence is important, but existing data sources are not accessible, transparent or consistent enough to be reliable; experimentation and testing are key to success

Engagement – Google AdWords

Engagement – Google Trends

Engagement & Action – Web Analytics

PR measurement should be done in a manner that is transparent and replicable for all steps in the process, including specifying:Media Measurement:• Source of the data (print, broadcast, internet, consumer generated media) along

with criteria used for collection

Principle #7: Transparency and replicability are paramount to sound measurement

Surveys:• Methodology – sampling frame and size,

margin of error etc• Questions –should be released as asked

(wording and order)• Statistical methodology- how specific metrics

are calculated

• Analysis methodology – for example, whether human or automated, tone scale, reach to target, content analysis parameters

Activities Reach Relevance Outcomes Worth

What activitieswere performed

to achieveresults?

Did you reachyour audience?

How manyimpressions,web visits,

reports,attendees, etc.

weregenerated?

Were yourelevant to youraudience? Were

you credible? Did your ideas and messages resonate? Did

you drive conversation?

What business results did you

achieve? Awareness?

Engagement?Reputation?

Leads? Sales? Loyalty?

Advocacy?

What is the estimated value

of your communication

efforts?

Communications Team Marketing Team Executive Team

Quantity/Output Quality/Outtakes Business Impact Value/Efficiency

ARROW: Flexible, Integrated Measurement Framework

Activities 47 Media, Blogger & Influencer Interviews

Reach 170 Earned & Social Media Placements

Relevance 64% Earned & Social Message Penetration

Outcomes 14% Increase in Brand Engagement

Worth $4.72 Earned CPM (Cost Per 1K Impressions)

94 Facebook, YouTube, Blog & Twitter Posts

3.9M Earned & Social Media Impressions

27% Earned & Social Media Share

27% Category Sales Share

$8.22 Social CPE (Cost Per Engagement)

Light ARROW Scorecard

Full ARROW Scorecard

OK – hopefully that wasn’t too dull…To end – a bit of fun…

Data Visualisations

Data doesn’t have to be boring…

…in fact the way you represent the data

can make the difference between

whether your audience engages

with it or not.

Jonathan Harris and Sep Kamvar – We Feel Fine

Thanks for listening.

Any Questions?

Jon WadeHead of Digital, APAC – Weber Shandwick+852 9789 8973@digital_apacjwade@webershandwick.com