Garbage in, garbage out, presented by David Rabjohns

SOCIALMEDIA.ORG/SUMMIT2016ORLANDOJANUARY 25–27, 2016

Garbage in, garbage out: What you don’t know could be destroyingyour measurement accuracy

DAVID RABJOHNSMOTIVEQUEST

GARBAGE IN GARBAGE OUT

What you don’t know could be destroying your measurement accuracy.

Social Data

2 ©MOTIVEQUEST 1/25/16


MOTIVEQUEST: We have been doing this for over ten years. I am going to share the secrets we have learned.

COVERAGE

MODELS

CUSTOMER SERVICE

The most strongly impassioned conversations about carrier loyalty were about customer service, models and coverage.

Shhhhh don’t tell anyone……

Q. What affects accuracy?


CONTENT

Data Terms Organization Analytics

HIGH QUALITY

HIGH QUALITY

INSIGHTS

=


DATA SOURCES: Better inputs give better insights.


SOURCE UNDERSTANDING: Helps improve insights and media strategy

Social Networks Best for understanding how consumers interact with brands

Micro-blogs Best for monitoring news, memes, and content of interest

Personal blogs Fairly good for deep behavioral insights, but watch spam

Multi-media sites Best for complementary behavioral insights

Best for deep behavioral insights Forums, Message Boards

Photo sharing sites Best for complementary behavioral insights


SOCIAL MEDIA: Consumers are discerning in where they talk. We must be discerning in where we listen.


KEY QUESTION: What is the data being used for?

SOCIAL MONITORING HUMAN INSIGHTS

Focuses on Twitter and Facebook Tracks and measures shares, likes, retweets Emphasizes real-time updates Informs PR and social strategy Looks at the “now and ongoing” Uncovers social media usage patterns

SOCIAL MEDIA

MEASUREMENT

Focuses on forum, board, and blog communities

Tracks and measures opinions and attitudes

Emphasizes deep understanding

Informs brand positioning and communication

Allows historical trending and comparisons

Uncovers attitudes and emotions

VS


CONTENT


HIGH QUALITY


CONSUMER-LED TAXONOMY: The data also needs to be organized to help with the analysis.

LIFESTYLE 34 million

PARENTING 40 million

AUTOMOTIVE 43 million

GAMING 68 million

TECHNOLOGY 42 million

CAUSES 23 million

HOBBIES 29 million

OUTDOORS 25 million

FITNESS 20 million

TRAVEL 15 million

PROFESSIONALS 26 million

PERSONAL FINANCE 14 million

Our 20-cohort taxonomy includes groupings such as the following (shown with annual conversation volume):


CONSUMER COHORTS: By following consumers’ own natural clustering behavior, you can quickly identify unrecognized segments and product opportunities.

SHA

RE

OF

PIC

KLE-

TALK


BEYOND BRANDS: By harvesting communities in their entirety, you are able to look beyond brands, to conversations about behaviors.

Brand Mentions 5%

No Brand Mentions 95%

FOOD

Brand Mentions

50%

No Brand Mentions

50%

FINANCIAL SERVICES


Brand Mentions 11%

HEALTH & MEDICAL


Brand Mentions 14%


WordMaps surface words that characterize conversation about a chosen topic by comparing the frequency of word usage in conjunction with the chosen topic with the words’ prevalence in the broader conversation. The relative size of the bubbles indicates the relative frequency of word usage. Proximity to the center term indicates strength of correlation.

NEW CONTENT: For example Kraft-Heinz’ brands identify new consumer behaviors and create new content around them.


CONTENT


HIGH QUALITY


LINGUISTIC STRINGS: Strong search terms filter out false positives and incorporate consumers’ misspellings and acronyms.

bacon cholesterol

AND returns will include “bacon” and “cholesterol”

bacon sausage

OR returns will include either “bacon” or “sausage”

bacon kevin

AND NOT

returns will include “bacon” but not “kevin”


((("Canon" OR canons) AND (camera OR cameras OR "point and shoot" OR "slr" OR "dslr" OR mirrorless OR cam OR cams)) OR ((A1100 AND (Canon OR Canons OR camera OR cameras)) OR A1100IS) OR ((A3000 AND (Canon OR Canons OR camera OR cameras)) OR A3000IS) OR ((A3100 AND (Canon OR Canons OR camera OR cameras)) OR A3100IS) OR (A480 AND (Canon OR Canons OR camera OR cameras)) OR (A490 AND (Canon OR Canons OR camera OR cameras)) OR (A495 AND (Canon OR Canons OR camera OR cameras)) OR (D10 AND (Canon OR Canons OR camera OR cameras)) OR (G11 AND (Canon OR Canons OR camera OR cameras)) OR (G12 AND (Canon OR Canons OR camera OR cameras)) OR ("PowerShot" OR "powershots" OR "power shot" OR "power shots") OR (S90 AND (Canon OR Canons OR camera OR cameras)) OR (S95 AND (Canon OR Canons OR camera OR cameras)) OR ((SD1200 AND (Canon OR Canons OR camera OR cameras)) OR "SD1200IS" OR ("sd 1200" AND (Canon OR Canons OR camera OR cameras)) OR "sd 1200IS") OR (("SD1300" AND (Canon OR Canons OR camera OR cameras)) OR "SD1300IS" OR ("sd 1300" AND (Canon OR Canons OR camera OR cameras)) OR "sd 1300IS") OR ((SD1400 AND (Canon OR Canons OR camera OR cameras)) OR "sd1400IS" OR ("sd 1400" AND (Canon OR Canons OR camera OR cameras)) OR "sd 1400IS") OR (("SD3500" AND (Canon OR Canons OR camera OR cameras)) OR "sd3500 IS" OR ("sd 3500" AND (Canon OR Canons OR camera OR cameras)) OR "sd 3500IS") OR ((SD4000 AND (Canon OR Canons OR camera OR cameras)) OR "SD4000 IS" OR "sd 4000 is" OR ("sd 4000" AND (Canon OR Canons OR camera OR cameras))) OR ("SD4500IS" OR (SD4500 AND (Canon OR Canons OR camera OR cameras)) OR ("sd 4500" AND (Canon OR Canons OR camera OR cameras)) OR "sd 4500is") OR (("SD780" AND (Canon OR Canons OR camera OR cameras)) OR "sd780IS" OR "sd 780IS" OR ("sd 780" AND (Canon OR Canons OR camera OR cameras))) OR (("SD940" AND (Canon OR Canons OR camera OR cameras)) OR "sd940IS" OR ("sd 940" AND (Canon OR Canons OR camera OR cameras)) OR "sd 940is") OR ((SD960 AND (Canon OR Canons OR camera OR cameras)) OR "sd960IS" OR ("sd 960" AND (Canon OR Canons OR camera OR cameras)) OR "sd 960IS") OR (("SD980" AND (Canon OR Canons OR camera OR cameras)) OR "SD980 IS" OR ("sd 980" AND (Canon OR Canons OR camera OR cameras)) OR "sd 980IS") OR ((sx120 AND (Canon OR Canons OR camera OR cameras)) OR "sx120is") OR (("SX130" AND (Canon OR Canons OR camera OR cameras)) OR sx130is) OR (("SX20" AND (Canon OR Canons OR camera OR cameras)) OR sx20is) OR (("SX200" AND (Canon OR Canons OR camera OR cameras)) OR sx200is) OR (("SX210" AND (Canon OR Canons OR camera OR cameras)) OR "sx210is") OR (("SX30" AND (Canon OR Canons OR camera OR cameras)) OR SX30IS) OR((Rebel OR Rebels) AND (EOS OR Canon OR Canons)) OR (t1i OR t1ief OR t1iefs) OR (t2i OR "t2ief" OR t2iefs) OR ("rebel x") OR ("Rebel XS") OR (EOS OR EOSs) OR ("EOS 1D" OR (Canon AND 1D)) OR ("EOS-1Ds" OR (Canon AND 1Ds)) OR ("EOS-50D" OR (Canon AND 50D)) OR ("EOS-5D" OR (Canon AND 5D) OR 5D2) OR ("EOS-60D" OR (Canon AND 60D)) OR ("EOS-7D" OR (Canon AND 7D) OR 7D2))

EXAMPLE: It’s not for everyone, but it’s worth spending a lot of time on linguistics and creating an internal library.


CONTENT


HIGH QUALITY


ANALYTICS: Lots of data does not predict behavior.

MODEL B ADVOCACY & SALES RELATIONSHIP

Model D Units Sold

Model D Share of Advocacy

MotiveQuest Online Promoter Score Developed in collaboration with Northwestern University.


$150,000

$160,000

$170,000

$180,000

$190,000

$200,000

$210,000

$220,000

$230,000

$240,000

Actual Average Store Sales Advocacy-Driven Sales Prediction

*The P-Value of the relationship between sales and advocates is 0.002

RECOMMENDATIONS ANALYSIS: Works in low interest categories also


ADVOCATES CORRELATED BEST WITH SALES: Our regression suggests that Advocacy is far more likely to drive sales than Detraction or Net Sentiment.

Variable measured against MONTHLY SALES P-Value

Monthly Advocates .04

Monthly Detractors .11

Monthly Net Sentiment .40

•  A P-Value is a measure of how likely it is that the relationship between the variables could simply be noise in the data. Lower is better.

•  The higher the value, the more likely it is that the relationship is attributable to randomness, and relationships where the value is greater than 0.1 are considered spurious.

•  When the P-value is between 0.01 and 0.05, there is a strong presumption that there is a real relationship.

Bett

er


REMEMBER: Approach research like scientists. They look for the presence or absence of evidence supporting hypotheses, rather than just reporting the news.


“There is nothing new under the sun, only

old things we don’t know.” Ambrose Bierce

MOTIVEQUEST David Rabjohns

[email protected] @MotiveQuest

www.motivequest.com

Learn more about past andupcoming events

SOCIALMEDIA.ORG/EVENTS

SOCIALMEDIA.ORG/SUMMIT2016ORLANDOJANUARY 25–27, 2016

Garbage in, garbage out, presented by David Rabjohns

Social Media

Transcript of Garbage in, garbage out, presented by David Rabjohns