Team Narrative Mind

Team Info

The Narrative Mind team contains experts in software engineering, social media design, and web-based information operations (IO). We seek to develop tools that will optimize discovery and investigation of communication trends on social media.

Weekly Total: 10 interviews Users: 21Experts: 50Buyers: 5Cumulative Total: 76 interviews

Hypotheses ❏ ARCYBER will find desired capabilities in existing software ❏ Hashtag co-occurrences may indicate larger narrative trends, but are not likely to be

sufficient in isolation

Experiments ❏ Demo’s of several commercial tools to ARCYBER sponsors❏ Analysis of hashtag data, each co-occurrence coded into narrative theme bucket

Results❏ Commercial tools viewed did not meet ARCYBER’s needs, too narrow❏ Analysis of hashtag data by itself does not present a holistic picture of narrative trends (text

limits; sample size)

Actions 1. Set up follow-on demo to show ARCYBER one remaining commercial tool with capabilities that seem to fit their social media network analysis needs

2. Further analysis of larger hashtag datasets acquired from researchers, run tests using hashtag co-occurrence for primitive categorizing abilities

Customer Discovery

Bg. General (decision maker)

❏ Quickly understand key thematic points of group’s use of social media put out by intelligence briefs.

MAJ/LTC/COL (operational planer)

❏ Determine what types themes are rising in popularity and better identify type of response

Analysts/Operators (actionable insights)

❏ New adversary IO movements can be understood and tracked with less direct cooperation of cultural/language experts.

ARCYBER -- Mission Achievement

ARCYBER/COCOMS

Big Picture Success Analogy:

“Most COCOMs and IO shops spend their whole day looking for a needle in a haystack: a user, a

post, an IP address. For narrative-level awareness, we need a strategy that helps us divide the

haystack into a bunch of smaller haystacks that don’t all look like same damn pile of hay.”

Private sector -- Mission Achievement

Brands(CMO, Marketing, SM Analysts)

❏ Analysts: Provide proactive information that captures emerging stories in previously targeted consumer groups, instead of just monitoring marketing plan successes.

❏ Marketing Managers: Get a higher-level overview of grassroots-level content and leverage trending topics.

Political Campaigns(Campaign Managers, Community Organizers, SM Analysts)

❏ Social Media campaign managers: Determine what types themes are rising in popularity and better identify type of response

Hedge Funds(Executives, Analysts)

❏ Hedge fund analysts: Correlate narrative growth of portfolio companies with changing fluctuations in valuation/revenues, influence narratives around companies (long/short)

Private sector

All the Hypotheses

Global

Tweet-level

Awareness Response

IO Org ChartNo baseline for monitoring/aggregating use of tech; cyber targeting

Automatic Narrative DetectionLanguage/culture experts aren’t able to work at scale

Important Event PredictorPreempt events

Counter-Narrative GeneratorDoD/Gvt. social media presence is weak

Site ScraperNeed more cached information access

Expedited Content CategorizationScale of social media makes manual efforts painful

Bot DetectorCan’t determine actual scale of following and support.

Virality PredictorUnderstand which narratives are the most salient

Persistent ID-Alias trackerAccount bans and multiple aliases across different networks make IDs hard to track

Annotate Speakers in VideosAnalysts don’t track which speeches and audio are used in videos.

Company FPersistent ID-Alias tracker

All the Hypotheses

Global

Tweet-level

Awareness Response

IO Org ChartNo baseline for monitoring/aggregating adversary use of tech; cyber targeting

Automatic Narrative DetectionLanguage/culture experts aren’t able to work at scale

Company CVirality Predictor

Counter-Narrative GeneratorDoD/Gvt. social media presence is weak

Company EFilesharing Site Scraper

Company DExpedited Content Categorization

Company BAdversary Bot Detector

Company AImportant Event Predictor

All the Opportunities

Global

Tweet-level

Awareness Response

Adversary IO Org ChartNo baseline for monitoring/aggregating adversary use of tech; cyber targeting

Automated Narrative DetectionLanguage/culture experts aren’t able to work at scale

Open Opportunities

The Visualization Problem

Source: hiveplot.net

Hashtag Co-occurrence MVP Testing

Dataset Features:

● 600k Unique Tweets ● Spanning October 2015 to May 2016● 200k Unique Hashtag Combinations

Procedure:

1. Merged records into frequency table of hashtag co-occurrences.

2. Manually coded 1300 most frequent hashtag sets based on themes.

3. Visualized volume of these hashtag sets over time as related to big-picture “narrative”.

Key Findings:

● Process is reasonably scalable.● Could be implemented quickly by

ARCYBER to supplement workflow.

● Deciding on narrative categories is difficult.

● Need to further condition input tweets to specific groups.

● How does hashtagged traffic compare to total traffic?

Customer Discovery - Get/Keep/Grow Diagram (ARCYBER)

Acquire Activate Keep Up-sell Next-sell Cross-sell Referral

Activity & People

-SIG (Strategic Initiatives Group), C5 consortium, ACT (Advanced Concepts and Technology directorate)

-LTC, Col seeking better visibility/reports on use of social media and power to deploy

-Social media analysts (CW3’s--information aggregators)

-LTC, Col receiving reports synthesized by CW3’s need advanced, additional features for in-depth views/analysis

?? -Expanding offering with additional analysis/visualization tools and integrations

-Expanding user base across CYBER ops/wings beyond ARCYBER

Metrics

-inbound interest from across ARCYBER and IC

# people who say “we want this”

-time spent with tool -# additional features bought per user

?? # other services being used per user

# payers using tool

Customer Discovery - Get/Keep/Grow Diagram (Politics)

Acquire Activate Keep Up-sell Next-sell Cross-sell Referral

Activity & People

-political campaigns -work with campaign social media managers

-provide high-level strategic understanding of\’ content and own narrative success

-sell advanced, additional features for in-depth views/analysis

?? -Expanding offering with additional analysis/visualization tools and integrations

-get other political allies on platform, get whole parties (DNC, RNC)

Metrics

-inbound interest from campaigns

# people who say “we want this”

-time spent with tool -# additional features bought per user

?? # other services being used per user

# payers using tool

Acquire Activate Keep Up-sell Next-sell Cross-sell Referral

Activity & People

-national consumer brands -work with social media/content managers

-provide high-level strategic understanding of competitors’ content and own narrative success

-sell advanced, additional features for in-depth views/analysis

?? -Expanding offering with additional analysis/visualization tools and integrations

-get other parent/subsidiary brands on platform

Metrics

-# earned demand leads to # paid demand leads

# people who say “we want this”

-time spent with tool -# additional features bought per user

?? # other services being used per user

# payers using tool

Customer Discovery - Get/Keep/Grow Diagram (Brands)

Customer Discovery - Get/Keep/Grow Diagram (Hedge funds)

Acquire Activate Keep Up-sell Next-sell Cross-sell Referral

Activity & People

-Hedge fund managers and analysts (small-cap to large-cap)

-provide turnkey solution for analytics

-provide high-level strategic understanding of companies’ content and narrative success or failure

-sell advanced, additional features for in-depth views/analysis

?? -Expanding offering with additional analysis/visualization tools and integrations with news aggregators

-get other hedge funds, but more difficult because of zero-sum outlook

Metrics

-inbound interest from hedge funds

# people who say “we want this”

-$’s generated -# additional features bought per user

?? # other services being used per user

# payers using tool

Mission Model Canvas

- Track how groups propagate narratives with co-occurring hashtags.

- Gnip/Twitter/Facebook

- CrowdFlower, Samasource, or Mechanical Turk

- Pre-existing social media service and micro-labor aggregators

- Third-party access platforms for social media, -Data visualization-Content analysis platforms

PrimaryARCYBER-Bg. General (decision maker)-MAJ/LTC/COL (operational plan)-Analysts/Operators (actionable insights)

COCOMs-General (decision maker)-MAJ/LTC/COL-Analyst/Operator

SecondaryPolitical Campaigns-Campaign managers-Supporters

Consumer Brands-CMO-Public Relations Team

OTA process

Sell to political campaign managers

Sell to PR firms

- Optimize workflow for social media analysts. -Deliver insights to commanders about online environment.

-Insights into responses to broadcasting narratives

-Detect narratives emerging in real time

-Enable faster problem awareness to problem response times for decision-makers across organizations

- MechanicalTurk or crowdsourcing labor (microtasks)- UI Development/Testing with ARCYBER analysts.- Software Development- Research aggregation? (e.g. Import and reformat existing knowledge about extremist organizational structure)

- Access to Twitter firehose or API

- Local language speaking crowdsourcing staff.

- Accurate testing for intercoder reliability

- ARCYBER: Bg. General, LTC, Strategic Initiatives Group, OTA, Purchasing PMs, End-User operator)

-COCOMs: OTA, Operators, Purchasing PMs

-Political Campaign: Opposition research team, ???

-Private Sector: CMO, ???

Beneficiaries

Mission AchievementMission Budget/Costs

Buy-In/Support

Deployment

Value Proposition

Key Activities

Key Resources

Key Partners

Value Proposition Canvas - ARCYBER

Products& Services

Web/Desktop Application Provide high-level

overview of groups, update research

Customer Jobs

- New mission priorities.- Little familiarity with overall environment.- No repository for high-level learnings

Gains

Pains

Gain Creators

Pain Relievers

- New opportunities

-Illustrates evolution of info dissemination networks- Provides insight for mapping “online terrain”

Visualize information about how organizations are using social media

ARCYBER - Operator/Analyst (Actionable Insights)

Value Proposition Canvas - ARCYBER

Products& Services

Web/Desktop Application

Aggregate insights from analysts,

compile into plan passed to decision

maker

- New mission priorities.- Proliferating/viral social media platforms- Urgency of issues ability to prioritize

Customer Jobs

Gains

Pains

Gain Creators

Pain Relievers

-Faster turnaround time with better data gained from information awareness software and technology

ARCYBER - MAJ/LTC/COL (Operational Plans)

-Better data in informing operational plans provided to decision makers-Reducing uncertainty in operational planning

Value Proposition Canvas - ARCYBER

Products& Services

Web/Desktop Application

Act on reports and plans generated

about various cyber threats across globe

- Little understanding of ground-level nuances- Little understanding of proliferating platforms

Customer Jobs

Gains

Pains

Gain Creators

Pain Relievers

- Organizational chart of technology broken down by functional purpose- Timeline-based viewer

-Reduce uncertainty of decision making-Add methodological rigor to ARCYBER’s offensive/defensive operations

ARCYBER - Brigadier General (Decision Maker)

Value Proposition Canvas - COCOM

Products& Services

Web/Desktop Application

Target audience analysis, finding opportunities to

optimize influence operations

Customer Jobs

-Loss of visibility on key communicators (when accounts shut down/opened)- No repository for high-level learnings

Gains

Pains

Gain Creators

Pain Relievers

-Auto-populated list & real-time visualization of key accounts, dissemination networks

-Allows more accurate-Illustrates evolution of info dissemination networks- Provides insight for mapping “online terrain”

Closure of visibility gap on key accounts

COCOM - Operator/Analyst (Actionable Insights)

Value Proposition Canvas - COCOM

Products& Services

Web/Desktop Application

-Long term operational planning- Measuring effectiveness

Customer Jobs

- Unclear picture of how some networks evolve, adapt over time

Gains

Pains

Gain Creators

Pain Relievers

- Organizational chart of technology broken down by functional purpose- Visualization of key accounts, dissemination networks over time

-Illustrates evolution of info dissemination networks- Provides insight for measuring operations, mapping “online terrain”

Greater awareness of network evolution improves planning, indicators of operational effectiveness

COCOM - Maj/LTC/Col. (O4-O6) (Operational Plans)

Value Proposition Canvas - COCOM

Products& Services

Web/Desktop Application

- Judge potential effect and risk of

operations- Approve/reject

plans

Customer Jobs

- Less familiarity with internet as an operational environment- Needs high level summary of how users use technology

Gains

Pains

Gain Creators

Pain Relievers

- Organizational chart of technology broken down by functional purpose- Timeline-based viewer

-Illustrates evolution of info dissemination networks- Provides insight for mapping “online terrain”

Visualize high-level information about how regional users use technology.

COCOM - Brigadier General (O7) (Decision Maker)

Value Proposition Canvas - Political Campaign

Products& Services

Web/Desktop Application

-Identify opposition narratives on SM

channels-Craft responses to

respond w/short turnaround

Customer Jobs

-Limited lead time in anticipating changes from opposition campaigns-New stories/themes can derail campaign momentum without a targeted response

Gains

Pains

Gain Creators

Pain Relievers

-Faster response time to opposition’s campaigns-Maintain the initiative in media environment by rapidly responding to threats

-Early detection of emerging opposition narratives on SM-Insight into evolving environment

Counter opposition themes quickly, identify newly spreading rumors that could undermine campaign.

Political Campaigns - Opposition Analysis/Campaign Director

Value Proposition Canvas - Consumer Product Company

Products& Services

Web/Desktop Application

-Monitor brand integrity and

customer themes-Identify and address emerging problems

Customer Jobs

-Identify themes about corporate brand, pinpoint problem areas early-Avoid negative narratives from becoming costly PR or marketing problems

Gains

Pains

Gain Creators

Pain Relievers

-List of customer narratives about brand that are being posted online-Proactive response to emerging brand issues, better customer satisfaction

-Early warning about negative themes-Identify problem areas, protect corporate brand from costly problems

-Protect brand integrity, create greater shareholder value by addressing customer concerns

Consumer Product Company - CMO/Marketing Organization