UN Global Pulse: Big Data for a Better World (Strata Conf NYC)

“BEYOND TARGETED ADS BIG DATA FOR A BETTER WORLD”

Robert Kirkpatrick Director, UN Global Pulse

O’Reilly Strata Conference | New York | October 2012

www.unglobalpulse.org @unglobalpulse

25 October 2012 | www.unglobalpulse.org

Photo Credit: John Oyuke

Kenyan Farm Workers

Hyperconnectivity

Airplane Flights

Internet Traffic

Telephone Calls

Social Networking

20th-Century tools…

PRIVATE SECTOR Monitor operations…in real time Track market trends…in real time

Get customer feedback…in real time

GLOBAL DEVELOPMENT Unemployment? Food Security? Public Health?

Education? Migration?

Disaster Relief?

1.  Better early warning: Earlier detection of anomalies, trends and events allows earlier response.

2.  Real-time awareness: A more accurate and up-to-

date picture of population needs supports more effective planning and implementation

3.  Real-time feedback: Understanding sooner where

needs are changing -- or are not being met -- allows for rapid, adaptive course correction

BIG DATA IN REAL TIME: 3 OPPORTUNITIES

BIG DATA IS A HUMAN RIGHTS ISSUE

•  Never analyze personally identifiable information •  Never analyze confidential data •  Never seek to re-identify individuals

RESEARCH BACKGROUND: A GROWING BODY OF EVIDENCE

How Mobile Phone Carriers See the World

Call Detail Records (CDRs) •  Caller ID (hashed phone #) •  Caller Tower Location •  Receiver ID (hashed phone #) •  Receiver Tower Location •  Call Start Time •  Call Duration

Airtime Expense Records •  Caller ID (hashed phone #) •  Caller Tower Location •  Amount of Purchase •  Time of Purchase •  Balance at Time of Purchase

Modeling Behaviors in Mobile Data

Consumption variables • Number of calls, call duration, SMS/MMS/voice • Size, frequency, total number of airtime purchases • Handset Type and Features

Social variables • Degree of the social network • Weight of the contacts, frequency of communication

Mobility variables • Diameter of mobility and social network • Radius of gyration • Mobility Patterns

Source: Telefonica Research, 2011 25 October 2012 | www.unglobalpulse.org

EXAMPLE: AIRTIME CREDIT PURCHASE DATA

Higher household income

Lower household income

Average # of purchases / month

SIZE AND FREQUENCY PREDICT HOUSEHOLD INCOME

Lower socioeconomic level

Higher socioeconomic level

CALLING PATTERNS AND ECONOMIC OPPORTUNITY

Men: •  Fewer calls •  Shorter calls •  Smaller social network •  More work-related calls

Women: •  More calls •  Longer calls •  Larger social network •  More personal calls

MEN AND WOMEN USE THEIR PHONES DIFFERENTLY

Tracking population movement to predict cholera

Source: Linus Bengtsson et. al., PLoS Medicine, 2011

A mobility index to evaluate H1N1 response in Mexico City

Source: Telefonica Research, 2011 See: http://www.unglobalpulse.org/publicpolicyandcellphonedata

dominican republic 11.7%

mexico 9.1% united kingdom 7.5%

canada 7.0%

guatemala 5.5%

ecuador 5.3%

jamaica 3.3%

india 3.0%

germany 2.5%

philippines 2.4%

TWITTER PREDICTS SPREAD OF INFLUENZA

r2 = .958

“You Are What You Tweet: Analyzing Twitter for Public Health. M. J. Paul and M. Dredze, 2011.” http://www.cs.jhu.edu/%7Empaul/files/2011.icwsm.twitter_health.pdf

Rumi Chunara et. al., American Journal of Tropical Medicine and Hygiene, 2012 86:39-45

GOOGLE SEARCHES FOR SYMPTOMS PREDICT DENGUE

2010 VS. 2011: INDONESIAN TWEETS ABOUT HIV

See: http://www.unglobalpulse.org/WorldAIDSDay-Part2 25 October 2012 | www.unglobalpulse.org

PROOF OF CONCEPT STUDIES

GLOBAL PULSE RESEARCH 2011

Online at: http://www.unglobalpulse.org/applyingbigdatatodevelopment

http://www.unglobalpulse.org/projects/can-social-media-mining-add-depth-unemployment-statistics

Online Discussions & Unemployment

Ireland

Online Discussions & Unemployment

United States

http://www.unglobalpulse.org/projects/twitter-and-perceptions-crisis-related-stress

Jakarta: nine million tweets per day

Map of Twi*er usage in Jakarta – by Eric Fischer 25 October 2012 | www.unglobalpulse.org

Tweets per day about food, during Ramadan in Indonesia

Start of Ramadan

End of Ramadan

Number of tweets per month commenting on the price of rice

Wordcloud1 (sunday)

Wordcloud2 (Sunday)

Food inflation

Data inflation

Tweets about the price of rice (per month)

Official Food Price Inflation (monthly from 25 cities)

Tweets predict food basket inflation (rice, chilies, fish, sugar, corn, cooking oil)

GLOBAL PULSE ABOUT

DIGITAL SERVICES AS HUMAN SENSOR NETWORKS: Observing fluctuations in well-being…in real-time

COPING STRATEGIES •  Buy cheaper foods •  Work longer hours •  Reduce energy use •  Draw down savings •  Sell assets •  Borrow from relatives

DIGITAL “SMOKE SIGNALS” •  Depletion of airtime credit •  Smaller mobile airtime

purchases •  Failure to repay microloans via

mobile financial services •  Changes in calling patterns •  Inbound money transfers •  Web searches for jobs, health •  Sales of livestock via mobile

trading network •  “Venting” on social media

Photo: Ministry of Foreign Affairs, Iceland

AGILE GLOBAL DEVELOPMENT?

•  This data may be less accurate that official sources.

•  But it’s faster. •  And it’s cheaper to

collect. •  How can we leverage

the speed to change the outcome?

Integrating real-time data into an institution

USGS Twitter Earthquake Detector (TED)

THE PROBLEM WITH TELESCOPES…

There’s a universe of data that we can’t see.

…AND MACROSCOPES

Data Philanthropy?

A global

real-time public/private

data commons?

EXAMPLE R&D PROJECT: Mobile Networks as Drought Sensors in the Sahel

Proposal •  Obtain 2011-2012 mobile CDRs and airtime purchases. •  Derive mobility, consumption, and social variables. •  Correlate variables with precipitation levels, survey

data. •  Identify signatures of drought impacts in 2011. •  Identify signatures of aid impact in 2012. •  Develop and evaluate prototype during next drought. •  Release open source “appliance” through GSMA.

PULSE LABS

Joint Research | Rapid Prototyping | Capacity Building

Pulse Lab Network

Pulse Lab Jakarta…………October 2012 Pulse Lab Kampala……….January 2013 Other locations…………...???

PULSE LABS R&D INNOVATION STRATEGY

1.  Partner with governments to establish Pulse Labs 2.  Build world-class teams of data scientists,

engineers, and policy experts 3.  Partner with private sector for real-time data and

cutting edge technology 4.  Work with UN agencies and academia to conduct

research around challenges in 5.  Build open source prototypes of tools to automatic

real-time monitoring 6.  Support broad adoption of useful innovations 7.  Share everything we learn and build

SO HOW DO I GET INVOLVED? ARE YOU.. •  A company with powerful data you think

could make the world a better place? •  A technology provider with screaming fast

computing or killer analytics? •  A whiz data scientist interested in hard

problems, positive impact, and global scale? •  A big data privacy expert who understands

that we cannot help unless we also protect? 25 October 2012 | www.unglobalpulse.org

Research Tool 1 Crimson Hexagon: ForSight

Food Prices: What a real crisis looks like

14 -‐ 21 Aug Ramadhan / Idul Fitri

23 July -‐ 2 Aug ‘tempeh’ and ‘tofu’ hot debate

Comparing Crises

23 July -‐ 2 Aug ‘tempeh’ and ‘tofu’ hot debate

during soybean shortage

18 Mar -‐ 7 Apr Fuel subsidy cut plan and

protests against it

Tweets about food

Research Tool 2 SAS Social Media Analytics and SAS Text Miner

Analytic Workflow

Internet Conversation

Global Pulse Relevance

Filter

Sentiment, Mood & Influence

Topic & Geography Categories

Interactive Dashboard

1) Over 200,000 new Indonesian language documents per day

3) Capture senKment and mood for Bahasa

2) Extract conversaKons about rice, cooking oil, fuel, employment, etc.

4) Detect locaKon, price, availability, specific govt. programs, etc.

5) Explore results and correlate with official staKsKcs to official BPS staKsKcs : Consumer Price Index (CPI) for 12 common foods

anxious, confident, confused, hosKle, sad, happy (-”-) ;-) ((+_+)) :@ :( :)

What’s the deal with Indonesians and eggs?

For every 5000 more tweets about eggs… …we see a 2-3% decrease in food CPI?

The Signals Are Getting Stronger

minyak (oil) ketahanan pangan (food security) budidaya (culKvaKon) telur (eggs)

è Big increase in volume of relevant conversations over 18 months

Indonesians are increasingly using social media to discuss basic needs

So Are the Temporal Correlations è Listening to social conversations provides insight on official data

-‐2.5

-‐1.5

-‐0.5

Social Media Food Index

BAPPENAS Food Price Index

Next up for Pulse Lab Jakarta research: 1 year of anonymized Indonesian CDRs

•  4 largest carriers •  170 million subscribers •  200 billion call records •  80 terabytes of data

Image credit: Aaron Koblin 24 hours of AT&T phone calls and Internet

traffic flowing through New York City

ROBERT KIRKPATRICK Director UN Global Pulse www.unglobalpulse.org kirkpatrick@un.org +1 (650) 796-5709

UN Global Pulse: Big Data for a Better World (Strata Conf NYC)

Documents

Transcript of UN Global Pulse: Big Data for a Better World (Strata Conf NYC)

Windtunnel Marketing (Internationalist Conf, NYC, April 2011)

STRATA LIFE SPRING 2015 - vic.strata.communityvic.strata.community/documents/Strata Life/Strata Life Spring 2015... · 1 Minister announces full review of Vic strata laws 2 Awards

The Strata Titles Act 1985 The Strata Titles (Amendment ...

Strata schemes management queensland colcan strata

Strata Facts Chapter3 the Strata Plan

Guardian Strata - ThinkFWD...Guardian Strata Microsoft ModernBiz Makeover Background Guardian Strata, based in Sydney, NSW, is a small company focusing on strata management. It has

Strata Reform | Landgate - Published May 2020...STA: Strata Titles Act 1985 as amended by the Strata Titles Amendment Act 2018. Scheme: a strata scheme or a survey-strata scheme. Consolidation

Strata Conference NYC 2013

P.U. (A) 107 STRATA MANAGEMENT ACT 2013 PART Ihorizonresidence.com.my/wp-content/uploads/2016/10/Strata-Management... · STRATA MANAGEMENT ACT 2013 STRATA MANAGEMENT (MAINTENANCE

THE INDUSTRY,AND STRATA STRATA HISTORY & RELEVANCE …

Strata Conference 2014 NYC with Twitter

Pivotal Strata NYC 2015 Apache HAWQ Launch

Google Cloud for Data Crunchers - Strata Conf 2011

Strata schemes management act victoria presentation dr strata management

Get involved - PSMG Strata | Strata Management Sydney1. Understandi ng the basics 5 • Key concepts in strata 5 • Strata scheme 5 – Common property 5 – Unit entitlement 6 –

Strata NYC 2015: Sketching Big Data with Spark: randomized algorithms for large-scale data analytics

Strata schemes management act new south wales presentation fresh strata

Strata schemes management queensland challenge strata mangement presentation

Strata schemes management act new south wales presentation strata professionals

UNDERSTANDING STRATA TITLING PRINCIPLES · 18/2/2019 1 UNDERSTANDING STRATA TITLING PRINCIPLES Sr Chan Keat Lim SEMINAR DAN BENGKEL HAKMILIK STRATA Why Strata Title is important?