eResearch Presentation - June '11

8/6/2019 eResearch Presentation - June '11

1/57

Harvesting the Crowd:

Experiments in Crowd-sourcing

Steven Gray

Research AssociateCentre for Advanced Spatial Analysis, UCL

Oxford DTC e-Research Seminar, June 2011


2/57

Mapper

Ability to ask any question

UK surveys down to postcode or 100metre grid

Realtime mapping

Data visualisation

Export to for integration with existingdatasets


3/57

Mapper

Born out of MapTube:A map portal developed at the Centre for Advanced Spatial Analysis


4/57

Mapper

Bespoke Mood Maps for BBC Radio 4, BBC Look East, BBC Look Northand BBC South


5/57

Mapper

Bespoke Mood Maps for BBC Radio 4, BBC Look East, BBC Look Northand BBC South


6/57

Mapper

Taking the Concept to the Cloud

Media Organisations

Government

Local Authorities

Community Organisations

Anyone...


7/57

Mapper

Creating an easy to use interface

Integrating the science behind the scenes

Allowing Global, National, US Zip Codes, Borough, Ward,Postcode or Pin Point maps

Quick and easy data input

Allowing people a free and open view on the the current issues


8/57

Mapper


9/57

Mapper.com

World/Nation/City/Borough/

Ward/Street

Survey Anything

Realtime Mapping/DataDownload

Used by The Mayor of London

BBC (currently running - 5000responses in 1 hour)

Scottish Water

The Public....


10/57

BBC Look East Survey - Broadband Speed Test


11/57

#5Acts4Wildlife Survey

- First SurveyMapper Survey integrated with our Twitter Collection Tools


12/57

Twitter API

http://dev.twitter.com/doc

User Profiles Tweet Database

API
http://dev.twitter.com/dochttp://dev.twitter.com/doc


13/57

Search API

HTT

PGET

Req

uest

Resu

lts

{"results":[

{"text":"@twitterapi is great!",

"to_user_id":396524,

"to_user":"TwitterAPI",

"from_user":"jkoum",

"metadata":

{

"result_type":"popular",

"recent_retweets": 100

},

"id":1478555574,

"from_user_id":1833773,

"iso_language_code":"nl",

....}

}

Returns tweets that: match specified query located in Lat/Lon box Historical Tweets User Profile Information Return Public Timeline

Service is rate limited

http://search.twitter.com/search.json?q=@twitterapi JSON or Atom
http://search.twitter.com/search.json?q=@twitterapihttp://search.twitter.com/search.json?q=@twitterapihttp://search.twitter.com/search.json?q=@twitterapihttp://search.twitter.com/search.json?q=@twitterapi


14/57

Streaming API

HTTP Response Stream 3 Levels of Service Different Tracking Services

Track Keywords Geo-Located Tweets Statistically Significant

Public Stream

Limitations

1 connection per IP Address Spritzer is default Stream

400 keywords

25 Lat/Lon Bounding Boxes


15/57

Streaming API



Public Stream

Limitations


400 keywords



16/57

Streaming API



Public Stream

Limitations


400 keywords



17/57

Twitter Data Collector


18/57

Tweet-o-Meter


19/57

Behind the Meters

Gauges Google Gauge [Javascript] 4 servers (Virtualised [Hyper-V]) Each running 4 geo-location searches

30km centriods

Graphs Google LineChart [Javascript] Show 1 hours activity Reset on the hour


20/57

Behind the Meters

Gauges Google Gauge [Javascript] 4 servers (Virtualised [Hyper-V]) Each running 4 geo-location searches

30km centriods

Graphs Google LineChart [Javascript] Show 1 hours activity Reset on the hour


21/57

Public Status Sample Stream

2 days 13 hours - 1,319,534 tweetsMonday 25th 2010 18:00 - Wednesday 27th 2010 12:30

Mon Tues Wed


22/57

Worldwide Geo-located Tweets in one weekend

2 days 14 hours - 2,531,287 tweetsFriday 22nd 2010 18:00 - Monday 25th 2010 08:30

Fri Sat Sun Mon


23/57

San Francisco Geo-located tweets


24/57

Twitter Messages in London over one Week

Fabian Neuhaus :- urbantick.blogspot.com/
http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238


25/57

Tweets in an Urban Context



26/57

Analysis of London Weekend Tweets


27/57

Data collected

0

175000

350000

525000

700000

NewYork London Paris Munich SanFrancisco Moscow Barcelona SydneyPRO Hong Kong Beijing Cnong Quing Guongzhou

Geo located all


28/57

Making the Data look nice!

New York Paris

http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238


29/57

Locating the messages

New YorkFabian Neuhaus :- urbantick.blogspot.com/


30/57

Message Density

New York

http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238


31/57

The New City Landscape Maps

Paris


32/57

Paris

The New City Landscape Maps


33/57

New City Landscape

London

http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238


34/57


35/57


36/57


37/57

New City Landscapes - China

Chongqing BeijingHong Kong

http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238


38/57

Munich London Paris New York

San FranciscoSydney BarcelonaMoscow

Hong Kong Chungqing Beijing


39/57

Aspects of time - Tracking



40/57

Recent London Twitter Map


41/57

Recent London Twitter Map


42/57

Tweet-o-meter in the Press


43/57



44/57



45/57



46/57



47/57



48/57



49/57


50/57


51/57


52/57

Main Objectives

- One stop shop for mapping data.

- Mapping your own data or other on-line data

-

Collection of data- Exporting our visualisations to GIS systems

- Making Mapping so easy even a child can do it!

http://gemma.casa.ucl.ac.uk


53/57

Oliver OBrien :- http://bikes.oobrien.com/http://oliverobrien.co.uk/

London Bike Share Visualisation


54/57

London Bike Share Flows

Martin Austwick & Oliver OBrien (@socialphysics, @oobr)
http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238


55/57

bigdatatoolkit.org


56/57


57/57

Steven Grayhttp://www.stevenjamesgray.comhttp://www.bigdatatoolkit.orgTwitter: @frogo
http://www.bigdatatoolkit.org/http://www.bigdatatoolkit.org/http://www.stevenjamesgray.com/http://www.stevenjamesgray.com/

eResearch Presentation - June '11

Documents

Transcript of eResearch Presentation - June '11