OS Open Data Masterclass
Ian Holt (@IanHolt)
Developer Programme Manager,
Ordnance SurveyNovember 2013
@ianholt #opendataAgenda
9.30 – Introduction
9.35 - OpenData presentation – Ian Holt, Developer Engagement Manager
10.00 – Introduction to OpenData Exercise
10.30 – OS OpenData exercises – based on a “Health and Wellbeing” theme
12.30 – Lunch
13.00 – Field Trip GB App
13.05 - Styling OS OpenData – A Cartography Masterclass
15.00 – OS OpenSpace introduction and exercises 16.30 - Close
Empowers
Citizens
To Innovate
Open Data….
More People
To Create More Services
More Easily
Empower….
@ianholt #opendata
Introductions
• The Team
• Ian Holt
• Luke Hampson
• Jon Field
• Charlie Glynn
• Today is all about:
• Hands-on use of Open Data.
• Appreciating the role geography can play in using Open Data.
• Meeting and learning from each other.
@ianholt #opendata
Global ‘Open’ movement
Open Source; Open Access; Open Data; Open Innovation
• Governments opening up; transparency and accountability; better value in public services; opportunities to realise significant economic benefits
• Internet business models based on ‘free’ and ‘open’
• Open systems are interoperable • Open access publishing is anti-publisher monopoly
@ianholt #opendata
@ianholt #opendata
Legal Framework
Sources of Open Data…Sources based on information from Open Data Cookbook 2010
“In order to unlock the potential of digital public sector information, developers and other prospective users must be able to find datasets they are interested in
reusing”
@ianholt #opendata
14955
@ianholt #opendata
Citizen Driven Catalogues(1) The Data Supermarkets ...
@ianholt #opendata
• Guardian World Data Store• A project to make data
widely available to everyone.
• Data Catalogue:• Search government data
sites• Ranking datasets• Collect similar datasets
together from around the world.
• Browse all datasets by each country
• API available
@ianholt #opendata
• Publicdata.eu• A catalogue bringing together data
from right across Europe.
• A single point of access to open, freely reusable datasets.
• Search, query, process, cache and perform other automated tasks.
• Co-production
Data Catalogues
@ianholt #opendata
Data Catalogues • opendatasearch.org
• A meta search engine for open data.
• Sunlight Foundation’s National Data Catalog (and with it a large number of US-based data sources), the World Bank data catalogue and Sweden’s DCat-enabled OpenGov.se
• Degree of interoperability between different data catalogues.
@ianholt #opendata
(2) Specialist Independents
@ianholt #opendata
Specialist Catalogues
@ianholt #opendata
Geo-specialists
@ianholt #opendata
(3) Foraging ... Searching for data
@ianholt #opendata
GetTheData.org
@ianholt #opendata
(4) Scrumping ... Screen-scrape the data
@ianholt #opendata
ScraperWiki
@ianholt #opendata
(5) Special Order
@ianholt #opendata
WhatDoTheyKnow.com
@ianholt #opendata
IsitOpenData.org
@ianholt #opendata
(6) Research and Crowdsourcing
@ianholt #opendata
Sources of Open Data
@ianholt #opendata
UK Open Data – Local Level
Local data that impacts most of us more directly.
New items of Local government spending over £500 – council by council from Jan 2011.
Openly Local is a new project to develop an open and unified way of accessing Local Government information.
@ianholt #opendata
• Ever more extensive catalogues of data that are being opened up.
• A meta search engine for open data: opendatasearch.org
• A simple and unified search interface to all the catalogues contained
Challenges - Information Overload
@ianholt #opendata
Open Data is only the first step ...
@ianholt #opendata
“There is no real guarantee of openness and transparency in the mere fact that some data are or became, somehow, available to the general public”
Build it and they will come
@ianholt #opendata
Create incentives to use the data
• Global competitions
• Create innovate software applications for development using WB data.
• Aim is to bring together software developers and development practitioners.
• Cash prizes available
@ianholt #opendata“Opportunities: Government as a Platform”
Government collects, validates and releases Citizens reformat, add value and create services
@ianholt #opendata
GeoVation® runs challenges to address specific GeoVation® runs challenges to address specific needs within communities, which may be satisfied inneeds within communities, which may be satisfied in
part through the use of geographypart through the use of geography.
www.geovation.org.ukwww.geovation.org.uk
Innovation challenges from Ordnance SurveyInnovation challenges from Ordnance Survey
@ianholt #opendata
Protect - Commerciial
GeoVation
6 challenges to date
@ianholt #opendataGeoVation – Camp
67 teams have attended a GeoVation Camp
Innovation = Problem x Solution x Execution
@ianholt #opendataChallenge 4 GeoVation winner:
Community Payback Visibility AppStaffordshire and West Midland Probation Trust
An app to help public nominate sites for Community Payback projects
@ianholt #opendata
Challenge 5 GeoVation winner: Real Food Wales – Mapkin.co.uk
”.
Source: http://www.mapkin.co.uk/realfoodwales
@ianholt #opendataChallenge 5 GeoVation winner:
MyTourTalk
”.
@ianholt #opendataChallenge 6: How can we help British Business
improve environmental performance?
260 registered 47 ideas
10 invited to GeoVation Camp - 21 -23 June
4 winners – share £101K in funding to get started.
6 March to 1 May 2013
Element Green RecyclingCarbon Prophet StreetkleenGeocraft
@ianholt #opendata
Minecraft Map of Great Britain
@ianholt #opendata
Protect - Commerciial
@ianholt #opendata
14955
@ianholt #opendata
data.gov.uk
• 14955 Datasets• 691 Publishers• 401 Ideas• 152 Apps• 104 Forum topics• 94 Blog entries• 44 Pages• 208 Data requests
@ianholt #opendata
Open Government Licence (OGL)
• In the past, licence variations were a significant barrier to data re-publication - the lowest common denominator
• The existing Click-Use Licence for central government ("Crown") works is now being replaced by the OGL which is based on the world-leading Creative Commons family of licences
• OGL allows anyone - businesses, individuals, charities and community groups - to re-use public sector information without having to pay or get permission
@ianholt #opendata
Data Formats and Frequency
• data.gov.uk is not a data store, it does not hold datasets but aim to • Provides a pointer to the resources• A rich and consistent resource description framework to facilitate
data discovery and, more important, linkage
• Available download formats• Statistical reports (.doc/.rtf/.pdf)• Data tables with raw or aggregate area statistics (.xls/.csv)• Linked data, customised datasets created on-the-fly (.xml)
• Normally only the latest edition of datasets/statistics is available so regular revisits to the data sites is necessary for time-series data analysis
49
@ianholt #opendata
Working with UK’s open geospatial data
• Locate open map and administrative datasets from data.ac.uk:
• Boundary and point data• Road traffic counts• Crime report figures• School performance statistics
• Mapping socio-economic attributes using GIS tools e.g. ArcGIS, Quantum GIS
• Visualising multiple area attributes spatially in web mapping GeoCommons services
50
@ianholt #opendata
220
@ianholt #opendata
460,000,000
@ianholt #opendata
5,000
@ianholt #opendata
99.9%
@ianholt #opendata
27,000,000
@ianholt #opendata
36,000
@ianholt #opendata
547,772
@ianholt #opendata
965,419
@ianholt #opendata
25+
@ianholt #opendata
04/2010
@ianholt #opendata
All Change…Encourage Open Innovation• As of April 2010 a range of Ordnance Survey mapping data available for free
• Two aims: to foster innovation and encourage government transparency
• Free for commercial and non-commercial use for anyone wanting to build applications underpinned by geography
@ianholt #opendata
What is OS OpenData?
• Rasters• OS Street View• MiniScale• 1: 250 000 Colour Raster
• Vectors• Meridian 2• Strategi• OS VectorMap District • 1: 50 000 Gazetteer• OS Locator• Boundary-Line• Code-Point Open• Terrain 50
@ianholt #opendata
11
@ianholt #opendata
1,700,000
25
42,000
@ianholt #opendata
Code-Point® Open- Provides a precise geographic location for each postcode unit in Great Britain. - Contains postcodes, grid references, NHS® health & regional health authority codes; administrative ward, district, county and country area codes. - There are 1.7 million approx. postcode units in England, Scotland and Wales. - Each postcode unit, e.g. SO16 4GU, contains an average of 15 addresses.
@ianholt #opendata
Boundary–Line™
- a vector digital mapping product that is a complete set of local government administrative boundaries and electoral boundaries used in local and general election voting. - specifically designed to show the area of each administrative or electoral boundary.
@ianholt #opendata
600,000+
@ianholt #opendata
@ianholt #opendata
@ianholt #opendata
@ianholt #opendata
@ianholt #opendata
How do I get OS OpenData?
@ianholt #opendataDeveloper Licenses
@ianholt #opendata
Free is not always open
Open is not always free
Analysis is not always easy
Open Data is not always good data
@ianholt #opendata
Photo by Desré Pickers
• Learn from each successes and failures• Help innovation around open data• Help data owners to publish their data.• Contributing to the development of tools, services and standards to support this work
@ianholt #opendata
Thank you and questions
@ianholt #opendata
Raw Data vs Data As A Service
@ianholt #opendata
10,000+
@ianholt #opendata
What is OS OpenSpace?
• It’s a free service that provides free access to mapping data
• Embed maps in any web site or web application
• All applications are free to consumers
• API based on OpenLayers and can be used to access OnDemand Service
@ianholt #opendata
Available raster • OS Street View
• OS Vector Map District
• 1:50 000 Scale Colour Raster
• 1:250 000 Scale Colour Raster
• Mini Scale (1:1 000 000)
• Overview of Great Britain
• Outline of Great Britain
@ianholt #opendata
Available Vectors
• OS OpenSpace Boundaries
• Simplified version of Boundaryline
• GeoJSON
• 1:50 000 Scale Gazetteer
• Place names
• Map centre function
• Codepoint Open
• Postcode lookup service
• Map centre
@ianholt #opendata
API adds controls for :- Scale- Zoom- Pan
API creates:- Points- Lines- Polygons-OverlaysIn any symbol style
API supports:- Pop-up / information boxes to hold any content or media- Reading in data from different formats / sources
API controls:- Data delivery- Map location- Copyright statements- Gazetteer searches
The Ordnance Survey API
@ianholt #opendataOS OpenSpace community tools
@ianholt #opendata
OpenSpace Examples