Scotland's Environment Web Data Journey 2011-2015 Dave Watson, Duncan Taylor.
-
Upload
nickolas-baker -
Category
Documents
-
view
217 -
download
0
Transcript of Scotland's Environment Web Data Journey 2011-2015 Dave Watson, Duncan Taylor.
Scotland's Environment Web
Data Journey 2011-2015
Dave Watson, Duncan Taylor
Session Outline• SEWeb data journey
– What has been encountered on that journey
• SEWeb as a data consumer– What do we do with the data?
• Five Star/Linked Data • SEWeb Data – what next?
Partners
Data Publication
Daughter Sites
INSPIREWMS
SSDI
Eye on Earth
Gemini2,
IPR
Data Protection
WFS
Data Download Service
Scottish Government
Digital Stategy
Data Visualisation
Linked Data
National Security
SEWeb Data Journey
Partners Business as Usual
Environmental Data Portal?
Scotland’s Environment Web - Data Journey
Data Consumer
Data Consumer
SEWeb Brand – Daughter Web Sites
Data at Source
Dataset Progress• ‘Data at Source’
– 55 WMS consumed by Map Viewer -> 239 Data Layers– 9 Rest Services consumed by Land Information Search (LIS) -> 39 Data
Layers– 10+?? Non spatial data consumed by Visualisation Tools
• Five Star /Linked Data– 68 SESO Data, 12 Water (SEPA WFD), 1 Site Conditioning (SNH)
• Data Holdings– Soils/Aquaculture Daughter Sites– Project Finder
What do we do with the data?
• Themed spatial maps• Advanced Maps• Visualisation Applications• Task Specific Applications• Linked Data Repository
Themed/Advanced Maps
Task Specific Maps – Land Information Search
Visualisation/Discover Data
# Available on the web (whatever format) but with an open licence, to be Open Data
# # Available as machine-readable structured data (e.g. excel instead of image scan of a table)
# # # as (2) plus non-proprietary format (e.g. CSV instead of excel)
# # # # All the above plus, Use open standards from W3C (RDF and SPARQL) to identify things, so that people can point at your stuff
# # # # # All the above, plus: Link your data to other people’s data to provide context
Why Linked Data? - 5 Star Model of Open Data
http://www.w3.org/DesignIssues/LinkedData.html
Linked Data Four Principles
1. Use URIs as names for things
2. Use HTTP URIs so that people can look up those names.
3. When someone looks up a URI, provide useful information, using the standards (RDF*, SPARQL)
4. Include links to other URIs so that they can discover more things.
http://www.w3.org/DesignIssues/LinkedData.html
State of Environment (SOE) – Linked Data Model
SOE(State of Environment)
has
soe:Chapter
consistsOf
soe:Topic
dct:Dataset
Metadata
describedBy
soe:State
has
State Of Environement(Linked Data)Graph Model
hasdataset
Essential|supporting
Importance
SOE – Implementation
Vocabulary/concept schemehttp://data.sepa.org.uk/def/soe Trial datahttp://data.sepa.org.uk/id/soe/chapters
SOE Data Linkages
Chapter Topic Dataset SEWEB
SOE Data Linkages
SOE Data Linkages
Chapter Topic=
national indicator
Dataset
European Indicator (SOE) EEA
SEWEB
relates to
SOE Data Linkages
SOE Data Linkages
SOE Data Linkages
Chapter Topic Dataset
Data view and download services
Data Provider
links to
Metadata
EEA
SEWEB
relates to
publishes
feeds
European Indicator (SOE)
SEWeb Data - What Next?• Continued Addition of Datasets• What’s in my Area? – Local Datasets/SEWeb Local• Scottish Government Digital Strategy – Data Portals• Graphical Data Models to support ‘State of
Environment’• Links to European Data Initiatives
Useful Links– SEWeb www.environment.scotland.gov.uk – Scottish Soils http://www.soils-scotland.gov.uk/ – Aquaculture http://aquaculture.scotland.gov.uk/– Linked Data Lab http://data.sepa.org.uk– SSDI http://scotgovsdi.edina.ac.uk/srv/en/main.home– INSPIRE http://inspire.ec.europa.eu/ – Water Classification Visualisation
http://www.environment.scotland.gov.uk/get_interactive/data_visualisation/water_body_classification.aspx
End of Presentation – Workshop Support Slides Follow
Linked Data Architecture
RDBMS
Repository
Relational Data
Consumers
Datasets.Related not Relational
Metadata
WMS
WFS
File Download
Linked Data
Apps
Bespoke Data Feed
Data Feed Future
Dataset Definition.Metadata
Cannot do any subsequent steps without this
definition. Business needs to define and prioritorise
Other Data Providers
INSPIRE
REPORTING SENSE 2/2015
SOE
Organisational,Eg EA,SG etc
SEPA Stakeholders
Public
Citizen Scientists
Data Ingestion
OntologiesVocabularies
DRIVERSSEPA
Architecture
Useful Links– SEWeb www.environment.scotland.gov.uk – Scottish Soils http://www.soils-scotland.gov.uk/ – Aquaculture http://aquaculture.scotland.gov.uk/– Linked Data Lab http://data.sepa.org.uk– SSDI http://scotgovsdi.edina.ac.uk/srv/en/main.home– INSPIRE http://inspire.ec.europa.eu/ – Water Classification Visualisation
http://www.environment.scotland.gov.uk/get_interactive/data_visualisation/water_body_classification.aspx
SENSE 3 – Schema Relationships
State of Environment Reporting
• Defined by chapters (air, water, land, etc)
• Chapters divided into topics, each with a summary quality assessment
• Datasets support and inform the assessment of the topic
• A dataset may be related to more than one topic
• Currently published as static pages
State of Environment Reporting
• Remodel as linked data
• Enable publication of metadata on datasets
• Link to data visualisation and download where available
• Provide contact details where data not yet published on line
• Provide support and examples of best practice to assist publication
SEPA as Data Provider
SEPA Reporting Requirements
Information required at many levels
• Internal – SEPA corporate systems
• National – State of Environment; SEWeb
• European – Directive Reports; INSPIRE
Where we were…
Many applicationsMany formats
Many versions
SEPA Database
ReportsGIS Applications
PublicationsWebsite
Information Requests
EU Reporting
What we decided to do
• Focus on data – not applications
• Identify key reporting datasets
• Define them once
• Use them many times…
• …in many formats
Where we’ve got to
Operational Database
Reporting Database
Publish Externally
Defined data “products”
Consistent metadata
GIS
Intranet
Reports & Analysis
SEWeb
SEPA Website
EU ReportingConsistent data
Where we’re getting to
Operational Database
Reporting Database
Publish as WMS; WFS; Linked data
Defined data “products”
Consistent metadata
GIS
Intranet
Reports & Analysis
EU ReportingConsistent data
Websites (SEPA, SEWeb,…)
Partners
Public
EU
What’s helped
• Scotland’s Spatial Data Infrastructure – provided framework and standards for metadata
• SEWeb – prioritisation of datasets
• Government direction – “digital by default“
• EU reporting frameworks – SEIS, SENSE
What we need now
• Agree to use existing standards and vocabularies
• Define new ones where appropriate
• Encourage use of common reference systems
• Encourage others to use the data
What we get out of it
• Wider (and cleverer) use of data
• Less bespoke development
• Fewer information requests to deal with
• Publish data once – let everyone else get on with it
Data Architecture
RDBMS
Repository
Relational Data
Consumers
Datasets.Related not Relational
Single Purpose Apps
E.g. RBMP
Bespoke Data Feed
Dataset Definition.Metadata
SEPA Architecture
Single Purpose Apps
RDBMS
Repository
Relational Data
Consumers
Datasets.Related not Relational
Metadata
WMS
WFSApplications
Dataset Definition.Metadata
Cannot do any subsequent steps without this
definition. Business needs to define and prioritorise
INSPIRE
DRIVERSSEPA
Architecture
Service Data Feed
INSPIRE Service Based Architecture
RDBMS
Repository
Relational Data
Consumers
Datasets.Related not Relational
Metadata
WMS
WFS
File Download
Linked Data
Apps
Bespoke Data Feed
Data Feed Future
Dataset Definition.Metadata
Cannot do any subsequent steps without this
definition. Business needs to define and prioritorise
Other Data Providers
INSPIRE
REPORTING SENSE 2/2015
SOE
Organisational,Eg EA,SG etc
SEPA Stakeholders
Public
Citizen Scientists
Data Ingestion
OntologiesVocabularies
DRIVERSSEPA
Architecture
Linked Data Architecture
RDBMS
Repository
Relational Data
Consumers
Datasets.Related not Relational
Metadata
WMS
WFS
File Download
Linked Data
JSON
RDF/XML
SPARQL
TURTLE
csv/tsv
HTML
Web Apps
Mashups
Linked Data Sites/Uers
“Big Data” Sites/Uers
“Traditional” Sites/Uers
Web Developers
Apps
Bespoke Data Feed
Data Feed Future
Dataset Definition.Metadata
Cannot do any subsequent steps without this
definition. Business needs to define and prioritorise
Other Data Providers
INSPIRE
REPORTING SENSE 2/2015
SOE
Organisational,Eg EA,SG etc
SEPA Stakeholders
Public
Citizen Scientists
Data Ingestion
OntologiesVocabularies
Define Equivalences
DRIVERSSEPA
Architecture
Rdf Triple StoreServer
ELDA
Linked Data ‘Technology Stack’
Linked Data
# Available on the web (whatever format) but with an open licence, to be Open Data
# # Available as machine-readable structured data (e.g. excel instead of image scan of a table)
# # # as (2) plus non-proprietary format (e.g. CSV instead of excel)
# # # # All the above plus, Use open standards from W3C (RDF and SPARQL) to identify things, so that people can point at your stuff
# # # # # All the above, plus: Link your data to other people’s data to provide context
5 Star Model of Open Data
http://www.w3.org/DesignIssues/LinkedData.html
What is Linked Data?
• Data in which real-world things are given addresses on the web (URIs), and data is published about them in machine-readable formats.
• Describes a method of publishing structured data so that it can be interlinked and become more useful.
• Builds upon standard Web technologies such as HTTP, RDF and URIs, but rather than using them to serve web pages for human readers, it extends them to share information in a way that can be read automatically by computers.
• Enables data from different sources to be connected and queried.
Linked Data Four Principles
1. Use URIs as names for things
2. Use HTTP URIs so that people can look up those names.
3. When someone looks up a URI, provide useful information, using the standards (RDF*, SPARQL)
4. Include links to other URIs so that they can discover more things.
http://www.w3.org/DesignIssues/LinkedData.html
Operational System
Typical Relational Data Table
Surface Water BodiesCOLUMN NAME DATA TYPE MANDATORY
ID Number Y
NAME Varchar2(30) Y
CATEGORY Varchar2(15) N
SUB_BASIN Varchar2(30) N
CATCHMENT Number N
STATUS Varchar2(30) N
Typical Relational Data
ID NAME CATEGORY SUB_BASIN
CATCHMENT STATUS
3001 River Almond (Breich Water confluence to Maitland Bridge)
River Forth 61 Poor
3809 River North Esk (Source to Penicuik House)
River Forth 63 High
100208 Loch Shiel Lake Argyll 117 Good
200019 South Arran Coastal Clyde Good
As Linked Data
Surface Water Body 3001 is of category River
Surface Water Body 3001 is called River Almond (Breich Water confluence to Maitland Bridge)
Surface Water Body 3001 is in sub-basin Forth
Surface Water Body 3001 is in catchment 61
Surface Water Body 3001 has status Poor
Surface Water Body 200019 is of category Coastal
Surface Water Body 200019 is called South Arran
Surface Water Body 200019 is in sub-basin Clyde
Surface Water Body 200019 has status Good
As Linked Data
Surface Water Body 3001 is of category River
Surface Water Body 3001 is called River Almond (Breich Water confluence to Maitland Bridge)
Surface Water Body 3001 is in sub-basin Forth
Surface Water Body 3001 is in catchment 61
Surface Water Body 3001 has status Poor
Surface Water Body 200019 is of category Coastal
Surface Water Body 200019 is called South Arran
Surface Water Body 200019 is in sub-basin Clyde
Surface Water Body 200019 has status Good
Surface Water Body 3001 is in local authority West Lothian
Surface Water Body 3001 is in local authority City of Edinburgh
Surface Water Body 200019 is in postcode district KA27
RDF/Triplestore
Subject Predicate Object
http://data.sepa.org.uk/id/water/surfacewaterbody/3001
rdf:type http://data.sepa.org.uk/def/water/WaterBody
http://data.sepa.org.uk/id/water/surfacewaterbody/3001
rdf:type http://data.sepa.org.uk/def/water/SurfaceWaterBody
http://data.sepa.org.uk/id/water/surfacewaterbody/3001
rdf:type http://data.sepa.org.uk/def/water/RiverWaterBody
http://data.sepa.org.uk/id/water/surfacewaterbody/3001
rdfs:label “River Almond (Breich Water confluence to Maitland Bridge)”
http://data.sepa.org.uk/id/water/surfacewaterbody/3001
http://data.sepa.org.uk/def/water/currentOverallClassification
“Overall status – Poor”
http://data.sepa.org.uk/id/water/surfacewaterbody/3001
http://data.sepa.org.uk/def/water/inCatchment
http://data.sepa.org.uk/id/water/catchment/61
http://data.sepa.org.uk/id/water/catchment/61
http://data.sepa.org.uk/def/water/surfaceArea
6503
http://data.sepa.org.uk/id/water/catchment/61
http://data.sepa.org.uk/def/water/catchmentType
“Main River”
http://data.sepa.org.uk/id/water/subbasindistrict/3
rdfs:label “Forth”
Non SEPA-SEWeb Linked Data Examples
• Data.gov.uk.http://data.gov.uk/linked-data/who-is-doing-what
• EA Bathing Watershttp://environment.data.gov.uk/bwq/explorer/index.html
Ordnance Survey
http://data.ordnancesurvey.co.uk/doc/postcodeunit/EH127AT • Winnipeghttp://now.winnipeg.ca/
• Legislationhttp://www.legislation.gov.uk/