Delivering  data  to  help  manage  Australia’s  Coastal  Ecosystems  

LAND  AND  WATER  

Jonathan  Hodge  Project  Leader  Environmental  Informa4on  Systems  

Overview  

•  Environmental  data  in  Australia  

•  Terrestrial  Ecosystem  Research  Network  

•   Australian  Coastal  Ecosystems  Facility  

•  eReefs  and  the  next  evolu4on  in  data  interoperability  

Environmental  data  in  Australia  

•  Large  amounts  of  high  quality  environmental  data  sit  with  government  and  research  organisa4ons  

• Many  of  these  organisa4ons  tend  to  be  risk  averse  and  slow  to  adopt  technology  

•  Strong  push  in  recent  years  towards    open  data  and  open  licensing  

• Major  na4onal  investments  are  changing  the  way  that  Australia  delivers  data  

Environmental  data  in  Australia  

• Many  now  deliver  OGC  services  

• Map  and  metadata  services  common  

• More  complex  services  less  common  

•  Prolifera4on  of  silos  of  informa4on  

•  Prolifera4on  of  portals    

• Minimal  cross-­‐domain  or    cross-­‐organisa4on  integra4on  

Data  evoluBon  in  Australia  

•  Some  systems,  such  as  the  Australian  Ocean  Data  Network,  have  started  to    aggregate  these  services  

•  Need  to  allow  these  service  silos  to    deliver  more  widely  across  more  than  one  community  

Environment  &  earth  science  data  IniBaBves  

TERN:  Transforming  Australian  ecosystem  science  

TERN:  •  Infrastructure  and  networks  to  support  coordinated,  

collabora4ve  ecosystem  science  community  •  Enabling  sustained,  long-­‐term  collecBon,  storage,  

synthesis  and  sharing  of  ecosystem  data  •  Connec4ng  science  with  policy  and  management      

What  is  TERN  

Key  objec)ves:  •  coordinate   and   build   na4onal   network   for   terrestrial   ecosystem  research  and  management  

•  provide  and  facilitate  open  access  to  ecosystem  science  data  

•  build   a   strong   ecosystem   science   community   to   solve   problems  facing  Australia’s  ecosystem  

•  enable   to   iden4fy   the   gaps   in   data   to   understand   Australia’s  ecosystem.  

What  is  TERN  

Funded  by  Australian  Government  via:  

•  Na4onal  Collabora4ve  Research  Infrastructure  Strategy  (NCRIS)  

•  Super  Science  Ini4a4ve  -­‐Educa4on  Investment  Fund  (EIF)  

•  Collabora4ve  Research  Infrastructure  Strategy  (CRIS)  

•  Na4onal  Collabora4ve  Research  Infrastructure  Strategy  2  (NCRIS2)  

Instruments    +  Sensors  

Policy  +    Management  

Analysis    +  Synthesis  

Modelling  

Data    Searching  

Data    Sharing  

Data  CuraBon    +  Publishing  

Data    Storage  

Processing    +  Analysis  

CollecBon    Methods  

TERN  Data  license  Suite  

Covering  all  licensed  materials  including  materials  not  subjected  to  copyright:    

TERN  ASribu4on  Licence  (TERN-­‐BY)  

TERN  ASribu4on-­‐No  Deriva4ves  Licence  (TERN-­‐BY-­‐ND)  

TERN  ASribu4on-­‐Share  Alike  Licence  (TERN-­‐BY-­‐SA)  

TERN  ASribu4on-­‐Share  Alike-­‐Non  commercial  (TERN-­‐BY-­‐SA-­‐NC)  

Crea4ve  Commons  Australia  open  source  licenses  for  copyright  materials  

Data  citaBon  

•  Digital  Object  Iden4fiers  for  datasets  

•  Allows  datasets  to  become  citable  scholarly  publica4ons  like  journal  ar4cles  

•  enables  tracking  of  data  usage    

doi:10.4227/05/508637F997933  hSp://dx.doi.org/10.4227/05/508637F997933    

How  TERN  fits  together  

•  Network  of  Facili4es  to  meet  TERN  overall  objec4ve  

•  Each  Facility  deals  with  par4cular  domain  of  ecosystem  science  

   

Biogeophysical  Data  •  AusCover  •  Australian  Coastal  Ecosystem  Facility  •  OzFlux  •  Soil  and  Landscape  Grid  of  Australia  Ecological  Data  •  Mul4-­‐Scale  Plot  Network  •  Eco-­‐informa4cs  Analysis,  Synthesis  and  Modelling  •  Australian  Centre  for  Ecological  

Analysis  and  Synthesis  •  Ecosystem  Modelling  and  Scaling  

Infrastructure  

How  TERN  fits  together  

Australian  Coastal  Ecosystems  Facility  

Australian  Coastal  Ecosystems  Facility  

•  Providing  the  servers  and  services  to  host  data  and  metadata  

• Mix  of  data  and  portal  solu4ons:  •  Geoserver  •  GeoNetwork  •  THREDDS  Data  Server  •  Drupal  •  OGC  services  

•  Virtual  machines  based  at  CSIRO  and  on  NeCTAR  

 

Australian  Coastal  Ecosystems  Facility  

•  Handling  most  data  types  

• Working  with  local,  state  and  commonwealth  governments,  research  agencies,  regional  bodies  and  others  

• Websites,  portals,  web  services  for  data  access  

TERN  

Yet  another  community  

Another  community  

A  different  community  

But...  

We  need  to  deliver  to  more  than  just  one  community  

Discoverability  

•  What  do  we  expect  out  of  the  internet  (eg.  websites)?  –  Always  there,  in  a  format  we  understand  –  Easy  to  search  for  –  Easy  to  contribute  your  own  informa4on  –  Up  to  date  

•  Latest  contact  informa4on  •  Up  to  date  prices  

•  But  what  about  data?  –  Not  always  available,  or  not  available  in  the  format  we  want  –  Hard  to  search  for  and  possibly  doesn’t  exist  –  Very  difficult  to  contribute  your  own  informa4on  –  Occasionally  updated  (if  ever)  

Need  to  bridge  the  gaps  

•  Need  to  develop  a  structured  approach  to  discovery  and  interoperability  

•  Need  to  answer  what,  how  and  where  (and  more)  

• What  is  available?  

•  How  do  I  access  it?  

• Where  is  it?  Where  has  it  come  from?  

• We  need  to  be  able  to  answer  these  consistently  

‘Linked  Open  Data’  

‘Linked  open  data  could  do  for  numbers    what  the  Web  did  for  words,  pictures,  video’  

Sir  Tim  Berners-­‐Lee,  inventor  of  the  internet  

make  your  stuff  available  on  the  Web  (whatever  format)  

under  an  open  license  

make  it  available  as  structured  data  (e.g.,  Excel  instead  of  image  

scan  of  a  table)  

use  non-­‐proprietary  

formats  (e.g.,  CSV  instead  of  Excel)  

use  URIs  to  denote  things,  so  that  people  can  point  at  your  stuff  

link  your  data  to  other  data  to  

provide  context  5-­‐star  Open  Data  

hSp://5stardata.info/  

0

10

20

30

40

50

60

70

13  Sep  2011 03  Oct  2011 23  Oct  2011 12  Nov  2011 02  Dec  2011

Nitrate  (µM)

EHMP  Site  211   (upstream) EHMP  Site  212   (downstream) In-­‐situ  measurments  (SUNA)

12  Oct  2011 14  Oct  2011

Depth  (indicative)

eReefs  •  Catchment  monitoring  •  Estuarine  monitoring  •  Catchment  modelling  •  Coastal  models  •  Receiving  water  models  •  Satellite  products  

eReefs  Data  Provider  node  

Data  Provider  Node  Template

Data  services  

Metadata  service

Vocabulary  Service

Data  Brokering  Layer

Provenance  service  

InformaBon  Model  Catalog  

Data  Provider  Node  Ontology  -­‐  Classes  DataProviderNode  

Dataset  

Website  

DataService  

WebFeatureService  

SensorObserva4on  Service  

WebCoverageService  

MetadataService  

CatalogServiceWeb  

VocabularyService  

SISSVoc  

InfoModelService  

FeatureTypeCatalog  

ProvenanceService  

ProvMgtSysService  

OWL  Class  to  Persistent  IdenBfier  Service  

OWL  Classes  

Dataset  

 DataService  

 MetadataService  

 VocabularyService  

 InfoModelService  

 ProvenanceService  

PID  Service  (URI)  PaSerns  

_view=alternates  

_view=data  

_view=metadata  

_view=vocab  

_view=infomodel  

_view=prov  

Dataset  Class  –  stored  as  RDF  

:Dataset    a                  owl:Class  ;    rdfs:subClassOf      :DataProviderNode  ;    rdfs:label      “{name}"^^xsd:string  ;    skos:prefLabel      "{name}"^^xsd:string  ;    :serviceEndpoint      "{uri}"^^xsd:anyURI  ;    :isPublic      "{true|false}"^^xsd:boolean  ;    dcterms:license    "{license_uri}"^^xsd:anyURI  ;  

   

For  example,  a  dataset:  hSp://dpn-­‐oc-­‐vc.nexus.csiro.au/datasets/BDKN-­‐B09    will  have  a  data  service  URI  of:  hSp://dpn-­‐oc-­‐vc.nexus.csiro.au/datasets/BDKN-­‐B09?_view=data    (Instead  of  the  original  THREDDS  end  point)    and  a  metadata  service  URI  of:  hSp://dpn-­‐oc-­‐vc.nexus.csiro.au/datasets/BDKN-­‐B09?_view=metadata  (Instead  of  the  original  GeoNetwork  end  point  on  different  server)  

Consistent,  predictable  end  points  

 hSp://environment.data.gov.au/water/quality/chlorophyll/    

hSp://dpn-­‐oc-­‐vc.nexus.csiro.au/datasets/BDKN-­‐B09?_view=data    

hSp://dpn-­‐oc-­‐vc.nexus.csiro.au/datasets/BDKN-­‐B09?_view=metadata    

TesBng  and  orchestraBon  services  

OrchestraBon  Service  test  sets  

Basic  DPN  structure  

Data  Service  configuraBon  

Provenance  records  

Vocab  term  references  

Service  level  provision  

Metadata  conformance  

MulBple  use  and  re-­‐use  

Data  Provider  Node  Ocean  Colour  

Consumer  Node  eReefs  Main  Portal

eReefs  Community  Central  Node

Other   Data  Provider  Nodes

Data  Provider  Node  Hydrodynamic  

Other  Community  Central  Node

Other  Consumer  Nodes

What  does  this  mean  for  data?  

•  Discoverability  (using  normal  internet  tools)  •  Availability  (in  the  format  you  want)  •  Currency  (updated  as  you  need  it)  •  Linked  (to  other  datasets,  metadata,  aSribu4ons,  licences,  etc)  

But,  only  if  the  community  changes  how  it  thinks  about  data:  •  Think  of  web  services  (xml)  as  data    • Move  from  offline  to  online  (eg.  email,  Google  Docs,  etc)  

And,  coupled  with  open  licensing:  •  Get  A[ribuBon  and  recogni4on  • Mul4ple  use  and  re-­‐use  

Summary  

• We  have  the  standards  (mostly)  

• We  have  the  tools  and  services  (commercial  or  open  source)    •  ASribu4on  &  licensing  are  mostly  solved  

But:  • We  need  to  connect  data  into  the  rest  of  the  internet  

•  Standards  and  tools  are  coming  

LAND  AND  WATER  

Jonathan  Hodge  Project  Leader  t  +61  7  3833  5515  m  +61  409  577  945  e  jonathan.hodge@csiro.au  w  www.csiro.au