Earth Data Open Search Specifications

Doug Newman (NASA ECHO)CWIC January 2014

The specifications

Open Search

http://www.opensearch.org/Specifications/OpenSearch/1.1

• Still in draft?!!! • Open Search descriptor document• URL element, template attribute• Query element

• Request• HTTP GET URL with standard query parameters• Constructed programmatically from URL element definition in OSDD

• Response• ATOM feed response with zero or more entries• Links to metadata, data, browse, more searches

The base extensions

• Geo extension - http://www.opensearch.org/Specifications/OpenSearch/Extensions/Geo/1.0/Draft_2

• Spatial constraint by point, box, circle and WKT (line polygon etc)• Spatial constraint by place name• Spatial constraint relation *• Search constraint by uid?!! *• Spatial extent via GeoRSS elements

• Time extension - http://www.opensearch.org/Specifications/OpenSearch/Extensions/Time/1.0/Draft_1

• Temporal constraint by start and end date using RFC-3339• No temporal extent guidelines

• Parameter extension - http://www.opensearch.org/Specifications/OpenSearch/Extensions/Parameter/1.0/Draft_2

• A more formalized specification of search parameters• Permissible ranges• Pattern matching for permissible values

• Relevancy extension - http://www.opensearch.org/Specifications/OpenSearch/Extensions/Relevance/1.0

• Ranking your results

* Recently we see that the GEO extension has been augmented by the OGC specification

The OGC specification

• Candidate standard • Rel=‘via’ still in examples - ^&#$#()))_!!!!• Elements of this standard have crept into the geo extension• Navigation by links (prev, self and next)

http://www.opengeospatial.org/node/1926

The ESIP best practices

• Dynamic, bespoke OSDDs • Nested discovery via hypermedia• Link ‘rel’ mapping (augmented by media type)• data – enclosure• metadata – describedBy• documentation – describedBy• browse – icon• OSDD – search

• OpenDAP• Spatial extent as minimum bounding rectangle• Temporal extent as dublin core date

http://wiki.esipfed.org/index.php/Discovery_Cluster

Navigating this mine field

7

Insert cheeky graphic here…

Attribution: http://imgs.xkcd.com/comics/standards.png

Open Search Specs

The hierarchy

OGC Specification

ESIP Best PracticesCEOS Best Practices

Geo Extension Time Extension

Parameter Extension

Relevancy Extension

Open Search Specification

via

describedBy

The CEOS best practices

1. Conform to the Open Search Specification2. Conform to the OGC Specification3. Take pertinent ideas from ESIP best practices4. Add our own recommendations

Such as,• Make best attempt at coercing clients to provide client ids• Extend ESIP ‘two step’ idea to link IDN dataset -> CWIC granule searching• Embrace hypermedia

• Navigation by ‘self’, ‘next’, ‘prev’• Media types

• Achieve a ‘programmatic client’ where possible.• Relevance• Utilizing the parameter extension for better clarity of service• Spatial (MBR+) and temporal (Dublin Core) extent representation

How it will look in theory

Visualization

HTTP Response

CWIC Dataset 1 CWIC Dataset 2

Metadata Link

Data Link

Search Link

Metadata Link

Data Link

Search Link

HTTP GET RequestIDN CWIC OSDD

HTTP Response

Granule 1 Granule 2

Metadata Link

Data Link

Browse Link

Metadata Link

Data Link

Browse Link

HTTP GET RequestCWIC OSDD for

dataset 1

Open Search Descriptor Document

‘We essentially ignore everything except the URL element’Chris Lynnes, ESIP Winter 2014

• IDN - one OSDD exposing CWIC datasets (dynamic*)• CWIC• one OSDD for general granule queries• one OSDD per dataset (dynamic*) for IDN coupling

• Obtain OSDD by URL with mandatory client id parameter• Should we exercise draft 2 of the Open Search parameter extension?

* Dynamic with respect to client id

HTTP GET domain/opensearch/resource(s)/descriptor_document.xml?clientId=d

Request

HTTP GET domain/opensearch/resource(s).atom

• Free text os:searchTermsdatasets.atom?keyword=MODIS

• Spatial geo:box, (geo:geometry)granules.atom?boundingBox=-180.0,-90.0,180.0,90.0

granules.atom?geometry=POINT (-5 -5)granules.atom?geometry=LINESTRING (5 5, 4 4)granules.atom?geometry=POLYGON ((30 10, 10 20, 40 40, 30 10))

• Temporal time:startTime, time:endTimedatasets.atom?startTime=2001-01-01T22:00:00Z

datasets.atom?endTime=2001-01-01T22:00:00Z

• Traversal os:pageNumber, os:numberOfResultsgranules.atom?pageNumber=1&numberOfResults=10

Response (1 of 3)

Navigation

<os:totalResults>3404</os:totalResults> <os:itemsPerPage>10</os:itemsPerPage> <os:startPage>2</os:startPage>

<link href=‘https://foo.gov/opensearch/datasets.atom’ rel=‘prev’ type=‘application/atom+xml’/><link href=‘https://foo.gov/opensearch/datasets.atom?cursor=2’ rel=‘self’ type=‘application/atom+xml’/><link href=‘https://foo.gov/opensearch/datasets.atom?cursor=3’ rel=‘next’ type=‘application/atom+xml’/>

Response (2 of 3)

Navigation

<?xml version="1.0" encoding="UTF-8"?> <feed … >

…<entry … > <id>http://idn.gov/opensearch/datasets/1234</id>

<link rel=‘search’ type=‘application/opensearchdescription+xml’>cwic.gov/opensearch/granules/descriptorDocument?datasetId=1234&client=d</link>

…</entry>…

</feed>

Response (2 of 3)

Content• ID – resolvable URI

<id>cwic.gov/foo?uid=Landsat_8:LC81240182014014LGN00</id>

• Spatial Extent – MBR bounding box<georss:box>-1.05426 104.36788 1.04721 106.41068</georss:box>

• Temporal Extent – Dublin Core date<dc:date>2014-01-14T03:11:45Z/2014-01-14T03:12:17Z</dc:date>

• Links– Data: <link rel=‘enclosure’ type=‘application/x-hdf’>– Metadata: <link rel=‘describedBy’ type=‘application/xml’>– Browse: <link rel=‘icon’ type=‘image/jpeg’>– Documentation: <link rel=‘describedBy’ type=‘application/pdf’>– Alternate: <link rel=‘alternate’ type=‘application/xml’>

Challenges

• Maintaining coupling between IDN and CWIC• Interpretation of existing Open Search

implementations• Handling specification divergence• Programmatic client from specification-

compliant implementations• Should we use the parameter extension?• Ranking

Questions