ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy...
-
Upload
sharon-taylor -
Category
Documents
-
view
215 -
download
0
Transcript of ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy...
![Page 1: ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy Morley, Jan-Peter Muller, Nuno Gil*, Cristiano Giovando**,](https://reader031.fdocuments.in/reader031/viewer/2022013101/56649e755503460f94b765e6/html5/thumbnails/1.jpg)
ICEDS – using open source software to serve data and maps of the globe at full resolution
Jeremy Morley, Jan-Peter Muller,Nuno Gil*, Cristiano Giovando**, Iain Willis^
Oliver Greening, Rob Scott^(ESYS plc)
*now with ESRI Portugal; **now at SDSU; ^ now at Qinetiq
Department ofGeomatic Engineering
![Page 2: ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy Morley, Jan-Peter Muller, Nuno Gil*, Cristiano Giovando**,](https://reader031.fdocuments.in/reader031/viewer/2022013101/56649e755503460f94b765e6/html5/thumbnails/2.jpg)
http://iceds.ge.ucl.ac.uk
Some OGC Implementation Specifications used now and in the future in ICEDS (Integrated CEOS European Data Server)
• WMS – Web Map Service– Provides rendered images (pictures) of maps
• WCS – Web Coverage Service– Provides raster data (coverages) on demand– Restricted to locally held US-sourced data (SRTM, Landsat 7 & 5)
• WFS – Web Feature Service– Provides vector data (features) on demand
• GML – The Geography Markup Language– Used as an interoperable standard for transmitting geographic data
![Page 3: ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy Morley, Jan-Peter Muller, Nuno Gil*, Cristiano Giovando**,](https://reader031.fdocuments.in/reader031/viewer/2022013101/56649e755503460f94b765e6/html5/thumbnails/3.jpg)
http://iceds.ge.ucl.ac.uk
ICEDS Phases 1, 2 & 3 (2004-6)
• Prepared OGC-compliant:– Web Map Service (WMS)– Web Coverage Service (WCS)– Portal website (based on Ionic Red Spider GAF
interface)
• Provided detailed ‘Guidelines’ document on data preparation process
• Exploited open access (i.e. open source / freeware / free to download) software– MapServer: http://mapserver.gis.umn.edu/
![Page 4: ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy Morley, Jan-Peter Muller, Nuno Gil*, Cristiano Giovando**,](https://reader031.fdocuments.in/reader031/viewer/2022013101/56649e755503460f94b765e6/html5/thumbnails/4.jpg)
http://iceds.ge.ucl.ac.uk
ICEDS Phases 1, 2 & 3 (2004-6)
• Provided Web access to global datasets– SRTM & Landsat 5 and 7 mosaics– GTOPO30/SRTM30 to extend beyond area of SRTM– Web map services (transportation, built settlements)– MODIS WMS products where available
• Developed new visualisation capabilities– Use of transparency, flicker, colour LUTs to combine
data
• Tested interfaces to Google Earth• Encouraged other CEOS members to interoperate
![Page 5: ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy Morley, Jan-Peter Muller, Nuno Gil*, Cristiano Giovando**,](https://reader031.fdocuments.in/reader031/viewer/2022013101/56649e755503460f94b765e6/html5/thumbnails/5.jpg)
http://iceds.ge.ucl.ac.uk
Meteorological Information Resource for Africa (MIRA) extension (March 2006)
• Aim to tie together interoperable components from BNSC ICP2 projects with a focus on real-time.
• Serving meteorology-related data (surface wind field from GFS model; Meteosat-8 Near Real-Time cloud imagery)
• Chain of services: Comsine Ltd. hosting WMS• Implies new temporal selection capability in ICEDS client• Cannot offer free operational service due to the restrictive
conditions of access to EUMETSAT data which forbid Internet broadcasting of any images of derived Metoesat data for any purpose (including educational)
![Page 6: ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy Morley, Jan-Peter Muller, Nuno Gil*, Cristiano Giovando**,](https://reader031.fdocuments.in/reader031/viewer/2022013101/56649e755503460f94b765e6/html5/thumbnails/6.jpg)
http://iceds.ge.ucl.ac.uk
ICEDS system
• RAID Server:– 1.7TB capacity;
dual Athlon MP 2400+ processors; 1 Gb of RAM
• Mandrake Linux 9• Apache Web Server• University of Minnesota MapServer
– version 4.4.1 (current is 4.8.3)– (Mandrake 9 difficult to upgrade MapServer)
![Page 7: ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy Morley, Jan-Peter Muller, Nuno Gil*, Cristiano Giovando**,](https://reader031.fdocuments.in/reader031/viewer/2022013101/56649e755503460f94b765e6/html5/thumbnails/7.jpg)
http://iceds.ge.ucl.ac.uk
Datasets hosted on ICEDS• Landsat 5 mosaic
– Africa: bands 1,2,3 (WMS/WCS)– Europe: bands 1,2,3 (WMS/WCS) + band 4 (WCS)
• SRTM DEM– version 1 (unfilled)– colourised hill-shaded, gap map (WMS)– Elevations with gaps indicated (WCS)– SRTM30 (merge with GTOPO30, etc.) for globe
• SRTM water bodies mask– developed by NGA during SRTM DEM v2 production– held in original shapefile tiles, rendered on the fly
• DMSP Night-time Lights• MODIS Blue Marble
– version 1– principally for back-drop
![Page 8: ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy Morley, Jan-Peter Muller, Nuno Gil*, Cristiano Giovando**,](https://reader031.fdocuments.in/reader031/viewer/2022013101/56649e755503460f94b765e6/html5/thumbnails/8.jpg)
http://iceds.ge.ucl.ac.uk
Cascaded WMS layers
• CustomWeather:– Near-real time global IR cloud map (within 3 hours)– Weather forecast symbols
• NASA Aster DEM coverage (from USGS)• SIGAFRIQUE (from BRGM)
– African geological maps (1:10M scale)
• JPL OnEarth:– SRTM amplitude mosaic– Global Landsat 7 mosaic– Multiple styles for each layer
![Page 9: ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy Morley, Jan-Peter Muller, Nuno Gil*, Cristiano Giovando**,](https://reader031.fdocuments.in/reader031/viewer/2022013101/56649e755503460f94b765e6/html5/thumbnails/9.jpg)
http://iceds.ge.ucl.ac.uk
Data processing
• Data processing & loading fully documented in downloadable Guidelines document
• Revised methodologies cf. phases 1 & 2– more efficient processing schemes– expanded hill-shading colour table for Himalayas!
• Scripts used in preparation also downloadable
![Page 10: ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy Morley, Jan-Peter Muller, Nuno Gil*, Cristiano Giovando**,](https://reader031.fdocuments.in/reader031/viewer/2022013101/56649e755503460f94b765e6/html5/thumbnails/10.jpg)
http://iceds.ge.ucl.ac.uk
SRTM data preparationMasking
The background colour (red) shows voids in the SRTM data
Hill shading
A colour table has been defined reflecting atlas-like colour ranges
SRTM tiles intersecting coastline
DEM mosaicked in 4 regions (NW, NE, S, Azores)
Azores included in Europe and Africa – political versus geographic affiliations
Inland tiles do not need masking
Smart processing
![Page 11: ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy Morley, Jan-Peter Muller, Nuno Gil*, Cristiano Giovando**,](https://reader031.fdocuments.in/reader031/viewer/2022013101/56649e755503460f94b765e6/html5/thumbnails/11.jpg)
http://iceds.ge.ucl.ac.uk
Pyramid layers
• Key for access speed is pyramid layers• Landsat 5 mosaic:
– 1 arcsec/pixel from scale 1:1 to 1:1 M (14302 tiles)– 10 arcsec/pixel from scale 1:1 M to 1:8 M (1020 tiles)– 100 arcsec/pixel from scale 1:8 M to max (1 tile)
• Hill-shaded SRTM:– 3 arcsec/pixel from scale 1:1 to 1:1.7 M (14621 tiles)– 30 arcsec/pixel from scale 1:1.7 M to 1:20 M (434 tiles)– 180 arcsec/pixel from scale 1:20 M to max (8 tiles)
![Page 12: ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy Morley, Jan-Peter Muller, Nuno Gil*, Cristiano Giovando**,](https://reader031.fdocuments.in/reader031/viewer/2022013101/56649e755503460f94b765e6/html5/thumbnails/12.jpg)
http://iceds.ge.ucl.ac.uk
![Page 13: ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy Morley, Jan-Peter Muller, Nuno Gil*, Cristiano Giovando**,](https://reader031.fdocuments.in/reader031/viewer/2022013101/56649e755503460f94b765e6/html5/thumbnails/13.jpg)
http://iceds.ge.ucl.ac.uk
![Page 14: ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy Morley, Jan-Peter Muller, Nuno Gil*, Cristiano Giovando**,](https://reader031.fdocuments.in/reader031/viewer/2022013101/56649e755503460f94b765e6/html5/thumbnails/14.jpg)
http://iceds.ge.ucl.ac.uk
ICEDS portal• Purely OGC-based services• Hence separation of client and server(s)• Increasing range of ‘off-the-peg’ OGC clients, e.g.
– ka-Map! (http://ka-map.maptools.org/)– Chameleon
(http://www.dmsolutions.ca/technology/chameleon.html)
• ICEDS portal based on a client from Ionic– HTML + Javascript only– Good browser compatibility: PC & Mac IE,
Mozilla/Firefox/Seamonkey, Mac Safari & Camino– NO need for separate plug-in or standalone programme
![Page 15: ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy Morley, Jan-Peter Muller, Nuno Gil*, Cristiano Giovando**,](https://reader031.fdocuments.in/reader031/viewer/2022013101/56649e755503460f94b765e6/html5/thumbnails/15.jpg)
http://iceds.ge.ucl.ac.uk
ka-Map!
![Page 16: ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy Morley, Jan-Peter Muller, Nuno Gil*, Cristiano Giovando**,](https://reader031.fdocuments.in/reader031/viewer/2022013101/56649e755503460f94b765e6/html5/thumbnails/16.jpg)
http://iceds.ge.ucl.ac.uk
Interface features
• Download source data extracts from ICEDS WCS via icon on web interface
• Polygon annotation demonstration– only on beta site at present– WMS for displaying annotations
• Colour scale for SRTM hill-shading• Browser compatibility
– tests on IE & Firefox / Mozilla / Seamonkey on PC & Mac; Safari on Mac
– Javascript object models, PNG display are main issues
![Page 17: ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy Morley, Jan-Peter Muller, Nuno Gil*, Cristiano Giovando**,](https://reader031.fdocuments.in/reader031/viewer/2022013101/56649e755503460f94b765e6/html5/thumbnails/17.jpg)
http://iceds.ge.ucl.ac.uk
Interface features – layer controls
• Added capabilities for layer overlay:– partial layer opacity– ‘swipe’ function– ‘flicker’ function
• WMS layer style selection– e.g. OnEarth Landsat 7 band combinations
![Page 18: ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy Morley, Jan-Peter Muller, Nuno Gil*, Cristiano Giovando**,](https://reader031.fdocuments.in/reader031/viewer/2022013101/56649e755503460f94b765e6/html5/thumbnails/18.jpg)
http://iceds.ge.ucl.ac.uk
Interface features – temporal selection
![Page 19: ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy Morley, Jan-Peter Muller, Nuno Gil*, Cristiano Giovando**,](https://reader031.fdocuments.in/reader031/viewer/2022013101/56649e755503460f94b765e6/html5/thumbnails/19.jpg)
http://iceds.ge.ucl.ac.uk
Interface features – temporal selection
Thumbnails are live & updated to match zoom/ pan of map
![Page 20: ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy Morley, Jan-Peter Muller, Nuno Gil*, Cristiano Giovando**,](https://reader031.fdocuments.in/reader031/viewer/2022013101/56649e755503460f94b765e6/html5/thumbnails/20.jpg)
http://iceds.ge.ucl.ac.uk
Chained Met-8 service cascaded
EUMETSAT Met-8 feed
LogicaCMG:
Processing and orthorectification service
Comsine:
Databasing & WMS service
WMS
UCL:
ICEDS client
![Page 21: ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy Morley, Jan-Peter Muller, Nuno Gil*, Cristiano Giovando**,](https://reader031.fdocuments.in/reader031/viewer/2022013101/56649e755503460f94b765e6/html5/thumbnails/21.jpg)
http://iceds.ge.ucl.ac.uk
Near Real-time (every 15 minutes) Meteosat-8 SEVIRI data.
![Page 22: ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy Morley, Jan-Peter Muller, Nuno Gil*, Cristiano Giovando**,](https://reader031.fdocuments.in/reader031/viewer/2022013101/56649e755503460f94b765e6/html5/thumbnails/22.jpg)
http://iceds.ge.ucl.ac.uk
ICEDS portal - demonstration
![Page 23: ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy Morley, Jan-Peter Muller, Nuno Gil*, Cristiano Giovando**,](https://reader031.fdocuments.in/reader031/viewer/2022013101/56649e755503460f94b765e6/html5/thumbnails/23.jpg)
http://iceds.ge.ucl.ac.uk
Interoperability
• Examples:– SRTM hill-shading served into BRGM SIGAFRIQUE– New agreement with JAXA to ‘swap’ layers
• Demonstrations of other clients connecting to ICEDS WMS & WCS
![Page 24: ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy Morley, Jan-Peter Muller, Nuno Gil*, Cristiano Giovando**,](https://reader031.fdocuments.in/reader031/viewer/2022013101/56649e755503460f94b765e6/html5/thumbnails/24.jpg)
http://iceds.ge.ucl.ac.uk
WMS Connection from ArcGIS
![Page 25: ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy Morley, Jan-Peter Muller, Nuno Gil*, Cristiano Giovando**,](https://reader031.fdocuments.in/reader031/viewer/2022013101/56649e755503460f94b765e6/html5/thumbnails/25.jpg)
http://iceds.ge.ucl.ac.uk
WMS connection from osgPlanet
![Page 26: ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy Morley, Jan-Peter Muller, Nuno Gil*, Cristiano Giovando**,](https://reader031.fdocuments.in/reader031/viewer/2022013101/56649e755503460f94b765e6/html5/thumbnails/26.jpg)
http://iceds.ge.ucl.ac.uk
WMS connection from osgPlanet
![Page 27: ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy Morley, Jan-Peter Muller, Nuno Gil*, Cristiano Giovando**,](https://reader031.fdocuments.in/reader031/viewer/2022013101/56649e755503460f94b765e6/html5/thumbnails/27.jpg)
http://iceds.ge.ucl.ac.uk
WCS + WMS Connection from Cadcorp SIS
![Page 28: ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy Morley, Jan-Peter Muller, Nuno Gil*, Cristiano Giovando**,](https://reader031.fdocuments.in/reader031/viewer/2022013101/56649e755503460f94b765e6/html5/thumbnails/28.jpg)
http://iceds.ge.ucl.ac.uk
Comsine: WCS demonstration - orthorectification wcs_url = 'http://iceds.ge.ucl.ac.uk/cgi-bin/wcs?map=wcs.map&SERVICE=WCS&REQUEST=GetCoverage&COVERAGE=srtm&FORMAT=GEOTIFFINT16' res = max(tif_pixsize_x, tif_pixsize_y) wcsRequest = wcs_url + "&CRS=EPSG:" + str(epsg) + "&BBOX=" + bbox + "&RESX=" + str(res) + "&RESY=" + str(res) logger.info("WCS REQUEST " + wcsRequest) if (dem == None): demFilename = str(time.clock()) + ".tif" urllib.urlretrieve(wcsRequest, filename) cleanup = 1
![Page 29: ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy Morley, Jan-Peter Muller, Nuno Gil*, Cristiano Giovando**,](https://reader031.fdocuments.in/reader031/viewer/2022013101/56649e755503460f94b765e6/html5/thumbnails/29.jpg)
http://iceds.ge.ucl.ac.uk
GeoConnections Canada
![Page 30: ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy Morley, Jan-Peter Muller, Nuno Gil*, Cristiano Giovando**,](https://reader031.fdocuments.in/reader031/viewer/2022013101/56649e755503460f94b765e6/html5/thumbnails/30.jpg)
http://iceds.ge.ucl.ac.uk
Google Earth – live connection Uses a PHP script at ESA to translate from WMS to KML for each refresh request
![Page 31: ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy Morley, Jan-Peter Muller, Nuno Gil*, Cristiano Giovando**,](https://reader031.fdocuments.in/reader031/viewer/2022013101/56649e755503460f94b765e6/html5/thumbnails/31.jpg)
http://iceds.ge.ucl.ac.uk
Site access stats. – IP addresses per day
![Page 32: ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy Morley, Jan-Peter Muller, Nuno Gil*, Cristiano Giovando**,](https://reader031.fdocuments.in/reader031/viewer/2022013101/56649e755503460f94b765e6/html5/thumbnails/32.jpg)
http://iceds.ge.ucl.ac.uk
Site access stats1st – 25th November 2005
• IP addresses:– accessing server: 471
– making a WMS request: 302– making a WMS request from outside ICEDS portal: 133
– making a WCS request: 62– making a WCS GetCoverage request: 23
![Page 33: ICEDS – using open source software to serve data and maps of the globe at full resolution Jeremy Morley, Jan-Peter Muller, Nuno Gil*, Cristiano Giovando**,](https://reader031.fdocuments.in/reader031/viewer/2022013101/56649e755503460f94b765e6/html5/thumbnails/33.jpg)
http://iceds.ge.ucl.ac.uk
Conclusions
• Open Source software solution (MapServer) appears to be very robust & effective
• Growing interest in ICEDS service– WCS as well as WMS
• A great deal can be achieved in creating a Web portal simply within the Web browser environment
• Plan to add X-SRTM WMS strips in the near future• Collaboration on OGC demonstration projects• ICEDS provides a good foundation for further OGC-related
developments– e.g. service into OGC GEOSS demonstrators