Mitg

lied

in d

er H

elm

holtz

-Gem

eins

chaf

t

WCS Server for CF-NetCDF An Overview

AQCoP Meeting, August 2011 | M. Decker, M. Schultz, K. Hoijarvi, R.B. Husar

Mitg

lied

in d

er H

elm

holtz

-Gem

eins

chaf

t

Functioning software package that implements a WCS compliant data server delivering NetCDF files Based on Python Runs on Windows and Linux Supports WCS 1.1.2 Protocol Supports NetCDF 3 and NetCDF 4 large file formats Supports station data from SQL database

10. April 2023 2

Introduction

Mitg

lied

in d

er H

elm

holtz

-Gem

eins

chaf

t

Metadata about... the service available datasets

Extraction of... a rectangular area (boundingbox) individual variables (fields) individual points or ranges in time (subsetting of other dimensions)

...the foundation for using interoperable web applications

10. April 2023 3

WCS gives you...

Mitg

lied

in d

er H

elm

holtz

-Gem

eins

chaf

t

10. April 2023 4

High-Level view of WCS

Mitg

lied

in d

er H

elm

holtz

-Gem

eins

chaf

t

10. April 2023 5

Example metadata response (GetCapabilities)

Mitg

lied

in d

er H

elm

holtz

-Gem

eins

chaf

t

10. April 2023 6

Software Dependencies

Operating system Windows (>= XP) GNU/Linux

Python >= 2.6 (but not 3 - yet)

Python web framework web.py (http://webpy.org/)

Python XML parser Lxml (http://lxml.de/)

Python numerical package NumPy (http://numpy.scipy.org/)

Logging Backend for web.py (optional)

WsgiLog (http://pypi.python.org/pypi/wsgilog/)

Python netcdf interface [currently part of server package]

PyNIO

(http://www.pyngl.ucar.edu/Nio.shtml

)More??

Mitg

lied

in d

er H

elm

holtz

-Gem

eins

chaf

t

10. April 2023 7

Server Concept

Mitg

lied

in d

er H

elm

holtz

-Gem

eins

chaf

t

10. April 2023 8

Server Directory Structure

OWS

web

static

datafed

testprovider

...

cache

NetCDF interface code

web interface code

web home (index.html, ...)

temp dir

example data dir

other data dirs

Mitg

lied

in d

er H

elm

holtz

-Gem

eins

chaf

t

Code is OpenSource under The MIT License Collaboration Tools:

Esipfed-Wiki: documentation SourceForge project page: forum, bugtracker DARCS distributed version control system: code

(currently) two main repositories: WUSTL and FZ Juelich easy to set up your own full-featured repository further information available on the SourceForge project page

Test-driven development process unit tests: OWS/test/unit.py integration tests: OWS/test/integ.py

10. April 2023 9

Development

Mitg

lied

in d

er H

elm

holtz

-Gem

eins

chaf

t

install 3rd party software (see dependencies)

get WCS server archive from SourceForge project page Windows: win32-ows-*.zip Linux: linux-ows-*.tar.gz

put archive contents into sensible location for server operation for example /usr/local/ows (Linux) or C:\ows (Windows)

add server dir to PYTHONPATH environment variable

10. April 2023 10

Getting Started I: Installation

Mitg

lied

in d

er H

elm

holtz

-Gem

eins

chaf

t

example data dir included: testprovider contains example files (index.html, wcs_capabilities.conf)

that you can adapt for your own data

create NetCDF test cubes call web/static/testprovider/create_all.py

extract metadata for operation: call web/owsadmin.py wcs_prepare testprovider

start the server: web/ows.py 1080

check if it works: http://localhost:1080/testprovider

10. April 2023 11

Getting Started II: test it

Mitg

lied

in d

er H

elm

holtz

-Gem

eins

chaf

t

create a data directory in web/static/ populate it with (CF-compliant) NetCDF files configure metadata

create wcs_capabilities.conf create human-readable index.html(optional, but encouraged) you can use the example files from testprovider

run metadata extractor: owsadmin.py wcs_prepare <data dir name> needs to be re-run after data dir content has been updated

also adapt the global index.html in web/static

10. April 2023 12

Getting Started III: serving your data (NetCDF)

Mitg

lied

in d

er H

elm

holtz

-Gem

eins

chaf

t

10. April 2023 13

Getting Started IV: serving station data (SQL) create a data directory in web/static/ create SQL views to:

location data: loc_code, lat, lon + optional parameter data: loc_code, lat, lon, datetime, parameter, flags

edit configuration: SQL database connection location SQL view name + fields time periodicity for each coverage SQL data view name + fields for each parameter

configure metadata as with NetCDF dataThe real work is done inside the SQL database when creatring SQL views, the rest is just listing the names!

Mitg

lied

in d

er H

elm

holtz

-Gem

eins

chaf

t

start the standalone server: ows.py [port number] (default port: 1080)

data will be available via WCS at http://localhost:<port>/<data dir>?...

10. April 2023 14

Getting Started V: running the server

Mitg

lied

in d

er H

elm

holtz

-Gem

eins

chaf

t

10. April 2023 15

Thank you for your attention!

Mitg

lied

in d

er H

elm

holtz

-Gem

eins

chaf

t

10. April 2023 16