Chart-o-Mat

ICG-WIGOS TT on Metadata

Management of Station Metadata

at Deutscher Wetterdienst (DWD)

Rainer MärzSection Administration of Observing Networks

rainer.maerz@dwd.de

Geneva, March 11, 2013

Principles for Metadata at DWDPrinciples for Metadata at DWD

Metadata is stored together with observing data in one database

Project ‚MIRAKEL‘ started around 1995 for CDC + GPCC

All conditions are valid for a specific time-range (from – until)

Integration of Metadata from other organisations

Flexible Import/Export for future needs

Semi-automatic update and quality checking procedures

All applications should benefit from the data (and use it)

Geneva, March 11, 2013

Amount of station Metadata Amount of station Metadata

Overview of the contents [count of largest categories in thousands]:

WMO Main Stations (GSN, RBSN, RBCN ….) 13.000

Airports (with ICAO loc.ind.) reporting METAR/SPECI 5.000

German additional stations (voluntary, federal states) 12.000

Related partners (roads, renewable energy ….) 2.000

Voluntary observing ships and other marine vessels 9.000

Additional GPCC stations of all countries 75.000

Currently counting over all: 117.000

Geneva, March 11, 2013

Storage of Metadata / groups of tables Storage of Metadata / groups of tables

Tables for metadata divided into :

Basic Information meta tables; allowed values

Joined Information intersection tables with validity dates (begin / end)

Tables for observation data use the same unique station ID !

Staff

Mea

surin

g co

ndid

ions

Station

Pub

9Vol

A

Equipment

Geneva, March 11, 2013

Storage of Metadata / classes of informationStorage of Metadata / classes of informationMain aspects of meta data storage:

Main station table (name, start_date, end_date, status)

IDs (WMO / ICAO / IATA / GPCC / Nat.Numbers …)

Geographical (Lat./Lon., elevation, country, region, area etc.)

Equipment

Staff

Operation and Production (programmes for encoding reports)

Visits (Maintenance, Administrative)

Representation (measuring conditions)

Geneva, March 11, 2013

Managing the MetadataManaging the Metadata

Access and permits managed by database-roles

4 accounts with full access - ‚Admin‘

10+ accounts to maintain general station changes (add, move, close)

Approx. 100 accounts allow a ‚restricted write-access‘

Approx. 1000 accounts have only read-permits (all users)

Personalized data like bank accounts with restricted roles

Geneva, March 11, 2013

Updating and completing the Metadata Updating and completing the Metadata

Weekly import of Pub9VolA of WMO and semi-automatic changes

Regional Network Groups report national changes to Offenbachand edit their local entries (persons, sensors, visits, history)

Import of bilateral exchanged data with individual loading-procedures

comparision with other sources when necessary (flagging of suspicious/unknown/inconsistent)

GPCC maintains meta-data for non-WMO stations of other NMCs

Geneva, March 11, 2013

Typical trouble with metadataTypical trouble with metadata

No information about changes at a station

Inconsistency between dependent sources (i.e. elevation…)

Lack of syncronization with Offline-procedures

Especially Lat./Lon. different various formats (dez.,°,min/sec)

4 sources / 5 different values ;-) …. Which is ‚the best‘

Representation of a complicated history for a time-series

standard exchange format + automated process is necessary!

Geneva, March 11, 2013

Stations operated by Deutscher Wetterdienst 4 Regional Oberserving Network Groups

Hamburg, Potsdam, Offenbach, Munich

65 Stations with professional Observers, 35 occupied 00-24 UTC, 30 only daytime48 measuring radioactivity (air+precipitation)

28 RBSN Stations 11 Climate Reference (traditional equipment)

4 GSN-Stations (GCOS) 1 GUAN Station 1 GAW Station

114 Automated Weather Stations with full equipment

17 Weather Radar Stations

9 Aerological Stations (4 autolaunchers, 1 ozone)

4 Wind Profilers

+34 Stations of Bundeswehr Geoinfo Service

Geneva, March 11, 2013

Additional Stations operated by DWD and Partners

1781 Voluntary Stations (climate; wind; precip), 1365 reporting online (24/1 reports per day)

Ship-based:

740 Voluntary Observing Ships (VOS)

19 Ship (AWS) (24 reports/day)

Last but not least …:

1292 Phenological Stations (observing plants)

1500 Partner stations (approx) (motorways, fed.states, wind-energy, (nuclear) power-plants, university, military)

Geneva, March 11, 2013

All 1018 GSN stations in WMO GCOS

Basic Meta Data and availability of FM12/FM94 observations received in Offenbach

Example

Geneva, March 11, 2013

Example

Precip-Stationand the surroundingpresented in a toolof GPCC called

‚Mirakel-Live‘

There is still a lot of research necessary!

Geneva, March 11, 2013

Outlook Outlook

WCDMP / Data rescue needs support (DARE @ CDC) national projects currently in progress:

-> KliDaDigi for land stations -> HISTOR for ship logbooks

Lots of undigitalized metadata from all German Stations

+Electronical archives of BUFR since 1970s may be loaded

Work for many years!

Geneva, March 11, 2013

QUESTIONS AND COMMENTS ?

Thank you for your attention !