Exploiting XML and a SOAP Service to Disseminate National Weather Service Digital Forecast Data

Exploiting XML and a SOAP Exploiting XML and a SOAP Service to Disseminate National Service to Disseminate National Weather Service Digital Weather Service Digital Forecast DataForecast Data

John L. SchattelJohn L. SchattelMeteorological Development LaboratoryMeteorological Development Laboratory

NOAA WebShop 2004NOAA WebShop 2004

July 27 - 29, 2004July 27 - 29, 2004


The NWS Strategic PlanThe NWS Strategic Plan

“…evolving our services from a text-based paradigm to one based on making National Weather Service (NWS) information available quickly, efficiently, and in convenient and understandable forms…” and “…by taking advantage of existing and emerging technologies to disseminate this information…”


OverviewOverview

• The value of digital dataThe value of digital data• NWS digital forecast databaseNWS digital forecast database

• How it is createdHow it is created• What it containsWhat it contains

• Digital Weather Markup Language (DWML)Digital Weather Markup Language (DWML)• XML OverviewXML Overview• DWML Data ModelDWML Data Model

• A SOAP service to disseminate NWS digital dataA SOAP service to disseminate NWS digital data• SOAP service introductionSOAP service introduction• National Digital Forecast Database (NDFD) Web ServiceNational Digital Forecast Database (NDFD) Web Service


Potential Uses of NWS DataPotential Uses of NWS Data

• Weather information along a Weather information along a pathpath

• Text generation in more than Text generation in more than one languageone language

• Forecasts for vehicles and Forecasts for vehicles and hand-held devices with GPShand-held devices with GPS

• Controls for smart appliances Controls for smart appliances (e.g., heating, cooling, (e.g., heating, cooling, irrigation)irrigation)

.TONIGHT...CLEAR. LOW IN THE MID 30S. LIGHT WIND EARLY BECOMING SOUTHWEST 10 TO 15 MPH AFTER MIDNIGHT.

Pre-digital Age …Pre-digital Age …


Digital Age DataDigital Age Data

Improved spatial Improved spatial resolutionresolution

EnhancedEnhancedtemporaltemporalcoveragecoverage


NDFD Data FlowNDFD Data Flow

LocalGrids

NYC

LocalGrids

Mt Holly

LocalGridsSterling

NDFD(Mosaic)

NWS 7-Day Digital ForecastNWS 7-Day Digital Forecast


Current NDFD ContentsCurrent NDFD Contents

• Daytime maximum and nighttime minimum temperatureDaytime maximum and nighttime minimum temperature• Probability of Precipitation (12 hour)Probability of Precipitation (12 hour)• Significant weatherSignificant weather• Sky coverSky cover• TemperatureTemperature• Dewpoint temperatureDewpoint temperature• Wind direction and speedWind direction and speed• Precipitation amount (QPF)Precipitation amount (QPF)• Snow amountSnow amount• Wave heightWave height


NDFD ResolutionNDFD Resolution

• Spatial resolution:Spatial resolution:- 5 km grids for now- 5 km grids for now- 2.5 km grids when hardware allows- 2.5 km grids when hardware allows

• Temporal resolution:Temporal resolution:- 3 hourly for days 1-3 - 3 hourly for days 1-3 - 6 hourly for days 4-7- 6 hourly for days 4-7

• Update frequency: every hourUpdate frequency: every hour


NDFD Data FlowNDFD Data Flow

LocalGrids

NYC

LocalGrids

Mt Holly

LocalGridsSterling

NDFD(Mosaic)

NWS 7-Day Digital ForecastNWS 7-Day Digital Forecast

Internet UsersNWS

Web Farm

ProductsProducts

ServicesServices


NDFD ProductsNDFD Products

GraphicsTonight: Scattered showers before 10pm.Chance for precipitation is 30%. Partly cloudy, with a low near 29. West wind 6 to 9 mph increasing to between 15 and 18 mph.

Friday: Partly cloudy, with a high near 49. Breezy, with a west wind between 14 and 21 mph.

Friday Night: Partly cloudy, then gradually becoming clear, with a low around 23. West wind between 8 and 14 mph.

Saturday: Sunny, with a high around 50.

Saturday Night: Mostly clear, with a low around 26.

Text

GRIB2YGCC00 KWBN 171951 GRIBQAwAAVJVS05XQwIBAABgChIMNgFCVEZOSFRKQTk4IEtOV0MgMTgxMjAwDQ0KR1JJQgA9sgEAABwCOkbygAtpAAJgChIMAAEAAAAAAAAUAAABAAAgAP8AAIUAaQAPoAFnYIAA2sACaTAB9AH0QAAAAAAAPWoDAABDpNrkCbzfL/gcHhcBdrTT5XH55ZcBjcdhLhYK5W7HeMRjsfjsZisRhsNh87777YGCD00 KWBN 171951GRIBRiMRiicTisPeK/RZbIpfS7RecFhsPiMVi8XisRhsJgsFgsHhMLhcJg8HhMLhcNhsNhsPh8RiMTicVi8dkMjkMfjsfj8djsfk8tmc7oFoM7msvlMjjsbjcfkMjkclkcjkMfjsfkMcnk8jj8fj8djcbj8nlsznM/odBns3mctksfjcbjsfkclkslkchj8djsfkcllMpkcTgr3fb9gcJhr3Z61U6lX7thsXjMTd6vR6ZVLHecLicXjcbjcViMPicTicTicTicTicVi8R7777

http://www.nws.noaa.gov/forecasts/graphical/





NDFD ServicesNDFD Services

<dwml xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

<head> <product concise-name="time-series"> <creation-date>2004-02-13T21:29:11Z</creation-date> </product> <source> <production-center>MDL</production-center> </source> </head> <data> <parameters> <temperature units="Fahrenheit"> <value>45</value> <value>31</value> <value>29</value> <value>37</value> <value>41</value> <value>41</value> <value xsi:nil="true" /> </temperature> </parameters> </data></dwml>

DWML SOAP Service DWML SOAP Service

User SelectableUser SelectableGRIB2 DataGRIB2 Data


XML OverviewXML Overview

<dwml xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

<head> <product concise-name="time-series"> <creation-date>2004-02-13T21:29:11Z</creation-date> </product> <source> <production-center>MDL</production-center> </source> </head> <data> <parameters> <temperature units="Fahrenheit"> <value>45</value> <value>31</value> <value>29</value> <value>37</value> <value>41</value> <value>41</value> <value xsi:nil="true" /> </temperature> </parameters> </data></dwml>

XML Schema +SAX, DOM,

XSLT

TODAY: HIGH TEMP OF 45 F.TOMORROW: HIGH TEMP OF 31 FSUNDAY: HIGH TEMP OF 29 FMONDAY: HIGH TEMP OF 37 FTUESDAY: HIGH TEMP OF 41 FWEDNESDAY: HIGH TEMP OF 41 F

1. Document needs one root element2. Elements come in start and end pairs3. XML is case sensitive4. Attributes must appear within quotes5. Elements must not overlap (only nested)

XML Rules

Extensible Markup Language (XML) n. 1. Internet standard for data exchange.2. Language to create other languages. 3. A way to provide NDFD data

http://www.recreation.gov/detail.cfm?ID=29







XML OverviewXML Overview

Existing technologyExisting technology• XML is widely used (W3C specification)XML is widely used (W3C specification)• De Facto standard for internet data exchangeDe Facto standard for internet data exchange• Infrastructure exists (UDDI, validators, XML Schema) Infrastructure exists (UDDI, validators, XML Schema)

Facilitates additional processingFacilitates additional processing• XML tools (XSLT, SAX, DOM) facilitate value added XML tools (XSLT, SAX, DOM) facilitate value added

applicationsapplications• Web application developers familiar with toolsWeb application developers familiar with tools• Schema precisely defines a languageSchema precisely defines a language


NDFD Data CubeNDFD Data Cube

TimeSp

ace

Weather Parameters


DWML Data ModelDWML Data Model

<dwml> <head> <product /> <source /> </head> <data> <location /> <time-layout /> <parameters /> </data></dwml>

more follows = …zero or one = ?zero or more = *one or more = +

dwml

head data

product source location time parameters

…

version

Element

Attribute

……………



location

location-key pointlatitudelongitudesummarization ?



Example




time-layout

layout-key start-valid-time end-valid-time

time-coordinate

period-name ?

+ *


Example



<dwml> <head> <product /> <source /> </head> <data> <location /> <time-layout /> <parameters /> </data></dwml> parameters

temperature precipitation probability-of-precipitation wind-speed direction

… … …

…

…typeunitstime-layout

typeunitstime-layout

* * * * *




…


applicable-location



<dwml> <head> <product /> <source /> </head> <data> <location /> <time-layout /> <parameters /> </data></dwml> parameters

cloud-amount weather humidity …

…* *

conditions-icons

icon-link


water-state… …

?

applicable-location



*


Example

…

?




parameters

temperature

name value

*

+



Example

NOTE: This structure applies to precipitation, probability-of-precipitation, wind-speed, direction, cloud amount and humidity

applicable-location


XML on the MoveXML on the Move

DataConsumer

(Client)

Internet

SOAP Request

SOAP Response SOAP Response

SOAP Request

Data Broker(UDDI)

WSDL

Data

Transport(HTTP)

Messaging(SOAP)

Description(WSDL/Schema)

Discovery(UDDI)

Communicating Process

WSDL

DataProvider(Server)

http://uddi.microsoft.com/search/frames.aspx?frames=true&search=51835a5d-2d64-4602-ab4a-2f24f5a7415e

http://weather.gov/mdl/survey/pgb_survey/XML/ndfdXMLserver.php


NWS NDFD Web ServiceNWS NDFD Web Service

Customeror Partner

CustomDisplay

UserAddress

End User(Public)

NWS WebFarm

SOAP(latitude/longitude*)

SOAP(DWML)

Potential Applications:•NWS: Fire Weather Spot Forecasts•Public: Park forecasts for recreation.gov•Private: Smart appliances

*Optional inputs include which times and weather parameters the user*Optional inputs include which times and weather parameters the userwants or the name of a pre-defined set of parameters (glance).wants or the name of a pre-defined set of parameters (glance).


NDFD Web Service DemoNDFD Web Service Demo

NDFD XML NDFD XML • User’s client creates SOAP service inputUser’s client creates SOAP service input

– The location (latitude/longitude)The location (latitude/longitude)– Product name (“time-series” or “glance”)Product name (“time-series” or “glance”)– Which weather elementsWhich weather elements– A start and end date/timeA start and end date/time

• User’s client creates & sends a SOAP requestUser’s client creates & sends a SOAP request• NDFD SOAP server returns DWML/dataNDFD SOAP server returns DWML/data• User application processes DWMLUser application processes DWML

http://www.nws.noaa.gov/forecasts/xml/sample_products/browser_interface/ndfdXML.htmhttp://www.nws.noaa.gov/forecasts/xml/sample_products/browser_interface/ndfdXML.htm


NDFD Web Service DemoNDFD Web Service Demo

Meteogram ExampleMeteogram Example• User’s client creates SOAP service inputUser’s client creates SOAP service input• Client uses WSDL to create SOAP requestClient uses WSDL to create SOAP request• NDFD SOAP server returns DWMLNDFD SOAP server returns DWML• Client uses SAX to parse the DWMLClient uses SAX to parse the DWML• Client uses VH Graph to create meteogramClient uses VH Graph to create meteogram

http://www.nws.noaa.gov/forecasts/xml/sample_products/meteogram/DWML_graph.htmhttp://www.nws.noaa.gov/forecasts/xml/sample_products/meteogram/DWML_graph.htm


Implementing a Web ServiceImplementing a Web Service

Deploying web serviceDeploying web service• Install web server (Apache)Install web server (Apache)• Install SOAP service software (NuSOAP)Install SOAP service software (NuSOAP)• Deploy prototype service (Deploy prototype service (NWSI 10-506 )

– Created Created Service Description DocumentService Description Document– Established Established service home pageservice home page

Collect user feedbackCollect user feedback• Users are encouraged to take Users are encouraged to take surveysurvey• Partners WorkshopPartners Workshop

Implement “official” serviceImplement “official” service

http://products.weather.gov/PDD/Extensible_Markup_Language.pdf

http://www.nws.noaa.gov/forecasts/xml/

http://weather.gov/survey/nws-survey.php?code=xmlsoap


Digital Age Is HereDigital Age Is Here

.TONIGHT...CLEAR. LOW IN THE MID 30S. LIGHT WIND EARLY BECOMING

Pre-digital Age …Pre-digital Age … NWS forecast data available NWS forecast data available quickly, efficiently, and in quickly, efficiently, and in convenient and understandable convenient and understandable formsforms


AcronymsAcronyms

DOM – Document Object ModelDOM – Document Object ModelDWML - Digital Weather Markup LanguageDWML - Digital Weather Markup LanguageGFE – Grid Forecast EditorGFE – Grid Forecast EditorNDFD - National Digital Forecast DatabaseNDFD - National Digital Forecast DatabaseNWS – National Weather ServiceNWS – National Weather ServiceSAX – Simple API for XMLSAX – Simple API for XMLSOAP - Simple Object Access ProtocolSOAP - Simple Object Access ProtocolUDDI - Universal Description, Discovery, and IntegrationUDDI - Universal Description, Discovery, and IntegrationWSDL - Web Service Description LanguageWSDL - Web Service Description LanguageXML - Extensible Markup LanguageXML - Extensible Markup LanguageXSLT – Extensible Stylesheet Language TransformationsXSLT – Extensible Stylesheet Language Transformations


ReferencesReferences

NDFD XML Home Page: NDFD XML Home Page: http://http://www.nws.noaa.govwww.nws.noaa.gov/forecasts/xml//forecasts/xml/NDFD XML ServiceNDFD XML Service

Service: Service: http://www.nws.noaa.gov/forecasts/xml/SOAP_server/ndfdXMLserver.phphttp://www.nws.noaa.gov/forecasts/xml/SOAP_server/ndfdXMLserver.phpWSDL: WSDL: http://http://www.nws.noaa.gov/forecasts/xml/DWMLgen/wsdl/ndfdXML.wsdlwww.nws.noaa.gov/forecasts/xml/DWMLgen/wsdl/ndfdXML.wsdlSchema: Schema: http://www.nws.noaa.gov/forecasts/xml/DWMLgen/schema/DWML.xsdhttp://www.nws.noaa.gov/forecasts/xml/DWMLgen/schema/DWML.xsdSample Products: Sample Products: http://http://weather.gov/forecasts/xml/sample_products/ndfdXML.tarweather.gov/forecasts/xml/sample_products/ndfdXML.tar

NDFD Home Page: NDFD Home Page: http://www.nws.noaa.gov/ndfd/index.htmhttp://www.nws.noaa.gov/ndfd/index.htmNuSOAP: NuSOAP: http://dietrich.ganx4.com/nusoap/index.phphttp://dietrich.ganx4.com/nusoap/index.phpSAX: SAX: http://http://sax.sourceforge.netsax.sourceforge.net//SOAP: SOAP: http://www.w3.org/TR/soap/http://www.w3.org/TR/soap/UDDI - http://www.uddi.org/UDDI - http://www.uddi.org/VH Graph: VH Graph: http://codingtheweb.users.phpclasses.org/browse/package/257.htmlhttp://codingtheweb.users.phpclasses.org/browse/package/257.htmlWSDL - WSDL - http://www.w3.org/TR/wsdlhttp://www.w3.org/TR/wsdlXML - XML - http://www.w3.org/XML/http://www.w3.org/XML/XSLT – XSLT – http://www.w3.org/TR/xslthttp://www.w3.org/TR/xslt




http://www.nws.noaa.gov/forecasts/xml/SOAP_server/ndfdXMLserver.php

http://www.nws.noaa.gov/forecasts/xml/DWMLgen/wsdl/ndfdXML.wsdl

http://www.nws.noaa.gov/forecasts/xml/DWMLgen/wsdl/ndfdXML.wsdl

http://www.nws.noaa.gov/forecasts/xml/DWMLgen/schema/DWML.xsd

http://weather.gov/forecasts/xml/sample_products/ndfdXML.tar

http://weather.gov/forecasts/xml/sample_products/ndfdXML.tar

http://www.nws.noaa.gov/ndfd/index.htm

http://dietrich.ganx4.com/nusoap/index.php

http://sax.sourceforge.net/



http://www.w3.org/TR/soap/

http://codingtheweb.users.phpclasses.org/browse/package/257.html

http://www.w3.org/TR/wsdl

http://www.w3.org/XML/

http://www.w3.org/TR/xslt

Exploiting XML and a SOAP Service to Disseminate National Weather Service Digital Forecast Data

Documents

Transcript of Exploiting XML and a SOAP Service to Disseminate National Weather Service Digital Forecast Data