Post on 04-Jan-2016
VOYAGER Data Explorer: Architecture and TechnologiesSee also Design and Applications
Built and used Used by a Virtual Community on Aerosols
Layered Map
Time Chart
Providers Users
Vector
GIS Data
XDim DataSQL Tables
WebImages
Voyager Web Services
Publish, Find, Bind
Data & Tool Catalog
Uniform Access
Scatter Chart
S u p p o r tC o o r d i n a t i o n T e c h n o l o g i e s
Web Services Components and Actions
• Service providers publish services to a service broker.
• Service users find the needed service and get access key from a service broker• With the access key, users bind to the service provider• The result is a dynamic binding mechanism between the service users and providers
Service Broker
Service Provider
PublishFind
BindServiceUser
Components: Provider – User – Broker
Actions: Publish – Find - Bind
Web Application: Chained Web Services
• A Web Service Provider may also be a User of other services • Multiple web services can be chained into an interactive workflow system • The result is an agile application that can be created ‘just in time’ by the user for a specific need
Service Broker
Service Provider/User
PublishFind
BindServiceUser
Chain
Service Provider
Bind
Chain
Web PublishHTTP, FTP
Data Access though a Web Service Adapter
Service Broker
Publish
Service Consumer
Find
Access
Ordinary web content can be delivered as a Web Service through a Proxy Server.
• The Adapter Service converts HTTP/FTP service to XML Web service
• The Adopter Service publishes the web service to the Broker
• The User finds the data from the broker and accesses the Adapter to get distributed data
Service Adapter
WebServer
ServiceUser Chain
Interoperability through Web Services
• Interoperability: Independent of platform and language, based on XML Web Services
• Service Transparency: Published capabilities of components; implementation opaque.
• Just-in-time integration: Discovery, access to and ad-hoc chaining of services.
• Legacy Support: Encapsulating existing components and exposing them as services.
4 D Geo-Environmental Data Cube (X, Y, Z, T)
Environmental data represent measurements in the physical world which has space (X, Y, Z) and time (T) as its dimensions.
The specific inherent dimensions for geo-environmental data are: Longitude X, Latitude Y, Elevation Z and DateTime T.
The needs for finding, sharing and integration of geo-environmental data requires that data are ‘coded’ in multidimensional data space
4D Geo-Environmental Data Illustration
• Animated 3D Maps
Hierarchy of Data Objects:
DataGranule, Data Series, DataCube
Measure A measure (in OLAP terminology) represent numerical
values for a specific entity to be analyzed (e.g. temperature, wind speed, pollutant).
A collection of measures form a special dimension ‘ Measures’ (??Can Measures be Dimensions??)
Data GranulesA data granules– discrete, atomic data entities that cannot be further broken down.
A data series is an ordered collection of data granules DataSeries is a collection of DataGranules having common attributes
All data points in a measure represent the same measured parameter e.g. temperature. Hence, they share the same units and dimensionality.
The data points of a measure are enclosed in a conceptual multidimensional data cube; each data point occupies a volume (slice or point) in the data cube.
Data points in a measure share the same dimensions; Conversely, each data point has to have the dimensional coordinates in the data cube of the measure that it belongs to.
Dimension Y
DataSeriesDimension Z
Dimension X
DataCube
DataGranule
Catalog of Multidimensional Datasets
• Designed to Publish, Find(Select), Bind(Access) and Render distributed datasets• Publishing is though an open web interface for user/broker registration of datasets• Finding a dataset is aided by a metadata on Provider and Dataset• Binding (data access) information is contained in the Dimensional Tables• Rendering parameters are also contained in the Dimensional Tables
Multi-dimensional Data Access
• In array notation, the granule ‘value’ is accessed as – MyGranule = My1DArray(i)
– MyGranule = My2DArray(i,j)
– MyGranule = MynDArray(i,j,…..n)
• In order to select a data granule, a controller is assigned to each data dimension
1D Dataset
e.g. Time selector
i
j
k
j
i
Data Granule
Selection
i2D Dataset
e.g. Param & Time selector
3D Dataset
e.g. Param, Time & Location
DVoy Data Space
XminZmin
Zmin
XmaxYmin
Ymax
Data Views
Z
X
Y
Data Space for a Measure
Ymax
Xmin XmaxYmin
DataZmin
Zmax
•aergqegqet
View Data Space
Data Granule
Measure Data Space
Geo-referenced Map Image: Time Indexed
Regular Time Series; Fixed Georect; Image Trim
<measure code="TOMS01" name="TOMS - Aerosol Index - Glob" descr="Earth probe TOMS aerosol index">
<data_source service_url="http://capita.wustl.edu/dataservice/data.asmx"/>
<data_facts>< fact_attributes >
<data name ="TOMS" type="map image" descript="/>
<granul_query handler_assembly= (access handler)…./>
<image width=“640" height=“480“ margin_left="69" margin_top="46"/>
< georect lat_ min="-65" lat_ max="65“ lon_min="-180“ lon_max="180“ unit ="DecDeg“ />
</ fact_attributes >
</ data_facts>
<data_dimensions>
<time_dimension
handler_assembly="CAPITA.data.index.impl"
handler_class="CAPITA.data.index.impl.datetime_interval"
start_time="1996-07-25"
end_time="yesterday">
sample_periodicity _unit = "day"
sample_periodicity_mult = “1"
</time_dimension>
</data_dimensions>
</measure>
TOMSFact
TimeIndex Int, Calc from date
DailyImage Img, Parsed from date
LatMin Fixed, -65
LatMax Fixed, 65
LonMin Fixed, - 180
LonMax Fixed, 180
TOMSTime
TimeIndex
DateTime
Virtual Tables (No physical StarTables) XML Example: TOMS
ftp://jwocky.gsfc.nasa.gov/pub/eptoms/images/aerosol/y2002/ea020510.gif
Geo-referenced Map Image: Image IndexedVariable Time , Var GeoRect, Var Sensor
SatGeoImg
CapImgID Universal ID
ProviderURL URL
SourceURL URL
ImageURL URL
ProviderAbbrev NASA GSFC
ImageDescription Text
Sensor Enumerated
DateTime DateTime
LatMin -65
LatMax 65
LonMin - 180
LonMax 180
ImageWidth For size verification
ImageHeight
MarginLeft For trimming
MarginTop
MarginRight
MarginBottom
SQL Table
<measure code=“SeaWiFS01" name=“SeaWiFS – Reflectance Images - Glob" descr=" SeaWiFS – Reflectance Images - Global">
<data_source service_url="http://capita.wustl.edu/dataservice/data.asmx"/>
<data_facts>< fact_attributes >
<data name ="SeaWiFS" type="map image" descript=“stuff stuff “/>
<granul_query handler_assembly= (access handler)…./>
<fact table, SQL same as CIRA >
</ fact_attributes > … see table left
</ data_facts>
</measure>
NO DIMENSION TABLES
XML Example: SeaWiFS
Astronaut Photos: Image IndexedVariable Time, Image Center, Platform SatImg
NASAImageID Universal ID
ProviderURL http://eol.jsc.nasa.gov/
SourceURL http://eol.jsc.nasa.gov/scripts/sseop/photo.pl?mission=ISS004&roll=E&frame=11079
ImageURL http://eol.jsc.nasa.gov/sseop/images/ESC/small/ISS004/ISS004-E-11079.JPG
ProviderAbbrev NASA JSC
ImageDescription Text
Platform ISS004
GeoRegion GUATEMALA
Features SMOKE, GULF OF HONDURAS
DateTime 20020501 212700
Lat 17.6
Lat -90.2
SQL Table
<measure code=“AstroPhoto01" name=“Astronaut Photographs " descr=" Astronaut Photographs ">
<data_source service_url="http://capita.wustl.edu/dataservice/data.asmx"
/>
<data_facts>< fact_attributes >
<data name ="SeaWiFS" type="map image" descript=“stuff stuff “/>
<granul_query handler_assembly= (access handler)…./>
<fact table, SQL same as CIRA >
</ fact_attributes > … see table left
</ data_facts>
</measure>
XML Example: ISS004
Atmospheric Photos: Image IndexedVariable Time, Image Center, PlatformAtmoImg
ImageID Universal ID
ProviderURL http://capita.wustl.edu/capita/people/RHusar/rhusar.html
SourceURL http://capita.wustl.edu/capita/people/RHusar/Pic/9812SanFranciscoAGU/9812SanFranciscoAGU.htm
ImageURL http://capita.wustl.edu/capita/people/RHusar/Pic/9812SanFranciscoAGU/Dcp01000Enh.jpg
Provider Abbrev R Husar
ImageDescription Text
Platform Airplane
GeoRegion NEVADA
Features DUST PLUME
DateTime 19981208 1800
Lat 36.75
Lat -115.15
SQL Table
<measure code=“AtmosPhoto01" name=“Atmospheric Photographs " descr=" Atmospheric Photographs ">
<data_source service_url="http://capita.wustl.edu/dataservice/data.asmx"
/>
<data_facts>< fact_attributes >
<data name ="SeaWiFS" type="map image" descript=“stuff stuff “/>
<granul_query handler_assembly= (access handler)…./>
<fact table, SQL same as CIRA >
</ fact_attributes > … see table left
</ data_facts>
</measure>
XML Example: Dust Plume Photo
Interoperability through a Layered Protocol Stack
• Web Services are implemented on a layered stack of technologies and standards
• The lower layers enable binding and exchange of messages; higher levels enable interoperability
• Applications are formed dynamically from distributed components through publish-find-bind mechanisms
TCP/IP, HTTP, FTP
ASCII, XML, etc.
HTML, XML OGC -GML
OGC Coverage, CoordTransfom, WMS
HTTP, SOAP
WSDL
UDDI OGC Catalog
WSFL, XLANG
StandardsInteroperability
Comm. Protocols
Data Encoding
Data Schema
Data Binding
Web Service
Service Integr.
Service Discovery
Service Descript.
Connectivity
Web Services Standards
• Discovery and Integration: UDDI (Universal Description, Discovery and Integration)
• Service Description: WSDL (Web Services Description Language)
• Content Envelope: SOAP (Simple Object Access Protocol)
• Data Encoding: XML (Extensible Modeling Language)
• Service Chaining: WSFL (Web Services Flow Language, IBM) or XLANG (Message flow language, Microsoft)
ChainWSFL/XLANG
Service Broker
Service Provider/User
PublishUDDI, WSDL
FindUDDI, WSDL
BindSOAP, XML
ServiceUser
Chain
Service Provider
Bind
Publish, Find, Bind
Voyager Data Services Architecture
Scatter Chart
Text, Table
Data View & ProcessLayered Map
Cursor
Homogeneous data used by Viewing and Processing
services in ‘agile’ applications
Heterogeneous data: different types, coding and
access protocols
Connects data providers with data users
Transforms hetero to virtual homogeneous data
Time Chart
Providers Users
XML WebServices
Satellite
Vector
GIS Data
XDim Data
OLAP Cube
SQL Table
HTTP, FTP
Web
Text
OpenGISServices
Images
Access Services
Data & Tool Catalog
Uniform Access/Retrieval
Select, Overlay, Explore; Multidimensional data
Maintain Distributed Data; Heterogeneous coding , access
Connect providers to users; Homogenize data access
Voyager Spatio-Temporal Data Explorer Built and Used by a Virtual Community
Layered Map
Time Chart
Providers Users
Vector
GIS Data
XDim Data
SQL Table
Web
Images
Voyager Web Services
Publish, Find, Bind
Data & Tool Catalog
Uniform Access/Retrieval Scatter Chart
Navigation Service
DVoy: Components and Data Flow
Legacy Data
Publish
(DataSet)
DataSet Records
• Provider Descript.
• Service Descript.
• Measure Access
Find (Measure)
DataSet Recs
Selected Measure Record
• Dvoy
Data Wrapping
WebServiceBind (Measure, FocusCube)
Data Delivery
WebServiceFocusCube, GlobCursor,
Measure, Granule
Time Chart
Layered Map
DataToView
DataForCursorAndView
Catalog Service Data Services
Data Delivery
WebService
Presentation Services
Data Focus Range
Rendering
Cursor
Viewer Layers
Dim1: LonDim2: Lat
Data provided by each dimension of a View:
Dim1.Type, Dim1.Min, Dim1.Max
Dim2.Type, Dim2.Min, Dim2.Max
….
Current Dim.Types:
Latitude, Longitude, DateTime, Elevation
Federated Data Services Architecture
XML WebServices
Satellite
Vector
GIS Data
XDim Data
OLAPCube
SQLTable
HTTPServices
Text Data
WebPage
TextData
Scatter Chart
Text, Table
Data View & Process TierLayered Map
Cursor
Data Warehouse Tier
Data View
Manager
Connection
Manager
Data Access
Manager
Cursor-Query
Manager
OpenGISServices
Data are rendered by linked Data Views (map, time, text)
Distributed data of multiple types (spatial, temporal text)
The Broker handles the views, connections, data access, cursor
Time Chart
Major Service Categories (OGC)
Service Category Description
Human Interaction Managing user interfaces, graphics, presentation.
Info. Management Managing and storage of metadata, schemas, datasets.
Workflow Services that support specific tasks or work-related activities.
Processing Data processing, computations; no data storage or transfer
Communication Services that encode and transfer data across networks.
Sys. Management Managing system components, applications, networks (access).
0000 OGC web service [ROOT]1000 Human interaction1100 Portrayal1110 Geospatial viewer1111 Animation1112 Mosaicing1113 Perspective1114 Imagery1120 Geospatial symbol editor1130 Feature generalization editor1200 Service interaction editor1300 Registry browser2000 Information Management2100 Feature access2200 Coverage access2210 Real-time sensor2300 Map access2400 Gazetteer2500 Registry2600 Sensor access2700 Order handling3000 Workflow3100 Chain definition3200 Enactment3300 Subscription4000 Processing4100 Spatial4110 Coordinate conversion4120 Coordinate transformation4130 Representation conversion4140 Orthorectification4150 Subsetting4160 Sampling4170 Feature manipulation4180 Feature generalization4190 Route determination41A0 Positioning4200 Thematic4210 Geoparameter calculation4220 Thematic classification4221 Unsupervised4222 Supervised4230 Change detection4240 Radiometric correction4250 Geospatial analysis4260 Image processing4261 Reduced resolution generation4262 Image manipulation4263 Image synthesis4270 Geoparsing4280 Geocoding4300 Temporal4310 Reference system transformation4320 Subsetting4330 Sampling4340 Proximity analysis4400 Metadata4410 Statistical analysis4420 Annotation5000 Communication5100 Encoding5200 Format conversion5300 Messaging6000 System Management
OGC code Service class
Semi-Static Views (slices) through 4D Data Space
• Possible Cross-sections through the 4 D Data space
- data point..Temperature (xi, yi, zi, ti)
- image Temperature (xrange , yrange, zrange, trange)
XY MAP: Z,T fixed
Vertical Profile:XYT fixed
Time Chart: X,Y,Z fixed
Vertical Cross sect: YT fixed Vertical Cross sect: XT fixed
Vertical Profile Trend:
X,Y fixed
Data Catalog
• All the data in the system are to be distributed on the Web and maintained by their custodians
• The purpose of the catalog is to help finding and and accessing the data
• Catalog would be limited to data that can be accessed/merged in DVOY
DVoy Queries as Web Services
Purpose: Locating relevant data measures fir specific location and time
Abstract Query: Find available measures in MyDataCube
Web Service: input: MyDataCube; output: List of measures, MeasureDataCube
Design: Measure table with bounding MeasureDataCube cubes
Implementation: SQL measures table with MeasureDataCubeSELECT Measures, MeasureDataCube WHERE MeasureDataCube in MyDataCube
Distinct Locations, Times, Heights
D-DADS Architecture (OOOOLD)
ARC/INFO
VirtualDataCube
ArcSDETranslator
OLAPService
Provider
DataCube
LegacyDatabase
CustomOLAP
Translator
DataCube
SQLDatabase
OLAP ServiceProvider
GISTable
OLAP
StandardizedDescription &
Format
Database(SQL,
Oracle,etc.)
ArcSDETranslator
GISMap
DataProviders
Data Access andManipulation Tools
UserInteraction
DataCube
ArcIMS
Dvoy Proto
DVoy Project
• Goal: Web based data finding/access/delivery system
• Support: – EPA WebVis (Stefan): Aug 2001- Apr 2003
– NSF Collaboration: Aug 2001- Aug 2003
• CAPITA $$ is secured until Apr 2003, after that only very limited
• Need to sell project NOW to continue after Apr 2003
• Partners: Bret, Stefan, Poirot, Sci peers
DVoy: Data Access, Display, Browser, Content
• Data Access: Indexed data access framework. (Kari, ‘done’)
• Data Display: Spatial, temporal other data rendering (Kari & Jer, not done)
• Browser: A specific ‘application’ - linked multidim views (Kari & Jer, not done)
• Content: Encoding and registering data sets (Bob & Sean, in progress)
Project Schedule: May-July 2002
Goal for May-July 2002: Data Viewer, Dvoy BrowserMay 16: Data access framework ‘ready’
Viewer, some data
May 24: Viewer Proto Demo/selling to partners: Bret, Stefan, Poirot, Add content CAPITA
Jun 7: Spatial/Temporal Viewer ReadyDemo/selling to ‘outsiders’: NASA, NSF Report, NOAA, EPA, Sci. Peers
Add content CAPITA, partners
Jul 17: Functional browserDevelop other ‘applications’ – data finding/renderings service web pages
Add content CAPITA, partners
SQL Server
Data Access though a Web Service Portal
Service Broker
PublishUDDI, WSDL
Service Consumer
FindUDDI, WSDL
Access
SOAP, XML
Ordinary web content can be delivered as a Web Service through a Proxy Server.
• The Portal Server supplies a web server-to-web-service ‘adapter’
• The Portal Server publishes the web service to the Broker
• The User accesses the Portal to get the distributed Web Server data
DataSets
TimeViewLocView
Star Schema
DimTablesFactTable
ServiceUser Chain
Service Portal