Giovanni-4: The Next Generation - NASA · Giovanni-4: The Next Generation C Lynnes, M Hegde, C...

11/28/2014

1 Goddard Earth Sciences

Data Information Services Center

GES – DISC

Giovanni-4:

The Next Generation

C Lynnes, M Hegde, C Smit, D DaSilva, K Bryant, P. Zhao, Z. Liu, S. Shen,

A. Savtchenko, W. Teng, J. Wei, J. Acker

https://ntrs.nasa.gov/search.jsp?R=20160005868 2018-09-08T00:55:37+00:00Z

11/28/2014

GES – DISC

Outline

• Overview of Giovanni

• New and Notable Features

• Discussion?

11/28/2014

GES – DISC

Giovanni and “Big Data”

WRITE CODE

Select Data

Download

Learn format

Summarize / Analyze

Analyze

Quality Filter

Subset

Publish

Derive Conclusions

Visualize

Exploratory Data Analysis

Main Analysis

11/28/2014

GES – DISC

Why So Difficult?

WRITE CODE

Select Data

Download

Learn format

Summarize / Analyze

Analyze

Quality Filter

Subset

Publish

Derive Conclusions

Visualize

Main Analysis

FTP Data Products up to 1 GB Volume

Summarize data over long time periods, large areas

Volume

11/28/2014

GES – DISC

Why So Difficult?

WRITE CODE

Select Data

Download

Learn format

Summarize / Analyze

Analyze

Quality Filter

Subset

Publish

Derive Conclusions

Visualize

Main Analysis

2000+ datasets Variety

HDF4, HDF5, HDF-EOS2, HDF-EOS5, netCDF, GRIB, binary

Variety

Files up to 800+ variables Variety

Locate, read, use geolocation info Variety

11/28/2014

GES – DISC

Remote Sensing Science: Why So Difficult?

WRITE CODE

Select Data

Download

Learn format

Summarize / Analyze

Analyze

Quality Filter

Subset

Publish

Derive Conclusions

Visualize

Main Analysis

FTP Data Products up to 1 GB

HDF4, HDF5, HDF-EOS2, HDF-EOS5, netCDF, GRIB, binary

Files up to 800+ variables

More studying, coding

Locate, read, use geolocation info

Volume

Variety

Veracity

Variety

2000+ datasets Variety

11/28/2014

GES – DISC

Giovanni provides (relatively) rapid exploration of datasets

WRITE CODE

Select Data

Download

Learn format

Summarize / Analyze

Analyze

Quality Filter

Subset

Publish

Derive Conclusions

Visualize

Main Analysis

Giovanni provides Server-Side, Quick-Start

Exploratory Data Analysis:

no coding necessary

no downloads necessary

Web-based Services

Reformat

Filter

Regrid

Summarize

Visualize

11/28/2014

GES – DISC

What’s New?

• Giovanni-4, relative to Giovanni-3 – Omnibus Giovanni Portal

– Federated Giovanni

– Variable selection interface

– Sharable/reproducible URLs

• New in Giovanni-4 – More comparison functions

– Vector Plots

– Seasonal Analysis

– Shapefile-based Area Averages

11/28/2014

GES – DISC

Omnibus Portal

• All variables in one portal

– Compare across instruments and disciplines

– Caveat: Multi-temporal comparisons not yet supported

• However: Requires more navigational aids

11/28/2014

GES – DISC

Federated Giovanni will further increase the availability of data in Giovanni

• Ongoing now: Funded by NASA-ACCESS: 2014-2016

• Eventual release of source and Virtual Machine to the community

MODAPS

Giovanni

LP DAAC

Giovanni

PO DAAC

Giovanni

11/28/2014

GES – DISC

MODIS NDVI from LP DAAC

11/28/2014

GES – DISC

SST from PO.DAAC

11/28/2014

GES – DISC

Giovanni-4 User Interface

Sortable columns

Search with Autosuggest

Faceted Browse

11/28/2014

GES – DISC

11/28/2014

GES – DISC

Keyword search on precipitation

11/28/2014

GES – DISC

Reset with button or by clearing search

11/28/2014

GES – DISC

Selecting a Discipline “Facet”

11/28/2014

GES – DISC

Selecting a Discipline “Facet”

Number of matching variables: 16 of 219

11/28/2014

GES – DISC

Narrow by selecting more facets

11/28/2014

GES – DISC

Selecting variables leads to hints on date range

11/28/2014

GES – DISC

Selecting variables leads to hints on date range

Valid Range: 1997-12-31 to 2014-08-31

11/28/2014

GES – DISC

You can also sort on columns

11/28/2014

GES – DISC

Spatial and temporal resolution facets are also available

11/28/2014

GES – DISC

Giovanni-4 selections are encoded in the URL Making them Sharable and Reproducible

http://giovanni.gsfc.nasa.gov/giovanni/#

service=INTERACTIVE_MAP&

starttime=2001-01-01T00:00:00Z&

endtime=2001-01-31T23:59:59Z&

bbox=-124.4531,24.2578,-68.2031,52.3828&

data=MOD08_D3_051_Optical_Depth_Land_And_Ocean_Mean&

variableFacets=dataFieldDiscipline%3AAerosols%3B

Service

Time Range

Bounding Box

Data Variable

11/28/2014

GES – DISC

Service

Time Range

Bounding Box

Data Variable

Giovanni-4 URLs are Sharable and Reproducible

11/28/2014

GES – DISC

Data comparison services to compare similar datasets

• Correlation Map

• Difference of Time-Averaged Maps

• Time Averaged Scatterplot

• Area Averaged Scatterplot

• Time Series of Differences of Area Averages

11/28/2014

GES – DISC

2010-2011

Correlation Map: TRMM Rainfall Versions 6 vs. 7

11/28/2014

GES – DISC

Difference of Time-Averaged Maps

• LPRM from X-band is more affected by vegetation than by C. This difference map shows that C and X don't differ that much when the surface is bare to sparsely vegetated, but do differ when the surface is more densely vegetated and soil moisture retrieval becomes problematic. (Courtesy Bill Teng)

11/28/2014

GES – DISC

MODIS NDVI from LP DAAC

11/28/2014

GES – DISC

New Services

• Vector Plots

• Seasonal Analysis

– User-defined Climatology

– Interannual Time Series

• Shapefile-based Area Average Time Series

11/28/2014

GES – DISC

Vector Plots

• Available for some wind-related variables

– More coming soon…

• Time Averaged Map service only

• Computes averages of U and V, then vector magnitude + direction

11/28/2014

GES – DISC

Vector Plots (Time-Averaged Map)

Wind Stress

Dec 1997 (El Nino)

Dec 2000 (La Nina)

Courtesy Chung-Lin Shie

11/28/2014

GES – DISC

User-Defined Climatology

Available for Monthly data Aggregate by Month or Meteorological Season (DJF, MAM, JJA, SON)

11/28/2014

GES – DISC

User-Defined Climatology

Click Here to Repeat

March, 2001-2013

September, 2001-2013

MODIS Monthly AOD

Comparison of aerosol regimes in March vs. Sept. over 2001-2013. (Courtesy Suhung Shen)

11/28/2014

GES – DISC

Seasonal Analysis: Interannual Time Series

• Available for Monthly data

• Plot the area average for <month> for the years <start> to <end>

11/28/2014

GES – DISC

Interannual Time Series Example

Strong summer (July) warming, on the order of 0.5K/dec during the past decade, is observed by AIRS over the permafrost of the Tundra region. (Courtesy Andrey Savtchenko)

Click here to repeat

11/28/2014

GES – DISC

Shapefile Support

• Area-Averaged Time Series

• Coming soon…time-averaged map, histogram

• Limitations: does not work across dateline

11/28/2014

GES – DISC

Shapefile-based Area Averaged Time Series

Compute average over area, weighted by % coverage by shape and cos(lat)

11/28/2014

GES – DISC

Shapefile-based Area Averaged Time Series

TRMM Rainfall for Oct. 2012, showing Hurricane Sandy landfall. (Courtesy Zhong Liu)

11/28/2014

GES – DISC

Shapefile based Computation of MODIS-Terra Monthly AOD

Courtesy Jennifer Wei

11/28/2014

GES – DISC

Giovanni 4 Performance

0 500 1000 1500 2000 2500 3000

Days of Data

Time Averaged Map

0 100 200 300 400 500 600 700 800

Days of Data

Area-averaged Time Series

11/28/2014

GES – DISC

DISCUSS…

11/28/2014

GES – DISC

Features Still on the TO DO list

Pre-computed Climatology (up next) OMI Level 2G Level 2 data support Cross sections (Z-Lat, Z-Lon, Z-Time) Time series of area statistics Zonal mean Multi-variable plots (overlays) Web Coverage Service interface MyGiovanni: user-contributed color palettes, shapefiles,

data, algorithms Geometric Mean Units conversion More download options (KMZ, more ASCII, batch)

Your input can help us prioritize them

11/28/2014

GES – DISC

Giovanni was designed for exploratory data analysis…

WRITE CODE

Select Data

Download

Learn format

Summarize / Analyze

Analyze

Quality Filter

Subset

Publish

Derive Conclusions

Visualize

Main Analysis

Web-based Services

i Find

Reformat

Filter

Regrid

Summarize

Visualize

1. …Should Giovanni evolve to support the Main Analysis Phase better? 2. What modifications would enhance “scientific rigor”?

Giovanni-4: The Next Generation - NASA · Giovanni-4: The Next Generation C Lynnes, M Hegde, C...

Documents

Transcript of Giovanni-4: The Next Generation - NASA · Giovanni-4: The Next Generation C Lynnes, M Hegde, C...

Best Practices to Promote Data Interoperability Chris Lynnes Joe Glassy Technology Infusion Working Group.

Giovanni colonization3

Resource sharing Giovanni De Micheliusers.ece.utexas.edu/~gerstl/ee382v-ics_f09/lectures/lecture_13.pdf · 4 (c) Giovanni De Micheli 7 Compatibility and conflicts Compatibility graph:

Giovanni Broggi

2010 Hyundai Genesis Brochure Lynnes Hyundai Bloomfield NJ

Christopher Lynnes, Richard Strub , Thomas Hearty, Bruce Vollmer

David C. Ku and Giovanni De Michelidemichel/publications/archive/... · David C. Ku and Giovanni De Micheli Center for Integrated Systems, Stanford University, Stanford, CA 94305-4055,

The Last Supper (c. 1520) by Giovanni Pietro Rizzoli.

2010 Hyundai Genesis Coupe Brochure Lynnes Hyundai Bloomfield NJ

Giovanni Michelucci BUILDING DOCUMENTATION GIOVANNI …

GIOVANNI CROCE (c.1557 - 1609) MISSA TERTIA OCTAVI ..._G._Missa...GIOVANNI CROCE (c.1557 - 1609) MISSA TERTIA OCTAVI TONI 1596 Performance edition by William Evans, 2016. Transcribed

Giovanni colonization2

2010 Hyundai Elantra Brochure Lynnes Hyundai Bloomfield NJ

Scheduling Giovanni De Micheli - University of Texas at Austinusers.ece.utexas.edu/~gerstl/ee382v-ics_f09/lectures/lecture_12.pdf · 3 (c) Giovanni De Micheli 5 Taxonomy Unconstrained

2010 Hyundai Azera Brochure Lynnes Hyundai Bloomfield NJ

Brown, Giovanni

Giovanni Crupiwebservices.unime.it/curriculum/uploads/cv_eng_810951fc... · 2017-02-28 · Giovanni Crupi C U R R I C U L U M V I T A E 1 M a r c h 2017 1 PERSONAL DATA Birth’s

GIOVANNI SHOWS HIS REGION WITH 5 C. Marche - all in a region Luigi & Mattia C.

A Menagerie of ESIP OpenSearch Clients C. Lynnes, NASA/GSFC K. Keiser, U. Alabama--Huntsville.

Lynnes Master Class Slides 5.21.07