ESDIS Project Status ESDIS Project Status

11/29/200611/29/2006

Dan Marinelli, Science Systems Dan Marinelli, Science Systems Development OfficeDevelopment Office

EOSDIS System EvolutionEOSDIS System Evolution

• ESDIS was directed to evolve the systems under its budget to accommodate vision identified by a joint EOSDIS Elements Study/Technical team

• Key vision elements include:

– Improve access and processing services, ensure available expert knowledge, reduce operational costs, ensure safe stewardship, maintain IT currency

EOSDIS System EvolutionEOSDIS System Evolution

• Top 3 cost drivers contribute to approx. 50 % of total budget:– EMD/ECS– GES DAAC– LaRC DAAC

• Factors that contribute to top 3 cost drivers:– Operating multiple systems (ECS, V0/V1, LaTIS, etc.) – DAAC-unique capabilities and science community

support beyond specific operation of ECS/SDPS – Providing sustaining engineering for ECS/SDPS at the

four ECS DAACs

ESDIS Evolution PathESDIS Evolution Path

• Approval has been given to embark down an evolution path

• GES DAAC and ASDC DAAC to evolve away from ECS SDPS at their sites

• MODAPS to evolve towards archive and distribution of all MODIS products

• ECS SDPS footprint to be reduced greatly in terms of hardware and custom code

• Summary of the plan can be found at http://eosdis-evolution.gsfc.nasa.gov/

EOSDIS TodayEOSDIS TodayEOSDIS provides

– A production capability for standard science data products from EOS instruments– An “active archive” of Earth science data from EOS and other past and present missions – A distributed information framework (data centers, SIPS, networks, interoperability, other system

elements) with partners supporting EOS investigators and other users in science, government, industry, education, and policy

EOSDIS_Today_11222006. xls

Non-ECS (reporting)ECSEOSDIS Metrics for FY2006

41.171.7Total Archive Products at the End of FY06 (in millions)

0.943.67Total Archive Volume at the End of FY06 (in PB)

4.165.6End User Distribution Products (in millions)

0.162.86End User Daily Distribution Volume (in TB)

0.652.40Daily Archive Growth with Deletion (TB/day)

0.653.21Daily Archive Growth without Deletion (TB/day)

2743,033Daily Ingest Volume (in GB)

Non-ECS (reporting)ECSEOSDIS Metrics for FY2006

41.171.7Total Archive Products at the End of FY06 (in millions)

0.943.67Total Archive Volume at the End of FY06 (in PB)

4.165.6End User Distribution Products (in millions)

0.162.86End User Daily Distribution Volume (in TB)

0.652.40Daily Archive Growth with Deletion (TB/day)

0.653.21Daily Archive Growth without Deletion (TB/day)

2743,033Daily Ingest Volume (in GB)

3,291,397Distinct Users over FY06 at DAACs

37.8M (est.)Number of Accesses at DAACs

3,362Unique Data Products

50System Interface Control Docs (ICDs)

EOSDIS SystemsEOSDIS Overall Metrics (FY2006)

3,291,397Distinct Users over FY06 at DAACs

37.8M (est.)Number of Accesses at DAACs

3,362Unique Data Products

50System Interface Control Docs (ICDs)

EOSDIS SystemsEOSDIS Overall Metrics (FY2006)

10 of 14SIPS

8DAACs

No.ESDIS Funded Entities

10 of 14SIPS

8DAACs

No.ESDIS Funded Entities

15International

8U.S.

No.Partnerships

15International

8U.S.

No.Partnerships

70Instruments Supported

26Archiving and Distribution

10Science Data Processing

No.Missions

70Instruments Supported

26Archiving and Distribution

10Science Data Processing

No.Missions

EOSDIS Mission ProfileEOSDIS Mission Profile

file name: EOS missions vs time6.xls

2011 2012 2013 20142007 2008 2009 20102003 2004 2005 20061999 2000 2001 20021995 1996 1997 19981991 1992 1993 1994

2011 2012 2013 20142007 2008 2009 20102003 2004 2005 20061999 2000 2001 20021995 1996 1997 19981991 1992 1993 1994

10/1984 - 2 years planned mission life

10/1978 - 10 yrs m life

ERBS (SAGE II)

ERS-1

NIMBUS-7 (TOMS I)

Meteor 3 TOMS (TOMS II)

Planned decay 2010UARS

TOPEX/Poseidon

JERS-1

OrbView-2 (SeaStar)

ERS-2

Earth Probe TOMS (TOMS III)

RadarSat 1

ADEOS I (Midori)

ADEOS II (Midori

Jason-1

ACRIMSAT

Meteor 3M

EOS Aqua

ICESat

EOS Aura

SORCE

3 yrs

Landsat 7

QuikSCAT

EOS Terra

3 yrs

TRMM CERES data only Heritage Missions

EOS Missions

Extended Mission Life

Planned Mission Life

4 yr Data Access Period(includes 3 yr reprocessing)

No Planned EOL

KEY

3 yr Reprocessing Pd

Archive Volume TrendArchive Volume Trend

FY03 FY04FY05 FY06

GSFCEDC

LARC

NSIDC

100

200

300

400

500

600

700

Archive Volume (TB)

Granules into Archive TrendGranules into Archive Trend

FY03FY04

FY05FY06

GSFCEDC

LARC

NSIDC

0

2,000,000

4,000,000

6,000,000

8,000,000

10,000,000

12,000,000

14,000,000

16,000,000

Archive Granules

Distribution Volume TrendDistribution Volume Trend

FY03 FY04 FY05 FY06

GSFCEDC

LARCNSIDC

0

50

100

150

200

250

300

DAAC Distribution Volume (TB)

Distribution Granules TrendDistribution Granules TrendDistribution Granules TrendDistribution Granules Trend

FY03 FY04 FY05 FY06

NSIDCEDC

LARCGSFC

0

1,000,000

2,000,000

3,000,000

4,000,000

5,000,000

6,000,000

7,000,000

8,000,000

Number of Granules Distributed

User Access StatisticsUser Access StatisticsDAAC Fiscal Year 2006 Access Statistics for ECS and Non-ECS Data

Number of Distinct Users Accessing DAACs

TITLE ASF EDC GHRC GSFC JPL LARC NSIDC ORNL SEDAC TOTAL

ECS Orders/Subscriptions   5,535   3,117   463 811       9,926

Non-ECS Orders/Subscrip 127 0 152 284 41 399 414 119 0   1,536

WWW (users) 25,563 0 174,764 294,271 86,862 279,545 644,943 170,943 814,770   2,491,661

FTP (users) 56 0 181 2,884 1,545 0 530 1,296 556   7,048

Off-line (users) 0 0 26 419 224 981 611 791 376   3,428

Datapool (users) 0 2,463 0 2,831 0 688 512 0 0   6,494

Total (users) 25,746 7,998 175,123 303,806 88,672 282,076 647,821 173,149 815,702   2,520,093

EOSDIS 2006 Customer EOSDIS 2006 Customer Satisfaction SurveySatisfaction Survey

EOSDIS 2006 Customer EOSDIS 2006 Customer Satisfaction SurveySatisfaction Survey

• EOSDIS’ third survey, about 2800 responders

• Survey has changed slightly each time, but the standard questions for measuring satisfaction are the same

• 2006 survey addressed product search, selection and order, distribution, quality, documentation and customer support

Respondent BackgroundRespondent BackgroundRespondent BackgroundRespondent Background

2%

3%

4%

6%

9%

9%

10%

11%

16%

17%

20%

22%

21%

22%

22%

25%

27%

26%

33%

37%

44%

47%

1%

3%

4%

5%

8%

8%

10%

11%

15%

15%

16%

19%

21%

21%

21%

23%

23%

25%

32%

36%

41%

45%

0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50%

Flight dynamics

Space Weather

Sun-Earth Connections

Socioeconomics

Solid Earth

Other

Cryosphere

Planning

Carbon Cycle

Geolocation

Radiance

Education

Resources

Natural Hazards

Weather

Water and Energy

Oceans

Agriculture

Ecosystems

Atmosphere

Climate/Climate Change

Land Cover/Land Use

2005 2006

Q8. For which disciplines do you need or use Earth science data? (n=2,857)*

Customer Survey Product Quality Customer Survey Product Quality

Q27. In what format were

your data products

provided to you?

Q28. What format would

/ do you prefer?

HDF-EOS 37% 21%HDF 30% 21%NetCDF 3% 8%Binary 6% 7%ASCII 6% 9%TIFF or GeoTIFF 10% 23%JPEG, GIF, PNG 3% 4%OGC Web services 1% 2%Other 3% 5%

Summary of HDF-related comments to the CFI Survey

(Informally assessed)

Summary of HDF-related comments to the CFI Survey

(Informally assessed)Need tools for data handling 20

More useful for various user types (GIS/educational/science) if in some other than HDF format. 19

Make the distribution format fit what the user wants (or at least add more flexibility) 14

Would prefer NetCDF 8

Need to pay attention to usability of data products 3

Need documentation to extract from HDF-EOS (or HDF) file 2

Need program to extract or convert HDF-EOS Data 2

Hard for nonprogrammers to handle HDF 1

HDF-EOS is a bad format for multi-spectral/multi-angle data sets 1

Need windows tool for HDF 1

No response to help 1

We read HDF-EOS data as HDF5 1

How May We Help You?How May We Help You?

• The data gleaned from the survey leads us to conclude that the ESDIS Project needs to examine solutions for the areas of:– Data handling support software– Preprocessed/flexibly-formatted data access

paths– NetCDF– GeoTIFF when applicable