Data Services Branch

U.S. Department of the InteriorU.S. Geological Survey

Douglas BinnieUSGS EROS Center

USGS EROS Center

Mission Area Data Acquisition, Archiving and Access

Ensure that scientists, decision makers, and the public have ready access to land change information and to safeguard and expand the national archive of remotely sensed land data.

• Preserve and distribute the centers data and information products

• Manage the NASA EOS/DAAC requirements

• Operate existing user interfaces and develop future information access methodologies

• Manage hazards and disaster response activities• Operate user services and application assistance support in concert with

the other branch's

• Support the development, implementation, and operation continuum thru direct coordination with science based personnel

Data Services Branch Chief

D. Binnie

Long Term Archive Project

Manager R. Longhenry

Spatial Data Warehouse

ManagerC. Rusanowski

Acting

LS Access and Dist. O&MSentinel 2

K. Kline

LP DAAC

Manager

C. Doescher

LP DAAC Scientist1 - Vacant Term1 - Vacant Perm

(AB Staff)

LP DAACOperations

ManagerC. Torbert

Emergency Operations

Project Manager R. Longhenry

Emergency Ops Liaison

B.Jones

Data Management

Specialist R. Saleh

Computer Scientist

C. Rusanowski

FMO Staff Support

L. DeBoer

Emergency Ops Liaison

Vacant

Administration Support

N. Vandenbussche

Data Services Branch Organization

Customer Engagement

R. Lamb

EROS Consolidated Report on Data ManagedAll Projects with Detail – Monthly/Cumulative

EROS Consolidated Report on Data DistributedAll Projects with Detail – Monthly/Cumulative

DSB - Functions• Data Ingest

• Data Management

• Data Processing

• Mass Storage

• System Backup

• Data Discovery

• Data Access

• Bulk Download

Data ManagementFilm Archives

. 1939 to Present

. 24 Major Collections

. Multiple film formats/sizes

. Multiple Agencies

Digital Archives

. 1972 to Present

. 1 to 2 Terabytes / Day

. Static and Active Collections

. Expanding to contain film archives Over 8 Petabytes 59,000 rolls of film

14.4 million images

Aerial Photography Collections• U.S. Geological Survey Photography 1939 - 1995

• National High Altitude Photography 1980 - 1989

• National Aerial Photography Program 1987 - 2004

• Space Acquired Photography 1969 - 1984

• Other U.S. Federal Aerial Photography 1937 - 1999

• Side-Looking Airborne Radar 1980 - 1994

• High Resolution Orthorectified 2003 - Present

• National Agriculture Imagery Program 2003 - Present

Cartographic - Topographic Collections• Digital Line Graphs

• 1:24,000* • 1:100,000

• Digital Elevation Models• 7.5o • 15’Alaska • 2 Arc Second** • 1:250,000

• Digital Raster Graphics• 1:24,000• 1:100,000 • 1:250,000

• Digital Orthophoto Quads• National Atlas • National Elevation Database• SRTM

Satellite Collections• Declass 1 (Corona) 1960 – 1972 • Declass 2 (KH-7, KH-9) 1963 – 1980 • SPOT Pan/Multispectral 1986 – 1998• Landsat MSS 1972 – 1992• Landsat Thematic Mapper 1982 – Present• Landsat 7 ETM+ 1999 – Present• Landsat 8 OLI/TIRS 2013 -- Present• EOS/MODIS 2000 – Present• EOS/ASTER 2000 – Present• AVHRR LAC/HRPT 1986 – Present• EO-1 2001 – Present• SIR-C 1994

• Commercial Data Purchases 2002 – Present

Data Holdings

• Collection Diversity• Aerial• Cartographic• Topographic• Satellite

• Digital and film archive media

• Static and growing collections

Robotic Tape Library System

• Utilizes an StorageTek SL8500 tapelibrary systems

• Currently holds approximately 8 PBwith 85 PB capacity

• Media are periodically updated as new technology becomes available

• Implementation of hierarchical store management enables distribution directly from the archive

• Utilizes varying “tiers” of disk (SATA/SAS/Solid State) that are used as a front end cache to the tape

Robotic Tape Library System

Data Archiving• Data is managed and preserved

using 3 copies: nearline, offline, and offsite• Offsite copies are stored

NARA’s Kansas City facility • Multiple media types are utilized to ensure

long term readability• Currently using LTO-6 and T10k-D tapes

• Media are periodically updated as tapes technology advances.

• Records management schedules are worked with NARA

Film Scanning

• Produce on-demand high-resolution scanned products from the historical film archive based on customer requests.• 25, 14 and 7 micron• $30/file - Internet delivery• All Declass scanned images put on-line for future access

• Archive scanning of not-standard film sizes• Chip film - TM, MSS, RBV• Declass, Large Format Camera

Phoenix V• Systematically scan USGS archive to 25 micron• Web enabled at no-charge over the internet• Nine Phoenix V scanners operational (3 more planned)• Designed, built and integrated on-site• 24 X 5 operations• Initial focus on vinegar syndrome (4 yrs)• 14,000 frames scanned per week on average• Currently, over 2.5 million images scanned

Leica Scanner

New Work – Sentinel 2 Activities

European Space Agency (ESA) Sentinel-2 consist of two satellites (one launched in June the other planned for next year), providing global land imagery comprising 13 spectral bands with a swath width of 290km.

Satellite orbit is approximately 800km with coverage repeat cycle of ten days at the equator, and six days at mid-latitudes.

With two satellites coverage cycle is doubled.

Mission lifecycle is designed for 7.5 years with consumables for 12 years. 400

nm600 nm

800 nm

1000 nm

1200 nm

1400 nm

1600 nm

1800 nm

2000 nm

2200 nm

2400 nm

10 m

20 m

60 m

VNIR

SWIR

Visible

VIS NIR SWIR

B1

B2 B3 B4 B8

B5

B6

B7 B8a

B9 B10

B11 B12

VegetationRed-edge

Aerosols Water-vapour Cirrus

Snow / ice / cloud discrimination

Baseline Requirements

1) Products distributed world-wide, without restriction, at no cost to users.

2) Product equivalent to the Landsat L1T product (implies the S2 L1C product).

3) S2 and Landsat products available via coincident searching (implies the same interface).

4) Data ingest of S2 L1C data requires no additional processing.

5) Data archived in the US (driven by the Purpose, above).

6) Data ingest at EROS via network from Europe (assume no downlink at EROS).

7) Imagery acquired covers global land and coastal areas (current Landsat practice).

8) Scalability and multi-mission capability

Sentinel 2 (cont’d)

We plan to distribute the L1C product as processed by ESA that is analogous to Landsat L1T.

L1C product is ortho-rectified 100x100km2 tiles with datasize of 500MB.

NASA Ames will retrieve L1C

data to create Surface

Reflectance (SR) product

to be archived and made

available for distribution.

Sentinel 2 (cont’d)

This estimate is for the data archive and distribution of Sentinel-2 L1C and Surface Reflectance products.

Total data archive 6.4TB per day L1C data archive of 3.2TB per day SR data archive of 3.2TB per day

Total data distribution of 9.6TB per day L1C data retrieval by NASA

Ames of 3.2TB per day L1C data retrieval by Canada

Centre for Remote Sensing (CCRS) of 3.2TB per day

L1C and SR public data distribution of 3.2TB per day

Current Status

• Implementation of Sentinel 2 Archive and Distribution• FY15 Equipment has been received and is on line but we are

still working minor configuration issues

• Initial Bandwidth data transfer test have been done but awaiting access to the international data hub for increased bandwidth

• Received test acquisition and have developed initial ingest script, we know this was preliminary, more ingest scripting is anticipated

• Planning the Characterization effort for implementation of Tier 2 priorities in FY16

17

Sentinel Network Connectivity

EROS

CCRSOttawa

PACSNASA Ames

3.2TB/day GÉANT

GSFC

3.2TB/day

Intern

et 1

Users

Chi

3.2T

B/day

Inte

rnet

2

Alt path

DC

NY

GÉANT

3.2TB/day

Alt path

KC

REED

REED

Mpls

Intern

et 23.2TB/day

Support of NGP and Other Agencies• Support data access and distribution requirements

of the National Geospatial Program‘s National Map• Historically supported the production, archiving and

distribution of LiDAR and elevation information. This has moved to the NGTOC in Rolla.

• Continue to maintain a historic archive of LIDAR data.

• Currently support Orthoimagery and Landcover distribution.

• NGP Orthoimagery ingest will stop in FY16 and the distribution of Orthoimagery will discontinue in the FY18 time frame.

• Landcover distribution is planned to move to the public cloud

Support of NGP and Other Agencies• Provides data, access and distribution services thru

service level agreements to multiple federal agencies including:

• National Geospatial-Intelligence Agency (NGA)

• Federal Emergency Management Agency’s (FEMA) Flood Insurance Rate Map (FIRM) Program

• USDA Forest Service (Fire and Aviation Management)

• USFS/USACE/USGS WRD - GOES

• National Guard Bureau – GIIEP

• NIFC – Greenness Mapping with AVHRR

• BLM – Archival Film Scanning

• NRCS – Color Imagery Scanning

• Liaison Activities• Provide 24x7 on-call support and coordination for

satellite and aerial image access, acquisition and delivery.

• Provide USGS representation and operational support to the International Charter Space and Major Disasters.

• Assist in requirements development, content definition, and usability testing for USGS emergency response systems.

• Emergency Operations Support• Assure appropriate data is ingested and available.

• Operate and maintain the Emergency Operations Portal web pages including establishment of event presence.

Emergency Operations

Fiscal Year 2014 Number of Events: 77

Domestic: 35 International: 42

Fiscal Year 2015 (As of Sept 12) Number of HDDS Events (total): 79

Domestic: 42 – includes more events as several storms and fires were grouped under a single HDDS event. 42 HDDS events = 72 actual events

International: 37

FY2015 events included:

• Active western U.S. fire season (CA, OR, WA, ID, MT, AK)

• Animas River / Gold King Mine Spill (CO)

• Ebola Crisis in West Africa

• Earthquake and Landslides in Nepal

• Sustained image collections for Kilauea, Mount Sinabung, and Cotopaxi Volcanoes – HVO, VDAP

EO Event Support

Image Access and Delivery Federal Government

Bureau of Indian Affairs (BIA) Bureau of Land Management (BLM) Center for Disease Control (CDC) Department of Health and Human Services (HHS) Department of Homeland Security (DHS) Department of Interior (DOI) Environmental Protection Agency (EPA) Federal Emergency Management Agency (FEMA) National Aeronautics and Space Administration

(NASA) National Geospatial Intelligence Agency (NGA) National Guard Bureau (NGB) National Park Service (NPS) National Oceanic and Atmospheric Administration

(NOAA) National Transportation and Safety Board (NTSB)

Federal Stakeholders

National Weather Service (NWS) US Air Force (USAF) US Army US Army Corps of Engineers (USACE) US Navy US Northern Command (NORTHCOM) US Department of Agriculture (USDA) US Forest Service (USFS) USGS (e.g. Earthquake Hazards, Water

Science Centers, Volcano Hazards) State Department US Agency for International Development Office of Foreign Disaster Assistance Department of Veterans Affairs

Data Contributors

International Charter Space and Major Disasters Civil Air Patrol (CAP) Federal Emergency Management Agency (FEMA) National Geospatial-Intelligence Agency (NGA) National Oceanic and Atmospheric Administration (NOAA) National Aeronautics and Space Administration (NASA) State and Local Offices US Army Corps of Engineers (USACE) US Department of Agriculture (USDA) US Geological Survey (USGS) US Department of Homeland Security (DHS) US National Guard (USNG) US Department of Defense Commercial entities

An International agreement among

participating Agencies to provide space-based

data and information in support of relief efforts

during emergencies caused by major

disasters.

Member of the International Charter

The Charter’s capacities can be activated through a single access point which is available 24 hours, seven days a week.

Space agencies contribute

• Priority satellite tasking

• Archive Retrievals

• Organisation of map production

Natural events Man-made eventsEarthquakes Oil spills

Fires Industrial accidents

Floods

Ice jams

Landslides

Tsunamis

Ocean storms

Volcanic eruptions

Disaster Types Supported

The Charter supports large, often sudden events with high impact in terms of lives, infrastructure, and/or environment.

(slow-onset disasters, such as droughts, are not covered by the Charter)

Charter Members

CSACanada

NOAAUSGSUSA

CONAEArgentina

CNESFrance

Europe ESAEUMETSAT

ISROIndia

JAXAJapanCNSA

China

KARIKorea

ROSCOSMOSRussia

INPEBrasil

UKSA/DMCUK

DLRGermany

Land Processes Distributed Active Archive Center (LP DAAC)

Land Processes Distributed Active Archive Center (LP DAAC)• Funded by NASA per interagency MOU• First went operational in 1994• Supports two types of instruments and three missions

• ASTER• MODIS• Other community type data sets (NASA proposals)

• South Dakota State U. – Web Enabled Landsat Data (WELD)• U. of Maryland - Global Forest Cover Change (GFCC)• U. of Arizona – Vegetation Index and Phenology (VIP)• Jet Propulsion Laboratory - NASA Digital Elevation Model

• Uses integrated set of NASA and USGS systems• NASA systems are owned and Configuration Managed by NASA• USGS, USGS contractor, and NASA contractor staff maintain the equipment

• Data ingested and distributed via USGS networks• NASA pushes hard for technology advancement

• 2.5 PB of disk, all data is on spinning disk

• Data will eventually transition to USGS in the 2020 timeframe

• Dataset level information provided to multiple government catalog services• Data.gov

• USGS Science Data Catalog

• Metadata information populated automatically from the Earth Explorer user interface

• Key collections tagged as FGDC National Geospatial Data Assets (NGDA)

Government Catalog Interaction

User Management and Demographics

• User registration required for data downloads• Includes Earth Explorer, HDDS,

GloVis, Landsat Look, and CIDR user interfaces

• Login/Registration enables the creation of detailed metrics and demographic reports• Agency/Affiliation

• Detail on data usage

• Tracking of downloads by dataset and user type

• Login requirements also allow for data rights management• Licensed commercial imagery

• Preliminary data

Accessing Data

• EarthExplorer http://earthexplorer.usgs.gov

• Global Visualization Viewer http://glovis.usgs.gov/

• LandsatLook http://landsatlook.usgs.gov/

• NASA ECHO https://wist.echo.nasa.gov/api/

• Hazards Data Dist. System http://hdds.usgs.gov/hdds

Discussion topics

• Customer Services• Expanding Customer Services (Tier 2 Activities)• Need to increase Level of effort.

• New Products, Product changes (Landsat Sentinel2, LCMAP)• Increase our Web work, Documentation - Training and Outreach

• Data Management Support for Science• Adherence to 4 USGS Instructional Memorandum

• Policy By October 2016• All Research data needs to have data management plans,

appropriate metadata, and be released to the pubic at the time of publication.

• Recommend putting in place the service organization to support this within DSB.

• Establish a Landsat (LSDS) User Working group which is in addition to the Landsat Science Panel.

Discussion topics

• Declassified Data Sets transferred from NGA to NARA.

• LRS Requirement for Periodic Archiving/Production distribution Reports from EROS

• Customer Satisfaction Survey

Questions