Post on 21-Jan-2016
Data Services Branch
U.S. Department of the InteriorU.S. Geological Survey
Douglas BinnieUSGS EROS Center
USGS EROS Center
Mission Area Data Acquisition, Archiving and Access
Ensure that scientists, decision makers, and the public have ready access to land change information and to safeguard and expand the national archive of remotely sensed land data.
• Preserve and distribute the centers data and information products
• Manage the NASA EOS/DAAC requirements
• Operate existing user interfaces and develop future information access methodologies
• Manage hazards and disaster response activities• Operate user services and application assistance support in concert with
the other branch's
• Support the development, implementation, and operation continuum thru direct coordination with science based personnel
Data Services Branch Chief
D. Binnie
Long Term Archive Project
Manager R. Longhenry
Spatial Data Warehouse
ManagerC. Rusanowski
Acting
LS Access and Dist. O&MSentinel 2
K. Kline
LP DAAC
Manager
C. Doescher
LP DAAC Scientist1 - Vacant Term1 - Vacant Perm
(AB Staff)
LP DAACOperations
ManagerC. Torbert
Emergency Operations
Project Manager R. Longhenry
Emergency Ops Liaison
B.Jones
Data Management
Specialist R. Saleh
Computer Scientist
C. Rusanowski
FMO Staff Support
L. DeBoer
Emergency Ops Liaison
Vacant
Administration Support
N. Vandenbussche
Data Services Branch Organization
Customer Engagement
R. Lamb
EROS Consolidated Report on Data ManagedAll Projects with Detail – Monthly/Cumulative
EROS Consolidated Report on Data DistributedAll Projects with Detail – Monthly/Cumulative
DSB - Functions• Data Ingest
• Data Management
• Data Processing
• Mass Storage
• System Backup
• Data Discovery
• Data Access
• Bulk Download
Data ManagementFilm Archives
. 1939 to Present
. 24 Major Collections
. Multiple film formats/sizes
. Multiple Agencies
Digital Archives
. 1972 to Present
. 1 to 2 Terabytes / Day
. Static and Active Collections
. Expanding to contain film archives Over 8 Petabytes 59,000 rolls of film
14.4 million images
Aerial Photography Collections• U.S. Geological Survey Photography 1939 - 1995
• National High Altitude Photography 1980 - 1989
• National Aerial Photography Program 1987 - 2004
• Space Acquired Photography 1969 - 1984
• Other U.S. Federal Aerial Photography 1937 - 1999
• Side-Looking Airborne Radar 1980 - 1994
• High Resolution Orthorectified 2003 - Present
• National Agriculture Imagery Program 2003 - Present
Cartographic - Topographic Collections• Digital Line Graphs
• 1:24,000* • 1:100,000
• Digital Elevation Models• 7.5o • 15’Alaska • 2 Arc Second** • 1:250,000
• Digital Raster Graphics• 1:24,000• 1:100,000 • 1:250,000
• Digital Orthophoto Quads• National Atlas • National Elevation Database• SRTM
Satellite Collections• Declass 1 (Corona) 1960 – 1972 • Declass 2 (KH-7, KH-9) 1963 – 1980 • SPOT Pan/Multispectral 1986 – 1998• Landsat MSS 1972 – 1992• Landsat Thematic Mapper 1982 – Present• Landsat 7 ETM+ 1999 – Present• Landsat 8 OLI/TIRS 2013 -- Present• EOS/MODIS 2000 – Present• EOS/ASTER 2000 – Present• AVHRR LAC/HRPT 1986 – Present• EO-1 2001 – Present• SIR-C 1994
• Commercial Data Purchases 2002 – Present
Data Holdings
• Collection Diversity• Aerial• Cartographic• Topographic• Satellite
• Digital and film archive media
• Static and growing collections
Robotic Tape Library System
• Utilizes an StorageTek SL8500 tapelibrary systems
• Currently holds approximately 8 PBwith 85 PB capacity
• Media are periodically updated as new technology becomes available
• Implementation of hierarchical store management enables distribution directly from the archive
• Utilizes varying “tiers” of disk (SATA/SAS/Solid State) that are used as a front end cache to the tape
Robotic Tape Library System
Data Archiving• Data is managed and preserved
using 3 copies: nearline, offline, and offsite• Offsite copies are stored
NARA’s Kansas City facility • Multiple media types are utilized to ensure
long term readability• Currently using LTO-6 and T10k-D tapes
• Media are periodically updated as tapes technology advances.
• Records management schedules are worked with NARA
Film Scanning
• Produce on-demand high-resolution scanned products from the historical film archive based on customer requests.• 25, 14 and 7 micron• $30/file - Internet delivery• All Declass scanned images put on-line for future access
• Archive scanning of not-standard film sizes• Chip film - TM, MSS, RBV• Declass, Large Format Camera
Phoenix V• Systematically scan USGS archive to 25 micron• Web enabled at no-charge over the internet• Nine Phoenix V scanners operational (3 more planned)• Designed, built and integrated on-site• 24 X 5 operations• Initial focus on vinegar syndrome (4 yrs)• 14,000 frames scanned per week on average• Currently, over 2.5 million images scanned
Leica Scanner
New Work – Sentinel 2 Activities
European Space Agency (ESA) Sentinel-2 consist of two satellites (one launched in June the other planned for next year), providing global land imagery comprising 13 spectral bands with a swath width of 290km.
Satellite orbit is approximately 800km with coverage repeat cycle of ten days at the equator, and six days at mid-latitudes.
With two satellites coverage cycle is doubled.
Mission lifecycle is designed for 7.5 years with consumables for 12 years. 400
nm600 nm
800 nm
1000 nm
1200 nm
1400 nm
1600 nm
1800 nm
2000 nm
2200 nm
2400 nm
10 m
20 m
60 m
VNIR
SWIR
Visible
VIS NIR SWIR
B1
B2 B3 B4 B8
B5
B6
B7 B8a
B9 B10
B11 B12
VegetationRed-edge
Aerosols Water-vapour Cirrus
Snow / ice / cloud discrimination
Baseline Requirements
1) Products distributed world-wide, without restriction, at no cost to users.
2) Product equivalent to the Landsat L1T product (implies the S2 L1C product).
3) S2 and Landsat products available via coincident searching (implies the same interface).
4) Data ingest of S2 L1C data requires no additional processing.
5) Data archived in the US (driven by the Purpose, above).
6) Data ingest at EROS via network from Europe (assume no downlink at EROS).
7) Imagery acquired covers global land and coastal areas (current Landsat practice).
8) Scalability and multi-mission capability
Sentinel 2 (cont’d)
We plan to distribute the L1C product as processed by ESA that is analogous to Landsat L1T.
L1C product is ortho-rectified 100x100km2 tiles with datasize of 500MB.
NASA Ames will retrieve L1C
data to create Surface
Reflectance (SR) product
to be archived and made
available for distribution.
Sentinel 2 (cont’d)
This estimate is for the data archive and distribution of Sentinel-2 L1C and Surface Reflectance products.
Total data archive 6.4TB per day L1C data archive of 3.2TB per day SR data archive of 3.2TB per day
Total data distribution of 9.6TB per day L1C data retrieval by NASA
Ames of 3.2TB per day L1C data retrieval by Canada
Centre for Remote Sensing (CCRS) of 3.2TB per day
L1C and SR public data distribution of 3.2TB per day
Current Status
• Implementation of Sentinel 2 Archive and Distribution• FY15 Equipment has been received and is on line but we are
still working minor configuration issues
• Initial Bandwidth data transfer test have been done but awaiting access to the international data hub for increased bandwidth
• Received test acquisition and have developed initial ingest script, we know this was preliminary, more ingest scripting is anticipated
• Planning the Characterization effort for implementation of Tier 2 priorities in FY16
17
Sentinel Network Connectivity
EROS
CCRSOttawa
PACSNASA Ames
3.2TB/day GÉANT
GSFC
3.2TB/day
Intern
et 1
Users
Chi
3.2T
B/day
Inte
rnet
2
Alt path
DC
NY
GÉANT
3.2TB/day
Alt path
KC
REED
REED
Mpls
Intern
et 23.2TB/day
Support of NGP and Other Agencies• Support data access and distribution requirements
of the National Geospatial Program‘s National Map• Historically supported the production, archiving and
distribution of LiDAR and elevation information. This has moved to the NGTOC in Rolla.
• Continue to maintain a historic archive of LIDAR data.
• Currently support Orthoimagery and Landcover distribution.
• NGP Orthoimagery ingest will stop in FY16 and the distribution of Orthoimagery will discontinue in the FY18 time frame.
• Landcover distribution is planned to move to the public cloud
Support of NGP and Other Agencies• Provides data, access and distribution services thru
service level agreements to multiple federal agencies including:
• National Geospatial-Intelligence Agency (NGA)
• Federal Emergency Management Agency’s (FEMA) Flood Insurance Rate Map (FIRM) Program
• USDA Forest Service (Fire and Aviation Management)
• USFS/USACE/USGS WRD - GOES
• National Guard Bureau – GIIEP
• NIFC – Greenness Mapping with AVHRR
• BLM – Archival Film Scanning
• NRCS – Color Imagery Scanning
• Liaison Activities• Provide 24x7 on-call support and coordination for
satellite and aerial image access, acquisition and delivery.
• Provide USGS representation and operational support to the International Charter Space and Major Disasters.
• Assist in requirements development, content definition, and usability testing for USGS emergency response systems.
• Emergency Operations Support• Assure appropriate data is ingested and available.
• Operate and maintain the Emergency Operations Portal web pages including establishment of event presence.
Emergency Operations
Fiscal Year 2014 Number of Events: 77
Domestic: 35 International: 42
Fiscal Year 2015 (As of Sept 12) Number of HDDS Events (total): 79
Domestic: 42 – includes more events as several storms and fires were grouped under a single HDDS event. 42 HDDS events = 72 actual events
International: 37
FY2015 events included:
• Active western U.S. fire season (CA, OR, WA, ID, MT, AK)
• Animas River / Gold King Mine Spill (CO)
• Ebola Crisis in West Africa
• Earthquake and Landslides in Nepal
• Sustained image collections for Kilauea, Mount Sinabung, and Cotopaxi Volcanoes – HVO, VDAP
EO Event Support
Image Access and Delivery Federal Government
Bureau of Indian Affairs (BIA) Bureau of Land Management (BLM) Center for Disease Control (CDC) Department of Health and Human Services (HHS) Department of Homeland Security (DHS) Department of Interior (DOI) Environmental Protection Agency (EPA) Federal Emergency Management Agency (FEMA) National Aeronautics and Space Administration
(NASA) National Geospatial Intelligence Agency (NGA) National Guard Bureau (NGB) National Park Service (NPS) National Oceanic and Atmospheric Administration
(NOAA) National Transportation and Safety Board (NTSB)
Federal Stakeholders
National Weather Service (NWS) US Air Force (USAF) US Army US Army Corps of Engineers (USACE) US Navy US Northern Command (NORTHCOM) US Department of Agriculture (USDA) US Forest Service (USFS) USGS (e.g. Earthquake Hazards, Water
Science Centers, Volcano Hazards) State Department US Agency for International Development Office of Foreign Disaster Assistance Department of Veterans Affairs
Data Contributors
International Charter Space and Major Disasters Civil Air Patrol (CAP) Federal Emergency Management Agency (FEMA) National Geospatial-Intelligence Agency (NGA) National Oceanic and Atmospheric Administration (NOAA) National Aeronautics and Space Administration (NASA) State and Local Offices US Army Corps of Engineers (USACE) US Department of Agriculture (USDA) US Geological Survey (USGS) US Department of Homeland Security (DHS) US National Guard (USNG) US Department of Defense Commercial entities
An International agreement among
participating Agencies to provide space-based
data and information in support of relief efforts
during emergencies caused by major
disasters.
Member of the International Charter
The Charter’s capacities can be activated through a single access point which is available 24 hours, seven days a week.
Space agencies contribute
• Priority satellite tasking
• Archive Retrievals
• Organisation of map production
Natural events Man-made eventsEarthquakes Oil spills
Fires Industrial accidents
Floods
Ice jams
Landslides
Tsunamis
Ocean storms
Volcanic eruptions
Disaster Types Supported
The Charter supports large, often sudden events with high impact in terms of lives, infrastructure, and/or environment.
(slow-onset disasters, such as droughts, are not covered by the Charter)
Charter Members
CSACanada
NOAAUSGSUSA
CONAEArgentina
CNESFrance
Europe ESAEUMETSAT
ISROIndia
JAXAJapanCNSA
China
KARIKorea
ROSCOSMOSRussia
INPEBrasil
UKSA/DMCUK
DLRGermany
Land Processes Distributed Active Archive Center (LP DAAC)
Land Processes Distributed Active Archive Center (LP DAAC)• Funded by NASA per interagency MOU• First went operational in 1994• Supports two types of instruments and three missions
• ASTER• MODIS• Other community type data sets (NASA proposals)
• South Dakota State U. – Web Enabled Landsat Data (WELD)• U. of Maryland - Global Forest Cover Change (GFCC)• U. of Arizona – Vegetation Index and Phenology (VIP)• Jet Propulsion Laboratory - NASA Digital Elevation Model
• Uses integrated set of NASA and USGS systems• NASA systems are owned and Configuration Managed by NASA• USGS, USGS contractor, and NASA contractor staff maintain the equipment
• Data ingested and distributed via USGS networks• NASA pushes hard for technology advancement
• 2.5 PB of disk, all data is on spinning disk
• Data will eventually transition to USGS in the 2020 timeframe
• Dataset level information provided to multiple government catalog services• Data.gov
• USGS Science Data Catalog
• Metadata information populated automatically from the Earth Explorer user interface
• Key collections tagged as FGDC National Geospatial Data Assets (NGDA)
Government Catalog Interaction
User Management and Demographics
• User registration required for data downloads• Includes Earth Explorer, HDDS,
GloVis, Landsat Look, and CIDR user interfaces
• Login/Registration enables the creation of detailed metrics and demographic reports• Agency/Affiliation
• Detail on data usage
• Tracking of downloads by dataset and user type
• Login requirements also allow for data rights management• Licensed commercial imagery
• Preliminary data
Accessing Data
• EarthExplorer http://earthexplorer.usgs.gov
• Global Visualization Viewer http://glovis.usgs.gov/
• LandsatLook http://landsatlook.usgs.gov/
• NASA ECHO https://wist.echo.nasa.gov/api/
• Hazards Data Dist. System http://hdds.usgs.gov/hdds
Discussion topics
• Customer Services• Expanding Customer Services (Tier 2 Activities)• Need to increase Level of effort.
• New Products, Product changes (Landsat Sentinel2, LCMAP)• Increase our Web work, Documentation - Training and Outreach
• Data Management Support for Science• Adherence to 4 USGS Instructional Memorandum
• Policy By October 2016• All Research data needs to have data management plans,
appropriate metadata, and be released to the pubic at the time of publication.
• Recommend putting in place the service organization to support this within DSB.
• Establish a Landsat (LSDS) User Working group which is in addition to the Landsat Science Panel.
Discussion topics
• Declassified Data Sets transferred from NGA to NARA.
• LRS Requirement for Periodic Archiving/Production distribution Reports from EROS
• Customer Satisfaction Survey
Questions