LSDS-1388 Version 2.0

Department of the Interior U.S. Geological Survey Landsat Cloud Optimized GeoTIFF (COG) Data Format Control Book (DFCB) Version 2.0 January 2020

- ii - LSDS-1388 Version 2.0

Landsat Cloud Optimized GeoTIFF (COG) Data Format Control Book (DFCB)

January 2020

Approved By: ______________________________ Chris Engebretson Date Landsat DPAS CCB Chair U.S. Geological Survey

EROS Sioux Falls, South Dakota

- iii - LSDS-1388 Version 2.0

Executive Summary

This Data Format Control Book (DFCB) presents a description of the Cloud Optimized Georeferenced Tagged Image File Format (GeoTIFF) (COG), an efficient way to store GeoTIFF files in the cloud. This document is under Data Processing and Archive System (DPAS) Configuration Control Board (CCB) control. Please submit changes to this document, as well as supportive material justifying the proposed changes, via Change Request (CR) to the Process and Change Management Tool.

- iv - LSDS-1388 Version 2.0

Document History

Document Number

Document Version

Publication Date

Change Number

LSDS-1388 Version 1.0 August 2019 CR 15020

LSDS-1388 Version 2.0 January 2020 CR 15235

- v - LSDS-1388 Version 2.0

Contents

Executive Summary ..................................................................................................... iii Document History ........................................................................................................ iv Contents ......................................................................................................................... v List of Figures ............................................................................................................... v List of Tables ................................................................................................................. v Section 1 Introduction .............................................................................................. 1

1.1 Purpose and Scope .......................................................................................... 1 1.2 Terminology ...................................................................................................... 1

Section 2 COG Data Format Definition ................................................................... 2 2.1 COG Format Overview ..................................................................................... 2 2.2 COG file structure ............................................................................................. 2

2.2.1 File Headers and Tags............................................................................... 3 2.2.2 Internal Overviews ..................................................................................... 3 2.2.3 Internal Tiling ............................................................................................. 3

2.3 COG Creation Specification .............................................................................. 4 Section 3 Product Distribution Data Format Definition ......................................... 5

3.1 Product Distribution Packaging ......................................................................... 5 3.2 Cloud Native Product Metadata ........................................................................ 5

3.2.1 JSON Metadata conversion ....................................................................... 5 3.2.2 STAC records .......................................................................................... 10

Appendix A Landsat DFCBs .................................................................................. 16 A.1 Collection 1 DFCBs ......................................................................................... 16 A.2 Collection 2 DFCBs ......................................................................................... 17

Appendix B Acronyms ............................................................................................ 18 References ................................................................................................................... 20 List of Figures

Figure 3-1. Example MTL Metadata File in JSON Format............................................. 10 Figure 3-2. Example STAC Item Record ....................................................................... 15 List of Tables

Table A-1. Landsat Collection 1 Data Format Control Books ........................................ 16 Table A-2. Landsat Collection 2 Data Format Control Books ........................................ 17

- 1 - LSDS-1388 Version 2.0

Section 1 Introduction

1.1 Purpose and Scope This Data Format Control Book (DFCB) provides a description of the Cloud Optimized GeoTIFF (COG) specifications. This DFCB describes the formats and data contents of cloud-optimized Landsat data, including the implementation of the SpatioTemporal Asset Catalog (STAC) specification.

1.2 Terminology TIFF – Tagged Image File Format (TIFF) is a computer file format for storing raster data. GeoTIFF – Georeferenced Tagged Image File Format (GeoTIFF) is a TIFF file containing additional georeferencing information such as map projection, coordinate systems, etc. More information on GeoTIFF can be found at opengeospatial.org/standards/geotiff (Open Geospatial Consortium 2019). COG – Cloud Optimized GeoTIFF (COG) is a regular GeoTIFF file that has been optimized for hosting in a cloud-based environment so that users can use Hypertext Transfer Protocol (HTTP) GET requests to extract only the requested portion of the data instead of downloading the entire file. More information on the COG format can be found at cogeo.org/in-depth.html (Holmes, Cloud Optimized GeoTIFF in depth 2017). JSON – JavaScript Object Notation (JSON) is a minimal, human and machine-readable format for structuring data. It consists of a collection of name/value pairs and ordered lists of values. STAC – “The SpatioTemporal Asset Catalog (STAC) specification provides a common language to describe a range of geospatial information, so it can more easily be indexed and discovered. A 'spatiotemporal asset' is any file that represents information about the Earth captured in a certain space and time.” (Gavin, Holmes and Broderic 2019) STAC is a community-driven effort to make spatiotemporal data accessible and discoverable by anyone, in a flexible and extensible way that will remain relevant even as technologies and tools evolve. More information on STAC can be found at https://stacspec.org/ (Gavin, Holmes and Broderic 2019). GeoJSON – GeoJSON, written in JSON, is an open standard format that represents simple geographical features such as points, line strings, polygons, and multi-part collections (Internet Engineering Task Force 2016).

https://www.opengeospatial.org/standards/geotiffhttps://www.cogeo.org/in-depth.htmlhttps://stacspec.org/

- 2 - LSDS-1388 Version 2.0

Section 2 COG Data Format Definition

This section details the COG data format, both in general and the specific version used for Landsat data in the cloud.

2.1 COG Format Overview The COG file format is an extension of the GeoTIFF file format which enables more flexible access to Geospatial data. It allows users to perform HTTP range GET requests to extract specific portions of the file’s data. COGs can provide the capability to view the file at a downsampled resolution with less data transfer and loading required, through internal file overviews. See Section 2.2.2 and Section Error! Reference source not found. for more information on COG overviews. COGs can also provide internal tiling over the data they contain, which makes the HTTP range GET requests faster. See Section 2.2.3 and Section 2.3 for more information on internal tiling. More information on the COG format can be found at cogeo.org/in-depth.html (Holmes, Cloud Optimized GeoTIFF in depth 2017). For the specifications of the GeoTIFF files used for Landsat data in Collection 1 (C1), refer to Section A.1.

2.2 COG file structure A COG product will contain all the bands and metadata indicated in the corresponding sensor-specific Data Format Control Book. Example: LC08_L2SP_040033_20130709_20191220_02_T1_ANG.txt LC08_L2SP_040033_20130709_20191220_02_T1_MD5.txt LC08_L2SP_040033_20130709_20191220_02_T1_MTL.txt LC08_L2SP_040033_20130709_20191220_02_T1_MTL.xml LC08_L2SP_040033_20130709_20191220_02_T1_QA_PIXEL.tif LC08_L2SP_040033_20130709_20191220_02_T1_QA_RADSAT.tif LC08_L2SP_040033_20130709_20191220_02_T1_SR_B1.tif LC08_L2SP_040033_20130709_20191220_02_T1_SR_B2.tif LC08_L2SP_040033_20130709_20191220_02_T1_SR_B3.tif LC08_L2SP_040033_20130709_20191220_02_T1_SR_B4.tif LC08_L2SP_040033_20130709_20191220_02_T1_SR_B5.tif LC08_L2SP_040033_20130709_20191220_02_T1_SR_B6.tif LC08_L2SP_040033_20130709_20191220_02_T1_SR_B7.tif LC08_L2SP_040033_20130709_20191220_02_T1_SR_QA_AEROSOL.tif LC08_L2SP_040033_20130709_20191220_02_T1_ST_ATRAN.tif LC08_L2SP_040033_20130709_20191220_02_T1_ST_B10.tif LC08_L2SP_040033_20130709_20191220_02_T1_ST_CDIST.tif LC08_L2SP_040033_20130709_20191220_02_T1_ST_DRAD.tif LC08_L2SP_040033_20130709_20191220_02_T1_ST_EMIS.tif

https://www.cogeo.org/in-depth.html

- 3 - LSDS-1388 Version 2.0

LC08_L2SP_040033_20130709_20191220_02_T1_ST_EMSD.tif LC08_L2SP_040033_20130709_20191220_02_T1_ST_QA.tif LC08_L2SP_040033_20130709_20191220_02_T1_ST_TRAD.tif LC08_L2SP_040033_20130709_20191220_02_T1_ST_URAD.tif In addition, the following files are generated in the cloud: LC08_L2SP_040033_20130709_20191220_02_T1_MTL.json LC08_L2SP_040033_20130709_20191220_02_T1_SR_stac.json LC08_L2SP_040033_20130709_20191220_02_T1_ST_stac.json LC08_L2SP_040033_20130709_20191220_02_T1_thumb_large.jpeg LC08_L2SP_040033_20130709_20191220_02_T1_thumb_small.jpeg The .jpeg files are thumbnails made from a 3-band true color mosaic of the scene. The .json files are explained in detail in Section 3.2

2.2.1 File Headers and Tags GeoTIFF defines a set of Tagged Image File Format (TIFF) tags, which describe cartographic and geodetic information associated with geographic TIFF imagery. COG Tags are inherited from the GeoTIFF format. GeoTIFF tags convey information about the image. The tags describe the image using information the GeoTIFF reader needs to control the appearance of the image on the user’s screen. The TIFF tags are embedded in the same file as the TIFF image. The GeoTIFF tags provide information on the image projection and corner points, which define the geographic location and extent of the image. A complete description of the raster data requires geo-referencing of the data, which is accomplished using tags. Landsat Product Generation System (LPGS) uses the transformation raster, model space tiepoints, and scaling parameters for the GeoTIFFs it generates. ModelTiepointTag and ModelPixelScaleTag are used for this purpose. Refer to the documents in Appendix A for detailed information on tags and keys.

2.2.2 Internal Overviews Internal overviews are pyramidal overlays on the TIFF files. Each overview divides the image into four parts, so a smaller amount of data can be read. In other words, internal overviews are zoomed-out versions of the original image stored in the same COG file. It is possible for a COG file to have several internal overviews at different resolutions, which make serving the data much faster, since applications that support zooming need only to retrieve the internal overview associated with the desired zoom level.

2.2.3 Internal Tiling COGs can store image pixels in internal tiles rather than linearly, which makes accessing data from a specific part of the image faster. These internal tiles enable COG data to be accessed using HTTP range requests, which were introduced in HTTP

- 4 - LSDS-1388 Version 2.0

Version 1.1. More information on HTTP range requests can be found at https://tools.ietf.org/html/rfc7233 (Internet Engineering Task Force 2014).

2.3 COG Creation Specification For Landsat C1 data, the GeoTIFF band files produced by LPGS are transformed into COGs in the cloud before distribution. The underlying data pixels of each band are not changed in any way, they remain identical to those found in the original GeoTIFF files. Starting in Landsat Collection 2 (C2), LPGS will output bands as COGs. The DFCBs for Landsat C1 and C2 can be found in Appendix A. For GeoTIFFs converted to COGs in the cloud, the following configuration options are used:

• Pixel interleaving • Internal tiling with block size 256x256 pixels • Internal overviews with block size 128x128 pixels and downsampling levels of 2,

4, 8, 16, 32, and 64 • Compression with the “deflate” algorithm

The downsampling method used to create the internal overviews depends on the type of band:

• For Quality Assessment (QA) bands, the “nearest-neighbors” algorithm is used. • For Landsat 7 (L7) external per-band gap masks, no internal overviews are

generated. • For all other bands, the “average” algorithm is used.

https://tools.ietf.org/html/rfc7233

- 5 - LSDS-1388 Version 2.0

Section 3 Product Distribution Data Format Definition

This section describes the storage and distribution format for Landsat data in the cloud.

3.1 Product Distribution Packaging Landsat products in the cloud will be stored and distributed in an unbundled format. They will not be in tar.gz packaged format. All files that make up one scene will be collocated in cloud storage, but each will be individually accessible.

3.2 Cloud Native Product Metadata Landsat products stored in the cloud maintain any metadata they originally had, as well as additional metadata to ease cloud distribution and processing.

3.2.1 JSON Metadata conversion The original Landsat metadata files, such as the MTL.txt file or the MTL.xml files, are translated into JSON format, which is easier to distribute in the cloud. An example of a JSON-converted MTL file is given in Figure 3-1.

- 6 - LSDS-1388 Version 2.0

{ "L1_METADATA_FILE": { "METADATA_FILE_INFO": { "ORIGIN": "Image courtesy of the U.S. Geological Survey", "REQUEST_ID": "0501612195382_08257", "LANDSAT_SCENE_ID": "LE70010052009170EDC00", "LANDSAT_PRODUCT_ID": "LE07_L1TP_001005_20090619_20161219_01_T1", "COLLECTION_NUMBER": "01", "FILE_DATE": "2016-12-19T20:32:40Z", "STATION_ID": "EDC", "PROCESSING_SOFTWARE_VERSION": "LPGS_12.8.2", "DATA_CATEGORY": "NOMINAL" }, "PRODUCT_METADATA": { "DATA_TYPE": "L1TP", "COLLECTION_CATEGORY": "T1", "ELEVATION_SOURCE": "GLS2000", "OUTPUT_FORMAT": "GEOTIFF", "EPHEMERIS_TYPE": "DEFINITIVE", "SPACECRAFT_ID": "LANDSAT_7", "SENSOR_ID": "ETM", "SENSOR_MODE": "BUMPER", "WRS_PATH": "001", "WRS_ROW": "005", "DATE_ACQUIRED": "2009-06-19", "SCENE_CENTER_TIME": "13:59:11.9408909Z", "CORNER_UL_LAT_PRODUCT": "78.20555", "CORNER_UL_LON_PRODUCT": "-26.59144", "CORNER_UR_LAT_PRODUCT": "78.18624", "CORNER_UR_LON_PRODUCT": "-14.49844", "CORNER_LL_LAT_PRODUCT": "75.79561", "CORNER_LL_LON_PRODUCT": "-25.65565", "CORNER_LR_LAT_PRODUCT": "75.77967", "CORNER_LR_LON_PRODUCT": "-15.58469", "CORNER_UL_PROJECTION_X_PRODUCT": "372600.000", "CORNER_UL_PROJECTION_Y_PRODUCT": "8687400.000", "CORNER_UR_PROJECTION_X_PRODUCT": "648300.000", "CORNER_UR_PROJECTION_Y_PRODUCT": "8687400.000", "CORNER_LL_PROJECTION_X_PRODUCT": "372600.000", "CORNER_LL_PROJECTION_Y_PRODUCT": "8417400.000", "CORNER_LR_PROJECTION_X_PRODUCT": "648300.000", "CORNER_LR_PROJECTION_Y_PRODUCT": "8417400.000", "PANCHROMATIC_LINES": "18001", "PANCHROMATIC_SAMPLES": "18381", "REFLECTIVE_LINES": "9001", "REFLECTIVE_SAMPLES": "9191", "THERMAL_LINES": "9001", "THERMAL_SAMPLES": "9191", "FILE_NAME_BAND_1": "LE07_L1TP_001005_20090619_20161219_01_T1_B1.TIF", "FILE_NAME_BAND_2": "LE07_L1TP_001005_20090619_20161219_01_T1_B2.TIF", "FILE_NAME_BAND_3": "LE07_L1TP_001005_20090619_20161219_01_T1_B3.TIF",

- 7 - LSDS-1388 Version 2.0

"FILE_NAME_BAND_4": "LE07_L1TP_001005_20090619_20161219_01_T1_B4.TIF", "FILE_NAME_BAND_5": "LE07_L1TP_001005_20090619_20161219_01_T1_B5.TIF", "FILE_NAME_BAND_6_VCID_1": "LE07_L1TP_001005_20090619_20161219_01_T1_B6_VCID_1.TIF", "FILE_NAME_BAND_6_VCID_2": "LE07_L1TP_001005_20090619_20161219_01_T1_B6_VCID_2.TIF", "FILE_NAME_BAND_7": "LE07_L1TP_001005_20090619_20161219_01_T1_B7.TIF", "FILE_NAME_BAND_8": "LE07_L1TP_001005_20090619_20161219_01_T1_B8.TIF", "FILE_NAME_BAND_QUALITY": "LE07_L1TP_001005_20090619_20161219_01_T1_BQA.TIF", "GROUND_CONTROL_POINT_FILE_NAME": "LE07_L1TP_001005_20090619_20161219_01_T1_GCP.txt", "ANGLE_COEFFICIENT_FILE_NAME": "LE07_L1TP_001005_20090619_20161219_01_T1_ANG.txt", "METADATA_FILE_NAME": "LE07_L1TP_001005_20090619_20161219_01_T1_MTL.txt", "CPF_NAME": "LE07CPF_20090401_20090630_01.02" }, "IMAGE_ATTRIBUTES": { "CLOUD_COVER": "4.00", "CLOUD_COVER_LAND": "5.00", "IMAGE_QUALITY": "9", "SUN_AZIMUTH": "-169.92167832", "SUN_ELEVATION": "36.21102154", "EARTH_SUN_DISTANCE": "1.0161383", "SATURATION_BAND_1": "Y", "SATURATION_BAND_2": "Y", "SATURATION_BAND_3": "Y", "SATURATION_BAND_4": "Y", "SATURATION_BAND_5": "Y", "SATURATION_BAND_6_VCID_1": "N", "SATURATION_BAND_6_VCID_2": "N", "SATURATION_BAND_7": "N", "SATURATION_BAND_8": "N", "GROUND_CONTROL_POINTS_VERSION": "4", "GROUND_CONTROL_POINTS_MODEL": "75", "GEOMETRIC_RMSE_MODEL": "8.051", "GEOMETRIC_RMSE_MODEL_Y": "6.520", "GEOMETRIC_RMSE_MODEL_X": "4.723" }, "MIN_MAX_RADIANCE": { "RADIANCE_MAXIMUM_BAND_1": "293.700", "RADIANCE_MINIMUM_BAND_1": "-6.200", "RADIANCE_MAXIMUM_BAND_2": "300.900", "RADIANCE_MINIMUM_BAND_2": "-6.400", "RADIANCE_MAXIMUM_BAND_3": "234.400", "RADIANCE_MINIMUM_BAND_3": "-5.000", "RADIANCE_MAXIMUM_BAND_4": "241.100", "RADIANCE_MINIMUM_BAND_4": "-5.100", "RADIANCE_MAXIMUM_BAND_5": "31.060", "RADIANCE_MINIMUM_BAND_5": "-1.000", "RADIANCE_MAXIMUM_BAND_6_VCID_1": "17.040",

- 8 - LSDS-1388 Version 2.0

"RADIANCE_MINIMUM_BAND_6_VCID_1": "0.000", "RADIANCE_MAXIMUM_BAND_6_VCID_2": "12.650", "RADIANCE_MINIMUM_BAND_6_VCID_2": "3.200", "RADIANCE_MAXIMUM_BAND_7": "10.800", "RADIANCE_MINIMUM_BAND_7": "-0.350", "RADIANCE_MAXIMUM_BAND_8": "243.100", "RADIANCE_MINIMUM_BAND_8": "-4.700" }, "MIN_MAX_REFLECTANCE": { "REFLECTANCE_MAXIMUM_BAND_1": "0.467931", "REFLECTANCE_MINIMUM_BAND_1": "-0.009878", "REFLECTANCE_MAXIMUM_BAND_2": "0.525896", "REFLECTANCE_MINIMUM_BAND_2": "-0.011186", "REFLECTANCE_MAXIMUM_BAND_3": "0.498590", "REFLECTANCE_MINIMUM_BAND_3": "-0.010635", "REFLECTANCE_MAXIMUM_BAND_4": "0.730236", "REFLECTANCE_MINIMUM_BAND_4": "-0.015447", "REFLECTANCE_MAXIMUM_BAND_5": "0.454660", "REFLECTANCE_MINIMUM_BAND_5": "-0.014638", "REFLECTANCE_MAXIMUM_BAND_7": "0.430594", "REFLECTANCE_MINIMUM_BAND_7": "-0.013954", "REFLECTANCE_MAXIMUM_BAND_8": "0.597855", "REFLECTANCE_MINIMUM_BAND_8": "-0.011559" }, "MIN_MAX_PIXEL_VALUE": { "QUANTIZE_CAL_MAX_BAND_1": "255", "QUANTIZE_CAL_MIN_BAND_1": "1", "QUANTIZE_CAL_MAX_BAND_2": "255", "QUANTIZE_CAL_MIN_BAND_2": "1", "QUANTIZE_CAL_MAX_BAND_3": "255", "QUANTIZE_CAL_MIN_BAND_3": "1", "QUANTIZE_CAL_MAX_BAND_4": "255", "QUANTIZE_CAL_MIN_BAND_4": "1", "QUANTIZE_CAL_MAX_BAND_5": "255", "QUANTIZE_CAL_MIN_BAND_5": "1", "QUANTIZE_CAL_MAX_BAND_6_VCID_1": "255", "QUANTIZE_CAL_MIN_BAND_6_VCID_1": "1", "QUANTIZE_CAL_MAX_BAND_6_VCID_2": "255", "QUANTIZE_CAL_MIN_BAND_6_VCID_2": "1", "QUANTIZE_CAL_MAX_BAND_7": "255", "QUANTIZE_CAL_MIN_BAND_7": "1", "QUANTIZE_CAL_MAX_BAND_8": "255", "QUANTIZE_CAL_MIN_BAND_8": "1" }, "PRODUCT_PARAMETERS": { "CORRECTION_GAIN_BAND_1": "CPF", "CORRECTION_GAIN_BAND_2": "CPF", "CORRECTION_GAIN_BAND_3": "CPF", "CORRECTION_GAIN_BAND_4": "CPF", "CORRECTION_GAIN_BAND_5": "CPF", "CORRECTION_GAIN_BAND_6_VCID_1": "CPF", "CORRECTION_GAIN_BAND_6_VCID_2": "CPF", "CORRECTION_GAIN_BAND_7": "CPF", "CORRECTION_GAIN_BAND_8": "CPF", "CORRECTION_BIAS_BAND_1": "INTERNAL_CALIBRATION", "CORRECTION_BIAS_BAND_2": "INTERNAL_CALIBRATION",

- 9 - LSDS-1388 Version 2.0

"CORRECTION_BIAS_BAND_3": "INTERNAL_CALIBRATION", "CORRECTION_BIAS_BAND_4": "INTERNAL_CALIBRATION", "CORRECTION_BIAS_BAND_5": "INTERNAL_CALIBRATION", "CORRECTION_BIAS_BAND_6_VCID_1": "INTERNAL_CALIBRATION", "CORRECTION_BIAS_BAND_6_VCID_2": "INTERNAL_CALIBRATION", "CORRECTION_BIAS_BAND_7": "INTERNAL_CALIBRATION", "CORRECTION_BIAS_BAND_8": "INTERNAL_CALIBRATION", "GAIN_BAND_1": "L", "GAIN_BAND_2": "L", "GAIN_BAND_3": "L", "GAIN_BAND_4": "L", "GAIN_BAND_5": "H", "GAIN_BAND_6_VCID_1": "L", "GAIN_BAND_6_VCID_2": "H", "GAIN_BAND_7": "H", "GAIN_BAND_8": "L", "GAIN_CHANGE_BAND_1": "LL", "GAIN_CHANGE_BAND_2": "LL", "GAIN_CHANGE_BAND_3": "LL", "GAIN_CHANGE_BAND_4": "LL", "GAIN_CHANGE_BAND_5": "HH", "GAIN_CHANGE_BAND_6_VCID_1": "LL", "GAIN_CHANGE_BAND_6_VCID_2": "HH", "GAIN_CHANGE_BAND_7": "HH", "GAIN_CHANGE_BAND_8": "LL", "GAIN_CHANGE_SCAN_BAND_1": "0", "GAIN_CHANGE_SCAN_BAND_2": "0", "GAIN_CHANGE_SCAN_BAND_3": "0", "GAIN_CHANGE_SCAN_BAND_4": "0", "GAIN_CHANGE_SCAN_BAND_5": "0", "GAIN_CHANGE_SCAN_BAND_6_VCID_1": "0", "GAIN_CHANGE_SCAN_BAND_6_VCID_2": "0", "GAIN_CHANGE_SCAN_BAND_7": "0", "GAIN_CHANGE_SCAN_BAND_8": "0" }, "RADIOMETRIC_RESCALING": { "RADIANCE_MULT_BAND_1": "1.1807E+00", "RADIANCE_MULT_BAND_2": "1.2098E+00", "RADIANCE_MULT_BAND_3": "9.4252E-01", "RADIANCE_MULT_BAND_4": "9.6929E-01", "RADIANCE_MULT_BAND_5": "1.2622E-01", "RADIANCE_MULT_BAND_6_VCID_1": "6.7087E-02", "RADIANCE_MULT_BAND_6_VCID_2": "3.7205E-02", "RADIANCE_MULT_BAND_7": "4.3898E-02", "RADIANCE_MULT_BAND_8": "9.7559E-01", "RADIANCE_ADD_BAND_1": "-7.38071", "RADIANCE_ADD_BAND_2": "-7.60984", "RADIANCE_ADD_BAND_3": "-5.94252", "RADIANCE_ADD_BAND_4": "-6.06929", "RADIANCE_ADD_BAND_5": "-1.12622", "RADIANCE_ADD_BAND_6_VCID_1": "-0.06709", "RADIANCE_ADD_BAND_6_VCID_2": "3.16280", "RADIANCE_ADD_BAND_7": "-0.39390", "RADIANCE_ADD_BAND_8": "-5.67559", "REFLECTANCE_MULT_BAND_1": "1.8811E-03", "REFLECTANCE_MULT_BAND_2": "2.1145E-03",

- 10 - LSDS-1388 Version 2.0

"REFLECTANCE_MULT_BAND_3": "2.0048E-03", "REFLECTANCE_MULT_BAND_4": "2.9358E-03", "REFLECTANCE_MULT_BAND_5": "1.8476E-03", "REFLECTANCE_MULT_BAND_7": "1.7502E-03", "REFLECTANCE_MULT_BAND_8": "2.3993E-03", "REFLECTANCE_ADD_BAND_1": "-0.011759", "REFLECTANCE_ADD_BAND_2": "-0.013300", "REFLECTANCE_ADD_BAND_3": "-0.012640", "REFLECTANCE_ADD_BAND_4": "-0.018382", "REFLECTANCE_ADD_BAND_5": "-0.016486", "REFLECTANCE_ADD_BAND_7": "-0.015705", "REFLECTANCE_ADD_BAND_8": "-0.013958" }, "THERMAL_CONSTANTS": { "K1_CONSTANT_BAND_6_VCID_1": "666.09", "K2_CONSTANT_BAND_6_VCID_1": "1282.71", "K1_CONSTANT_BAND_6_VCID_2": "666.09", "K2_CONSTANT_BAND_6_VCID_2": "1282.71" }, "PROJECTION_PARAMETERS": { "MAP_PROJECTION": "UTM", "DATUM": "WGS84", "ELLIPSOID": "WGS84", "UTM_ZONE": "27", "GRID_CELL_SIZE_PANCHROMATIC": "15.00", "GRID_CELL_SIZE_REFLECTIVE": "30.00", "GRID_CELL_SIZE_THERMAL": "30.00", "ORIENTATION": "NORTH_UP", "RESAMPLING_OPTION": "CUBIC_CONVOLUTION", "SCAN_GAP_INTERPOLATION": "2.0" } } }

Figure 3-1. Example MTL Metadata File in JSON Format

3.2.2 STAC records In addition to converting existing metadata to JSON, additional metadata is generated known as STAC item records. STAC allows data to be exposed for programmatic discovery and access over the web, and is designed to make access to large, cloud-based datasets easier and more consistent. See Section 1.2 for more information on STAC. Each Landsat scene stored in the cloud has at least one STAC item record associated with it. Landsat scenes with more than one product, for example Level 2 products with Surface Reflectance (SR) and Surface Temperature (ST) bands, may have more than one STAC item record. One STAC item record is generated per type of output that a product contains, as the STAC item records are organized under STAC collections. These collections are not the same as Landsat Collections, and represent datasets with shared characteristics. For Landsat data in the cloud, the STAC collections group STAC item records which refer to data from one Landsat Level and Collection that share an

- 11 - LSDS-1388 Version 2.0

output type, for example Collection 1 Level 1 Top of Atmosphere (TOA) radiation data, as in the example given in Figure 3-2. { "stac_version": "0.7.0", "stac_extensions": [ "eo", "landsat" ], "id": "LE07_L1TP_001005_20090619_20161219_01_T1", "type": "Feature", "bbox": [ -26.59215192604375, 75.77952687939155, -14.497719281559315, 78.26024529257339 ], "geometry": { "type": "Polygon", "coordinates": [ [ [ -25.656149216737866, 75.7954660298195 ], [ -15.584195604114601, 75.77952687939155 ], [ -14.497719281559315, 78.1863618997909 ], [ -26.59215192604375, 78.20567407241408 ], [ -25.656149216737866, 75.7954660298195 ] ] ] }, "properties": { "datetime": "2009-06-19", "eo:cloud_cover": 4.0, "landsat:cloud_cover_land": 5.0, "eo:sun_azimuth": -169.92167832, "eo:sun_elevation": 36.21102154, "landsat:wrs_type": 2, "landsat:wrs_path": 1, "landsat:wrs_row": 5, "eo:gsd": 30, "eo:platform": "LANDSAT_7",

- 12 - LSDS-1388 Version 2.0

"eo:instrument": "ETM", "eo:off_nadir": 0, "landsat:scene_id": "LE70010052009170EDC00", "landsat:collection_category": "T1", "landsat:collection_number": "01", "eo:bands": [ { "name": "B1", "common_name": "blue", "gsd": 30, "center_wavelength": 0.485 }, { "name": "B2", "common_name": "green", "gsd": 30, "center_wavelength": 0.56 }, { "name": "B3", "common_name": "red", "gsd": 30, "center_wavelength": 0.66 }, { "name": "B4", "common_name": "nir", "gsd": 30, "center_wavelength": 0.835 }, { "name": "B5", "common_name": "swir", "gsd": 30, "center_wavelength": 1.65 }, { "name": "B6", "common_name": "thermal", "gsd": 30, "center_wavelength": 11.45 }, { "name": "B7", "common_name": "swir22", "gsd": 30, "center_wavelength": 2.22 }, { "name": "B8", "common_name": "panchromatic", "gsd": 30, "center_wavelength": 0.71 } ], "collection": "landsat-c1l1-tarad"

- 13 - LSDS-1388 Version 2.0

}, "assets": { "ANG": { "href": "s3://dev-usgs-landsat/collection01/level-1/standard/etm/2009/001/005/LE07_L1TP_001005_20090619_20161219_01_T1/LE07_L1TP_001005_20090619_20161219_01_T1_ANG.txt", "title": "Angle coefficients file", "type": "text/html" }, "MTL": { "href": "s3://dev-usgs-landsat/collection01/level-1/standard/etm/2009/001/005/LE07_L1TP_001005_20090619_20161219_01_T1/LE07_L1TP_001005_20090619_20161219_01_T1_MTL.txt", "title": "Product Metadata File/Object Description Language Format", "type": "text/plain" }, "JSON": { "href": "s3://dev-usgs-landsat/collection01/level-1/standard/etm/2009/001/005/LE07_L1TP_001005_20090619_20161219_01_T1/LE07_L1TP_001005_20090619_20161219_01_T1_MTL.json", "title": "Product Metadata File/JavaScript Object Notation Format", "type": "application/json" }, "thumbnail": { "href": "s3://dev-usgs-landsat/collection01/level-1/standard/etm/2009/001/005/LE07_L1TP_001005_20090619_20161219_01_T1/LE07_L1TP_001005_20090619_20161219_01_T1_thumb_small.jpeg", "title": "Thumbnail image", "type": "image/jpeg" }, "reduced_resolution_browse": { "href": "s3://dev-usgs-landsat/collection01/level-1/standard/etm/2009/001/005/LE07_L1TP_001005_20090619_20161219_01_T1/LE07_L1TP_001005_20090619_20161219_01_T1_thumb_large.jpeg", "title": "Reduced resolution browse image", "type": "image/jpeg" }, "BQA": { "href": "s3://dev-usgs-landsat/collection01/level-1/standard/etm/2009/001/005/LE07_L1TP_001005_20090619_20161219_01_T1/LE07_L1TP_001005_20090619_20161219_01_T1_BQA.tif", "title": "BQA Quality Assessment Band", "type": "image/vnd.stac.geotiff; cloud-optimized=true" }, "B1": { "href": "s3://dev-usgs-landsat/collection01/level-1/standard/etm/2009/001/005/LE07_L1TP_001005_20090619_20161219_01_T1/LE07_L1TP_001005_20090619_20161219_01_T1_B1.tif", "eo:bands": [ 0 ], "title": "Band 1 (blue)", "type": "image/vnd.stac.geotiff; cloud-optimized=true" },

- 14 - LSDS-1388 Version 2.0

"B2": { "href": "s3://dev-usgs-landsat/collection01/level-1/standard/etm/2009/001/005/LE07_L1TP_001005_20090619_20161219_01_T1/LE07_L1TP_001005_20090619_20161219_01_T1_B2.tif", "eo:bands": [ 1 ], "title": "Band 2 (green)", "type": "image/vnd.stac.geotiff; cloud-optimized=true" }, "B3": { "href": "s3://dev-usgs-landsat/collection01/level-1/standard/etm/2009/001/005/LE07_L1TP_001005_20090619_20161219_01_T1/LE07_L1TP_001005_20090619_20161219_01_T1_B3.tif", "eo:bands": [ 2 ], "title": "Band 3 (red)", "type": "image/vnd.stac.geotiff; cloud-optimized=true" }, "B4": { "href": "s3://dev-usgs-landsat/collection01/level-1/standard/etm/2009/001/005/LE07_L1TP_001005_20090619_20161219_01_T1/LE07_L1TP_001005_20090619_20161219_01_T1_B4.tif", "eo:bands": [ 3 ], "title": "Band 4 (nir)", "type": "image/vnd.stac.geotiff; cloud-optimized=true" }, "B5": { "href": "s3://dev-usgs-landsat/collection01/level-1/standard/etm/2009/001/005/LE07_L1TP_001005_20090619_20161219_01_T1/LE07_L1TP_001005_20090619_20161219_01_T1_B5.tif", "eo:bands": [ 4 ], "title": "Band 5 (swir)", "type": "image/vnd.stac.geotiff; cloud-optimized=true" }, "B6": { "href": "s3://dev-usgs-landsat/collection01/level-1/standard/etm/2009/001/005/LE07_L1TP_001005_20090619_20161219_01_T1/LE07_L1TP_001005_20090619_20161219_01_T1_B6.tif", "eo:bands": [ 5 ], "title": "Band 6 (thermal)", "type": "image/vnd.stac.geotiff; cloud-optimized=true" }, "B7": { "href": "s3://dev-usgs-landsat/collection01/level-1/standard/etm/2009/001/005/LE07_L1TP_001005_20090619_20161219_01_T1/LE07_L1TP_001005_20090619_20161219_01_T1_B7.tif", "eo:bands": [ 6

- 15 - LSDS-1388 Version 2.0

], "title": "Band 7 (swir22)", "type": "image/vnd.stac.geotiff; cloud-optimized=true" }, "B8": { "href": "s3://dev-usgs-landsat/collection01/level-1/standard/etm/2009/001/005/LE07_L1TP_001005_20090619_20161219_01_T1/LE07_L1TP_001005_20090619_20161219_01_T1_B8.tif", "eo:bands": [ 7 ], "title": "Band 8 (panchromatic)", "type": "image/vnd.stac.geotiff; cloud-optimized=true" } }, "links": [ { "rel": "root", "href": "s3://dev-usgs-landsat-stac-browser/stac/catalog.json" }, { "rel": "parent", "href": "s3://dev-usgs-landsat-stac-browser/stac/collection01/level-1tarad/catalog.json" }, { "rel": "collection", "href": "s3://dev-usgs-landsat-stac-browser/stac/collections/landsat-c1l1-tarad/collection.json" }, { "rel": "self", "href": "s3://dev-usgs-landsat/collection01/level-1/standard/etm/2009/001/005/LE07_L1TP_001005_20090619_20161219_01_T1/LE07_L1TP_001005_20090619_20161219_01_T1_TARAD_stac.json" } ], "description": "Landsat Collection 1 Level 1 Top of Atmosphere Radiation product" }

Figure 3-2. Example STAC Item Record

- 16 - LSDS-1388 Version 2.0

Appendix A Landsat DFCBs

A.1 Collection 1 DFCBs Document Number Document Name and link LSDS-272 Landsat 7 Enhanced Thematic Mapper

Plus (ETM+) Level 1 Data Format Control Book (U.S. Geological Survey 2016)

LSDS-284 Landsat Thematic Mapper (TM) Level 1 Data Format Control Book (U.S. Geological Survey 2018)

LSDS-286 Landsat Multispectral Scanner (MSS) Level 1 Data Format Control Book (U.S. Geological Survey 2018)

LSDS-809 Landsat 8 Level 1 Data Format Control Book (U.S. Geological Survey 2017)

Table A-1. Landsat Collection 1 Data Format Control Books

https://www.usgs.gov/media/files/landsat-7-level-1-data-format-control-bookhttps://www.usgs.gov/media/files/landsat-7-level-1-data-format-control-bookhttps://www.usgs.gov/media/files/landsat-7-level-1-data-format-control-bookhttps://www.usgs.gov/media/files/landsat-4-5-thematic-mapper-level-1-data-format-control-bookhttps://www.usgs.gov/media/files/landsat-4-5-thematic-mapper-level-1-data-format-control-bookhttps://www.usgs.gov/media/files/landsat-1-5-multispectral-scanner-level-1-data-format-control-bookhttps://www.usgs.gov/media/files/landsat-1-5-multispectral-scanner-level-1-data-format-control-bookhttps://www.usgs.gov/media/files/landsat-8-level-1-data-format-control-bookhttps://www.usgs.gov/media/files/landsat-8-level-1-data-format-control-book

- 17 - LSDS-1388 Version 2.0

A.2 Collection 2 DFCBs Document Number Document Name and link LSDS 1328 Landsat 8-9 Operational Land Imager

(OLI) – Thermal Infrared Sensor (TIRS) Collection 2 Level 2 Data Format Control Book (U.S. Geological Survey 2019)

LSDS 1336 Landsat TM Collection 2 Level 2 Data Format Control Book (U.S. Geological Survey 2019)

LSDS 1337 Landsat ETM+ Collection 2 Level 2 Data Format Control Book (U.S. Geological Survey 2019)

LSDS 1414 Landsat 7 ETM+ Collection 2 Level 1 Data Format Control Book (U.S. Geological Survey 2019)

LSDS 1415 Landsat TM Collection 2 Level 1 Data Format Control Book (U.S. Geological Survey 2019)

LSDS 1416 Landsat MSS Collection 2 Level 1 Data Format Control Book (U.S. Geological Survey 2019)

LSDS 1822 Landsat 8-9 OLI-TIRS Collection 2 Level 1 Data Format Control Book (U.S. Geological Survey 2019)

Table A-2. Landsat Collection 2 Data Format Control Books

https://www.usgs.gov/media/files/landsat-8-collection-2-level-2-data-format-control-bookhttps://www.usgs.gov/media/files/landsat-8-collection-2-level-2-data-format-control-bookhttps://www.usgs.gov/media/files/landsat-8-collection-2-level-2-data-format-control-bookhttps://www.usgs.gov/media/files/landsat-8-collection-2-level-2-data-format-control-bookhttps://www.usgs.gov/media/files/landsat-4-5-tm-collection-2-level-2-data-format-control-bookhttps://www.usgs.gov/media/files/landsat-4-5-tm-collection-2-level-2-data-format-control-bookhttps://www.usgs.gov/media/files/landsat-7-collection-2-level-2-data-format-control-bookhttps://www.usgs.gov/media/files/landsat-7-collection-2-level-2-data-format-control-bookhttps://www.usgs.gov/media/files/landsat-7-collection-2-level-1-data-format-control-bookhttps://www.usgs.gov/media/files/landsat-7-collection-2-level-1-data-format-control-bookhttps://www.usgs.gov/media/files/landsat-4-5-tm-collection-2-level-1-data-format-control-bookhttps://www.usgs.gov/media/files/landsat-4-5-tm-collection-2-level-1-data-format-control-bookhttps://www.usgs.gov/media/files/landsat-1-5-mss-collection-2-level-1-data-format-control-bookhttps://www.usgs.gov/media/files/landsat-1-5-mss-collection-2-level-1-data-format-control-bookhttps://www.usgs.gov/media/files/landsat-8-9-olitirs-level-1-data-format-control-bookhttps://www.usgs.gov/media/files/landsat-8-9-olitirs-level-1-data-format-control-book

- 18 - LSDS-1388 Version 2.0

Appendix B Acronyms

BQA Quality Assessment Band C1 Collection 1 C2 Collection 2 CCB Configuration Control Board COG Cloud Optimized GeoTIFF CPF Calibration Parameter File CR Change Request DFCB Data Format Control Book DPAS Data Processing and Archive System EROS Earth Resources Observation and Science ETM Enhanced Thematic Mapper ETM+ Enhanced Thematic Mapper Plus GCP Ground Control Point GeoJSON Geographical JavaScript Object Notation GeoTIFF Georeferenced Tagged Image File Format GLS2000 Global Land Survey 2000 HTTP Hypertext Transfer Protocol JSON JavaScript Object Notation L1 Level 1 Data Product L2 Level 2 Data Product L7 Landsat 7 L8 Landsat 8 L9 Landsat 9 LPGS Landsat Product Generation System MSS Multispectral Scanner MTL Metadata Text File Extension NIR Near Infrared ODL Object Description Language OLI Operational Land Imager QA Quality Assessment RMSE Root Mean Square Error S3 Amazon Simple Storage Service SR Surface Reflectance ST Surface Temperature STAC SpatioTemporal Asset Catalog SWIR Shortwave Infrared TIFF or TIF Tagged Image File Format TIRS Thermal Infrared Sensor TM Thematic Mapper TOA Top of Atmosphere

- 19 - LSDS-1388 Version 2.0

USGS U.S. Geological Survey UTM Universal Transverse Mercator VCID Virtual Channel Identification WGS84 World Geodetic System 1984 WRS Worldwide Reference System WRS-2 Worldwide Reference System-2 XML Extensible Markup Language

- 20 - LSDS-1388 Version 2.0

References

Please see https://www.usgs.gov/land-resources/nli/landsat/glossary-and-acronyms for a list of acronyms.

Gavin, David, Chris Holmes, and Phil Broderic. 2019. SpatioTemporal Asset Catalogs.

Accessed 2019. https://stacspec.org/. Holmes, Chris. 2017. Cloud Optimized GeoTIFF in depth. Accessed 2019.

https://www.cogeo.org/in-depth.html. Holmes, Chris, and Even Roualt. 2019. Cloud optimized GeoTIFF. Accessed 2019.

https://github.com/cogeotiff/cog-spec/blob/master/spec.md. Internet Engineering Task Force . 2016. GeoJson. Accessed 2019. https://geojson.org/. Internet Engineering Task Force. 2014. RFC 7233 - Hypertext Transfer Protocol

(HTTP/1.1): Range Requests. June. Accessed 2019. https://tools.ietf.org/html/rfc7233.

U.S. Geological Survey. 2018. "Landsat Multispectral Scanner Level 1 Data Format

Control Book." usgs.gov. February. Accessed 2019. https://www.usgs.gov/media/files/landsat-1-5-multispectral-scanner-level-1-data-format-control-book.

U.S. Geological Survey. 2018. "Landsat Thematic Mapper Level 1 Data Format Control

Book." usgs.gov. February. Accessed 2019. https://www.usgs.gov/media/files/landsat-4-5-thematic-mapper-level-1-data-format-control-book.

U.S. Geological Survey. 2019. "Landsat TM Collection 2 Level 1 Data Format Control

Book." usgs.gov. August. Accessed 2019. https://www.usgs.gov/media/files/landsat-4-5-tm-collection-2-level-1-data-format-control-book.

U.S. Geological Survey. 2019. "Landsat TM Collection 2 Level 2 Data Format Control

Book." usgs.gov. September. Accessed 2019. https://www.usgs.gov/media/files/landsat-4-5-tm-collection-2-level-2-data-format-control-book.

U.S. Geological Survey. 2019. "Landsat 7 ETM+ Collection 2 Level 1 Data Format

Control Book." usgs.gov. August. Accessed 2019. https://www.usgs.gov/media/files/landsat-7-collection-2-level-1-data-format-control-book.

https://www.usgs.gov/land-resources/nli/landsat/glossary-and-acronyms

- 21 - LSDS-1388 Version 2.0

U.S. Geological Survey. 2019. "Landsat ETM+ Collection 2 Level 2 Data Format Control Book." usgs.gov. September. Accessed 2019. https://www.usgs.gov/media/files/landsat-7-collection-2-level-2-data-format-control-book.

U.S. Geological Survey. 2016. "Landsat 7 ETM+ Level 1 Data Format Control Book."

usgs.gov. August. Accessed 2019. https://www.usgs.gov/media/files/landsat-7-level-1-data-format-control-book.

U.S. Geological Survey. 2019. "Landsat 8-9 OLI-TIRS Collection 2 Level 2 Data Format

Control Book." usgs.gov. September. Accessed 2019. https://www.usgs.gov/media/files/landsat-8-collection-2-level-2-data-format-control-book.

U.S. Geological Survey. 2017. "Landsat 8 Level 1 Data Format Control Book."

usgs.gov. Accessed 2019. https://www.usgs.gov/media/files/landsat-8-level-1-data-format-control-book.

U.S. Geological Survey. 2019. "Landsat 8-9 OLI/TIRS Level 1 Data Format Control

Book." usgs.gov. April. Accessed 2019. https://www.usgs.gov/media/files/landsat-8-9-olitirs-level-1-data-format-control-book.

U.S. Geological Survey. 2019. "Landsat MSS Collection 2 Level 1 Data Format Control

Book." usgs.gov. August. Accessed 2019. https://www.usgs.gov/media/files/landsat-mss-collection-2-level-1-data-format-control-book.

U.S. Geological Survey. 2019. Landsat Project Documents. Accessed 2019.

https://www.usgs.gov/land-resources/nli/landsat/landsat-project-documents.

Executive SummaryDocument HistoryContentsList of FiguresList of TablesSection 1 Introduction1.1 Purpose and Scope1.2 Terminology

Section 2 COG Data Format Definition2.1 COG Format Overview2.2 COG file structure2.2.1 File Headers and Tags2.2.2 Internal Overviews2.2.3 Internal Tiling

2.3 COG Creation Specification

Section 3 Product Distribution Data Format Definition3.1 Product Distribution Packaging3.2 Cloud Native Product Metadata3.2.1 JSON Metadata conversion3.2.2 STAC records

Appendix A Landsat DFCBsA.1 Collection 1 DFCBsA.2 Collection 2 DFCBs

Appendix B AcronymsReferences

2020-02-11T09:55:10-0600CHRISTOPHER ENGEBRETSON