Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data...

35
Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data Distribution System (DDS) And The SDO Science Operations Centers (SOC) 464-GS-ICD-0010 Revision A Effective Date: 06/23/2005 Expiration Date: 06/23/2010 Goddard Space Flight Center Greenbelt, Maryland National Aeronautics and Space Administration CHECK THE SDO MIS AT https://sdomis.gsfc.nasa.gov TO VERIFY THAT THIS IS THE CORRECT VERSION PRIOR TO USE.

Transcript of Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data...

Page 1: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

Solar Dynamics Observatory (SDO)

Interface Control Document (ICD) Between

The SDO Data Distribution System (DDS) And

The SDO Science Operations Centers (SOC)

464-GS-ICD-0010 Revision A

Effective Date: 06/23/2005

Expiration Date: 06/23/2010

Goddard Space Flight Center

Greenbelt, Maryland

National Aeronautics and

Space Administration

CHECK THE SDO MIS AT https://sdomis.gsfc.nasa.gov

TO VERIFY THAT THIS IS THE CORRECT VERSION PRIOR TO USE.

Page 2: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

464-GS-ICD-0010 Revision A

CM FOREWORD This document is a Solar Dynamics Observatory Project controlled document. Changes to this document require prior approval of the SDO Project CCB Chairperson. Proposed changes shall be submitted to the SDO Project Configuration Management Office (CMO), along with supportive material justifying the proposed change.

Questions or comments concerning this document should be addressed to:

SDO Configuration Management Office Mail Stop 464 Goddard Space Flight Center Greenbelt, Maryland 20771

June 23, 2005 ii

Page 3: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

464-GS-ICD-0010 Revision A

Reviewed/Approved By (For Revision -): Hollys Allen

Michael Anderson

Tom Anderson

Velma Anderson

Bob Calvo

Jeffery Ferrara

Pete Gonzales

Eric Grob

Joe Howard

Deepak Kaul

Eliane Larduinat

Manuel Maldonado

Jack McCabe

Ray Pages

Dean Pesnell

Bill Potter

John Ruffa

Chad Salo

Michael Scott

Frank Scoville

Hun Tann

Mike Uffer

John VanBlarcom

Dave Ward

Craig Weikel

John Wiedman

June 23, 2005 iii

Page 4: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

464-GS-ICD-0010 Revision A

Revision A (Reviewed By):

Hollys Allen Michael Anderson Tom Anderson Mike Bay Tom Bialas Ron Blazek Bob Calvo Paula Everson Jeffrey Ferrara Pete Gonzales Eliane Larduinat Manuel Maldonado Ray Pages Dean Pesnell Bill Potter Brent Robertson John Ruffa Chad Salo Brett Sapper Michael Scott Frank Scoville Hun Tann Mike Uffer John VanBlarcom Craig Weikel

June 23, 2005 ii

Page 5: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

464-GS-ICD-0010 Revision A

DOCUMENT CHANGE RECORD

REV/ VER

LEVEL DESCRIPTION OF CHANGE

APPROVED BY

DATE APPROVED

-

A

Baseline Release of Sections 1, 2, 3, & 4 per SDO-CCR-0192

Update Sections 1-4 and release Section 5 per the approval of SDO-CCR-0319

R. Lilly

R. Lilly

10/25/2004

06/23/2005

June 23, 2005 iii

Page 6: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

464-GS-ICD-0010 Revision A

TABLE OF CONTENTS

1.0 SCOPE................................................................................................................1-1

1.1 Document Identification.............................................................................................................................. 1-1

1.2 Interface Overview ...................................................................................................................................... 1-1

1.3 Document Organization .............................................................................................................................. 1-3

2.0 REFERENCE DOCUMENTS ..............................................................................2-1

3.0 INTERFACE OVERVIEW ...................................................................................3-1

3.1 Science data Interface.................................................................................................................................. 3-1 3.1.1 Data Distribution System Overview ...................................................................................................... 3-1 3.1.2 Delivery of Science Telemetry Overview.............................................................................................. 3-1

3.1.2.1 Science Telemetry Delivery Scenario............................................................................................ 3-1 3.1.2.2 Data Volumes ................................................................................................................................ 3-3 3.1.2.3 Fault Recovery............................................................................................................................... 3-4

4.0 DATA FORMAT SPECIFICATION .....................................................................4-1

4.1 Science Data Specifications ......................................................................................................................... 4-1 4.1.1 Science Telemetry Files (TLM and ERR) ............................................................................................. 4-1

4.1.1.1 TLM File Naming Convention ...................................................................................................... 4-4 4.1.1.2 TLM File Format ........................................................................................................................... 4-4

4.1.2 ERR File Naming Convention............................................................................................................... 4-6 4.1.2.1 ERR File Format............................................................................................................................ 4-6

4.1.3 Quality and Accounting (QAC) Files .................................................................................................... 4-6 4.1.3.1 QAC File Naming Convention ...................................................................................................... 4-7 4.1.3.2 QAC File Format ........................................................................................................................... 4-8

4.1.4 Delivery Status File (DSF) .................................................................................................................. 4-10 4.1.4.1 DSF File Naming Convention ..................................................................................................... 4-10 4.1.4.2 DSF File Format .......................................................................................................................... 4-10

4.1.5 Acknowledgement status File (ASF) ................................................................................................... 4-11 4.1.5.1 ASF File Naming Convention ..................................................................................................... 4-11 4.1.5.2 ASF File Format .......................................................................................................................... 4-11

4.1.6 Archive Status File (ARC)................................................................................................................... 4-12 4.1.6.1 ARC File Naming Convention..................................................................................................... 4-12 4.1.6.2 ARC File Format ......................................................................................................................... 4-12

5.0 COMMUNICATIONS PROTOCOLS...................................................................5-1

5.1 Delivery Address.......................................................................................................................................... 5-1

5.2 File Transfer Protocols................................................................................................................................ 5-1

5.3 Directory Structures.................................................................................................................................... 5-1

5.4 Security Measures........................................................................................................................................ 5-1

June 23, 2005 iv

Page 7: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

464-GS-ICD-0010 Revision A

APPENDIX A. LIST OF ACRONYMS............................................................................. 1

APPENDIX B – CONFIGURABLE PARAMETERS........................................................ 1

June 23, 2005 v

Page 8: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

464-GS-ICD-0010 Revision A

LIST OF FIGURES Figure Page

Figure 1-1. Document Scope..................................................................................................................1-1

Figure 3-1. File Exchange Between SOCs and DDS .............................................................................3-3

Figure 4-1. Science Data Packet, MPDU, and VCDU Structure ...........................................................4-3

Figure 4-2. TLM File Format. ..................................................................................................................4-5

Figure 5-1 SDO network diagram………………………………………………………………………………..5-2

LIST OF TABLES Table Page

Table 3-1. Science Data Volumes ............................................................................................................3-4

Table 3-2. Transmission/Retransmission Capacity .............................................................................3-4

Table 4- 1, Science Telemetry Products ................................................................................................4-1

Table 4- 2, Instrument VCID and IM_PDU_ID Assignments.................................................................4-2

Table 4- 3, QAC File Key Words .............................................................................................................4-8

Table 4- 4, DSF File Keywords ..............................................................................................................4-10

Table 4- 5, Status Values .......................................................................................................................4-11

Table 4- 6, ASF File Keywords ..............................................................................................................4-11

Table 4- 7, ASF Status Values...............................................................................................................4-12

Table 4- 8, ARC Keywords.....................................................................................................................4-12

June 23, 2005 vi

Page 9: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

464-GS-ICD-0010 Revision A

List of TBDs/TBRs

Item No.

Location Summary Ind./Org. Due Date

June 23, 2005 vii

Page 10: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

464-GS-ICD-0010 Revision A

June 23, 2005 1-1

1.0 SCOPE

1.1 DOCUMENT IDENTIFICATION This Interface Control Document (ICD) defines the interface between the Solar Dynamics Observatory (SDO) Science Operations Centers (SOC) and the SDO Data Distribution System (DDS).

It applies to the operational phase of the mission. All special accommodations needed for the I&T phase will be documented separately. Figure 1-1 illustrates the scope of this ICD.

SDO Ground System

HMISOC

Stanford, CA

EVESOC

Boulder, CO

Interfacedescribedin this ICD

AIASOC

Palo Alto, CA

DDSWhite Sands, NM

Science TelemetryKa-Band

Protocol Data

Figure 1-1. Document Scope

1.2 INTERFACE OVERVIEW SDO carries a complement of three instruments, each with its own SOC:

• The Extreme Ultraviolet Variability Experiment (EVE) led from the Laboratory for Atmospheric and Space Physics (LASP) in Boulder, CO

• The Helioseismic and Magnetic Imager (HMI) led from Stanford University in Stanford, CA.

Page 11: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

464-GS-ICD-0010 Revision A

• The Atmospheric Imaging Assembly (AIA) led from the Lockheed Martin Solar and Astrophysics Laboratory (LMSAL) in Palo Alto, CA.

The HMI and AIA science operations will be co-located within the Joint Science Operations Center (JSOC), which will include sites at LMSAL and Stanford University. The command and control of the instruments will be handled at the LMSAL site. Science data will be received at the Stanford University site. The link between JSOC and the MOC will consist of two functionally separate interfaces, one for each instrument.

The Data Distribution System (DDS), located at the White Sands Complex in New Mexico, provides the interface between the SDO Ground Station and the SOCs for the science data. DDS is connected to the SOCs through high rate data lines. DDS distributes the data to each SOC on a continuous basis and supports retransmissions as needed. Figure 1-2 provides a context diagram of the DDS/SOC interface.

Data DistributionSystem

(DDS)

Quality and Accounting (QAC)

SDO Ground System

Retransmit Science Data files (TLM & ERR)

Daily Archive File (ARC)

Delivery Status File (DSF)

Acknowledgement status file (ASF)

Real time Science Data files (TLM) HMISOC

AIASOC

JSOC

EVESOC

SDO SOCS

Figure 1-2. Interface Context Diagram

June 23, 2005 1-2

Page 12: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

464-GS-ICD-0010 Revision A

June 23, 2005 1-3

1.3 DOCUMENT ORGANIZATION This document is structured as follows:

• Section 1 provides an overview of this document and the interface it describes.

• Section 2 lists all referenced documents.

• Section 3 defines the functional interface for the distribution of Science Data to the SOCs.

• Section 4 defines the detailed format of all data products exchanged between the DDS and the SOCs.

• Section 5 defines the communication protocols used to support the data transfers

• Appendix A provides an acronym list.

• Appendix B provides a list of configurable parameters.

Page 13: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

464-GS-ICD-0010 Revision A

June 23, 2005 2-1

2.0 REFERENCE DOCUMENTS

The following documents are referenced in this ICD:

• Reference 1. SDO High Speed Bus (HSB) ICD, 464-CDH-ICD-0012

• Reference 2. SDO CCSDS Implementation Document, 464-SYS-SPEC-0033

The following documents provide the requirements flow down on which this ICD is based:

• Reference 3. Mission Requirements Document (MRD), 464-SYS-REQ-0004

• Reference 4. Detailed Mission Requirements (DMR) for the SDO Ground System, 464-GS-REQ-0005

• Reference 5. SDO DDS Requirements Specification, 464-GS-REQ-0049

If conflicts exist between this document and other referenced documents, the following order of precedence will apply:

1. Mission Requirements Document (MRD), 464-SYS-REQ-0004 (Reference 3)

2. Detailed Mission Requirements (DMR) for SDO Mission (Reference 4)

3. SDO High Speed Bus (HSB) ICD (Reference 1)

4. SDO CCSDS Implementation Document (Reference 2)

5. Interface Control Document between the SDO DDS and the SOCs (this document)

Page 14: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

464-GS-ICD-0010 Revision A

June 23, 2005 3-1

3.0 INTERFACE OVERVIEW

3.1 SCIENCE DATA INTERFACE

3.1.1 Data Distribution System Overview The SDO Data Distribution System (DDS) is located in the White Sands Ground Terminal (WSGT) at the White Sands Complex. DDS receives the Ka-Band high rate data, creates files of science data and distributes these files to the SOCs in near real-time. Each SOC receives only the data produced by their own instruments.

The SDO DDS has two main elements: the high-rate Front End Processors and the DDS Core System.

The high-rate Front End Processors (FEP) are physically co-located with the SDO antennas. One antenna is at the Second TDRS Ground Terminal (STGT) and the other at WSGT. Two FEPs for each antenna provide full redundancy. The FEPs perform demodulation, Viterbi decoding, frame synchronization, and Reed-Solomon decoding. They then store the data in a 5-day temporary circular buffer used for replays and failure recovery when necessary. The data is sent in near-real time to the DDS Core System.

The DDS Core System accepts data from either FEP or both FEPs simultaneously. Overlapping data is compared and the better quality data is stored in files by Virtual Channel Identifier (VCID). The files are fixed-length and contain approximately 1 minute of data. Thus, they are distributed to the SOCs with a latency of roughly one minute. The files are also archived in a short-term 30-day archive providing the capability to retransmit data as necessary. The DDS archive system is a single fault tolerant RAID disk system, with a capacity of about 42 terabytes.

3.1.2 Delivery of Science Telemetry Overview

3.1.2.1 Science Telemetry Delivery Scenario The operational scenario for the distribution of science data to the SOCs is based on the concept of sending files covering approximately one minute of downlink and monitoring the transmissions by exchanging directory files and status files. DDS will attempt to deliver the data files only one time when they are first created. If the transmission fails, the SOCs are responsible for requesting a retransmission. Figure 3-1 shows the exchange of files.

For the purposes of this ICD a day starts at 00:00 UTC and ends at 23:59 UTC. Daily activity files will be based on this 24-hour period.

The operational scenario can be summarized as follows:

1. DDS creates telemetry data files (TLM), covering approximately one minute's worth of downlink for each VCID.

2. DDS also creates an error file (ERR) that contains VCDUs that were flagged by the Spacecraft as being corrupted (i.e. too long, too short, error). ERR files will be delivered only if requested via a retransmission request by the SOC.

3. Once a file is created, closed and stored in the DDS 30-day storage, DDS sends (pushes) it, to the receiving SOC with minimal delay. Each SOC receives only the VCIDs from its own instrument group.

4. A Quality and Accounting (QAC) file accompanies each telemetry data file. It provides information about the associated telemetry data file, such as its size in bytes, and the number of valid VCDUs that it contains. The TLM file will always be transferred first followed by the QAC file.

Page 15: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

464-GS-ICD-0010 Revision A

June 23, 2005 3-2

5. DDS maintains a catalog of all the data files and their transmission status. Once per hour on the hour the DDS will attempt to send a Delivery Status File (DSF) to each SOC. The DSF lists all the files for a given SOC/Instrument that exist within DDS that are closed and available for delivery and have not yet been acknowledged by the SOC.

Files will remain listed in the DSF until they have been acknowledged in an Acknowledgement Status File (ASF) or Archival Status File (ARC) as successfully received by the SOCs. Files requested for retransmission will be removed from the DSF while queued for delivery and put back into the DSF after the retransmission attempt has been made. Both the DSF and ASF have no fixed size limit and could contain 30 days worth of entries.

6. The SOCs answer the DSF with an Acknowledgement Status File (ASF), which is similar in format to the DSF file and either acknowledges a file receipt or requests its re-transmission. The ASF is retrieved by the DDS every hour on the half hour (configurable).

7. When a re-transmission request is received, the requested file(s) is removed from subsequent DSF files until delivery has again been attempted. DDS queues all the re-transmission requests on a First-In-First-Out basis and performs them as allowed by the communication line capacity.

8. On a daily basis, each SOC creates an Archival Status file (ARC) confirming the acknowledgement of all the files the SOC successfully received and archived on that day. The ARC is retrieved by the DDS every day at a fixed time (configurable) after 00:15 UTC.

9. On a daily basis, the DDS notifies the SOC via email of files that have not been acknowledged. The emails will be sent at 10 days after file creation and again at 25 days after file creation. All files that are 10 and 25 days old on a given day and have not been acknowledged will be included in the email. Email addresses will be configurable and defined in the operational documentation.

10. DDS deletes all files older than 30 days.

Page 16: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

464-GS-ICD-0010 Revision A

HMI SOCSTANFORD

STANFORD,CA

DDS

DSF on the hour

ASF on the 1/2 hour

ARC daily, 00:15 GMT

Email as needed

TLM and QAC files, every minute

ERR on request

DSF on the hour

ASF on the 1/2 hour

ARC daily, 00:15 GMT

Email as needed

TLM and QAC files, every minute

IP over OC3

ERR on request

DSF on the hour

ASF on the 1/2 hour

ARC daily, 00:15 GMT

Email as needed

TLM and QAC files, every minute

IP over OC3

ERR on request

AIA SOCLMSAL/

STANFORDPALO ALTO,CA

JSOCSTANFORD

STANFORD,CA

DDSFile OutputProcessor

DDSFile OutputProcessor

DDSFile OutputProcessor

EVE SOCLASP

BOULDER, CO

IP over T3

Figure 3-1. File Exchange Between SOCs and DDS

3.1.2.2 Data Volumes On average, DDS distributes a total of 1.4 Terabytes of data per day to the three SOCs. The data is collected in files based on VCID, and the number and size of files created depend on the number of VCIDs used by each instrument, and the number of VCDUs in each file. Table 3-1 summarizes the nominalvolumes of data and number of TLM files transferred to each SOC. The telemetry files have a predetermined (configurable) length corresponding to approximately one minute of downlink.

June 23, 2005 3-3

Page 17: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

464-GS-ICD-0010 Revision A

June 23, 2005 3-4

Table 3-1. Science Data Volumes

SOC Total Data Rate

VCIDs Daily Volume Approx. 30-day Volume

Number of TLM

files/day

TLM file Size

# of VCDUs/file

(1) AIA 67 Mbps 2 active 724 Gigabytes 22 Terabytes 2880 251 Mbytes 140520

HMI 55 Mbps 2 active 594 Gigabytes 18 Terabytes 2880 206 Mbytes 115352

EVE 7 Mbps 1 active 76 Gigabytes 2.3 Terabyte 1440 53 Mbytes 29362

(1) Configurable

Table 3-2 summarizes the capacity of the communication lines provided for initial transmission and for retransmissions of the science data. These estimates were obtained assuming a 20% transmission overhead. The average latency for the initial transmission is approximately one minute. The latency for retransmissions depends on the total capacity of the communications lines.

Table 3-2. Transmission/Retransmission Capacity

SOC Data Rate

Data Service (B/W)

Transmission Rate with 20% Overhead

Available for Retransmission w/ 20%

Overhead

Time to recover 1

hour of data AIA 67

Mbps OC3

(155Mbps) 80 Mbps 75 Mbps 65 min

HMI 55 Mbps

OC3 (155Mbps)

66 Mbps 89 Mbps 44 min

EVE 7 Mbps T3 (45 Mbps)

8.4 Mbps 36 Mbps 14 min

3.1.2.3 Fault Recovery This section discusses the procedures used to recover from hardware, software and network failures. The failure mechanisms discussed in this section are separated into two categories based on the DDS status.

3.1.2.3.1 DDS Is Nominal, But Cannot Deliver Data

If the DDS is nominal but cannot deliver data, reasons for the delivery failures include but are not limited to:

1. DDS mis-configured

2. Network failure

3. SOC mis-configured

4. SOC machine disk full/broken

Delivery of a given file will be attempted for 2 minutes (configurable). After delivery has been attempted for 2 minutes, the deilivery will be failed and the file will be archived and listed in the DSF. Backlog files (files more than 5 minutes (configurable) old and no delivery attempt made) will be archived and listed in the DSF. If no files have been delivered to a given SOC for more than 3 minutes (configurable), an alarm will be issued to the SDO MOC. The 3 minutes starts at the completion of the last successful delivery. The response by MOC/SOC personnel to such alarms will be defined in the SDO operational documentation. After troubleshooting/repairs, backlog and failed delivery files can be recovered by the SOCs via retransmission requests.

Page 18: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

464-GS-ICD-0010 Revision A

June 23, 2005 3-5

Delivery of files requested for retransmission will also be attempted for 2 minutes (configurable). After 2 minutes the delivery attempt will be failed and the file will again be listed in the DSF. There will be no backlog timeout on retransmission requests. The attempt will be made to deliver all requested files in First In First Out (FIFO) order.

The DDS will attempt to deliver to the SOCs the latest DSF files once every hour on the hour. If delivery fails, no retransmission will be attempted. The SDO MOC will be issued an alarm in the event of such a failure.

The DDS will attempt to retrieve the ASF file from each SOC once per hour on the half-hour. The attempt will be made once. If it fails, an alarm will be issued to the SDO MOC. If multiple ASF files are retrieved, they will be processed in chronological order, starting with the oldest.

The DDS will attempt to retrieve the ARC file from each SOC once per day at a fixed time after 00:15 UTC. The attempt will be made once. If it fails, an alarm will be issued to the SDO MOC. If multiple ARC files are retrieved, they will be processed in chronological order, starting with the oldest.

3.1.2.3.2 DDS Is Broken

If components of the DDS fail such that file delivery stops, the SDO MOC will either be issued an alarm or depending on the nature of the failure, stop getting status data. In either event MOC and SOC personnel will be notified. After troubleshooting/repairs, delivery will resume with the newest files. Backlog files created by such a failure will be archived and listed in the DSF. Real-time delivery of these files will not be attempted.

If the failure is such that reprocessing is necessary, reprocessed files will be archived and listed in the DSF. Real-time delivery of these files will not be attempted. Backlog and reprocessed files can be recovered by the SOCs via retransmission requests. Reprocessing will involve replaying archived data from a FEP and either create new or update existing TLM files. If components of the DDS fail such that files are incomplete or corrupted, the action taken will depend on the nature of problem. The SDO MOC will be notified of any files not meeting minimum completeness criteria. If the cause of the missing data is found to be DDS equipment malfunction, reprocessing will be attempted. The SOCs may also request via the MOC reprocessing of any file that is incomplete or corrupted given that the data is available (5 days from time of receipt for reprocessing). Reprocessing requests will be made manually via phone or e-mail to MOC personnel. Specific contact information will be defined in the SDO MOC-SOC Operations Agreement. If an existing TLM file is updated by reprocessing, its version will be incremented. All versions of a given file will be retained and available for delivery. Reprocessed files will be listed in the DSF only if they are of better quality than the original. Real-time delivery will not be attempted.

If delivery of the hourly DSF file is not possible due to equipment failure, the DDS will deliver only the latest DSF after repairs are made.

Page 19: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

464-GS-ICD-0010 Revision A

June 23, 2005 4-1

4.0 DATA FORMAT SPECIFICATION

4.1 SCIENCE DATA SPECIFICATIONS This section defines the format, transfer protocol, and naming conventions for the data items exchanged over the DDS/SOC interface. These data items are listed in Table 4- 1.

Table 4- 1. Science Telemetry Products

File Ext. From/To Description Comments Science Telemetry File (TLM)

.tlm DDS to SOC Fixed length file containing approximately one minute’s worth of downlink data for a given VCID as contained in VCDU primary header.

DDS “Pushes” to write directly to designated directory within SOC. Directory: dds2soc/tlm.

Science Telemetry ERROR File (ERR)

.err DDS to SOC Variable length file containing Spacecraft defined VCDUs in error received while the associated TLM file was open

DDS “Pushes” to write directly to designated directory within SOC. Directory: dds2soc/err. Delivery is by retransmission request.

Quality and Accounting File (QAC)

.qac DDS to SOC Quality capsule accompanying each TLM file.

DDS “Pushes” to write directly to designated directory within SOC. Directory: dds2soc/tlm.

DDS delivery Status file (DSF)

.dsf DDS to SOC List of all files in DDS that have not been acknowledged by the SOC. One per SOC per hour.

DDS “Pushes” to write directly to designated directory within SOC. Directory: dds2soc/dsf.

Acknowledgement Status file (ASF)

.asf SOC to DDS Response to the DSF file allowing SOC to either acknowledge receipt of TLM files or request their retransmission.

SOC writes to specified directory. Then DDS “Pulls”. Directory: soc2dds/asf.

Archival Status File (ARC)

.arc SOC to DDS Daily confirmation of Acknowledgements/Archival. One per SOC per day.

SOC writes to specified directory. Then DDS “Pulls”. Directory: soc2dds/arc

4.1.1 Science Telemetry Files (TLM and ERR) The DDS generates one TLM file per VCID on approximately a minute basis (configurable). TLM files for a given VCID have a fixed size and include a fixed number of VCDUs. The actual number will be determined by a predefined modulus or VCDU count. The 4-byte Sync pattern will be included with each VCDU. Missing or non-correctable VCDUs are replaced by a VCDUs length of zero fill, thus allowing the TLM files to always have the same length. Table 3-1 defines the nominal file sizes for each instrument.

There will also be a timeout associated with TLM files. If a TLM file is open for more than 2 minutes (configurable), the TLM file will be closed, have a QAC created and have delivery attempted. If additional VCDUs are received that belong in the closed file, the file will be reopened, updated, saved to a new version, have a QAC created, and have delivery attempted. This process will continue until the modulus

Page 20: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

464-GS-ICD-0010 Revision A

June 23, 2005 4-2

for the file has been reached. This scenario is expected to occur during eclipses when the Instruments may reduce their output rate.

An ERR file will be generated if a VCDU is identified by the spacecraft as being in error. Table 4- 2 gives the VCID and IM_PDU_ID assignments per instrument (Please refer to 464-CDH-ICD-0012). ERR VCDUs will have predefined VCIDs and will be associated with a given Instrument VCID. ERR files will contain all ERR VCDUs associated with a given instrument VCID in the order in which they were received. Duplicates will not be removed. The ERR files will cover the same time frame as the like named TLM file (see Section 4.1.2 for ERR filenaming convention). ERR files will be listed in the DSF but not automatically delivered by the DDS in real-time. Delivery of these files will be by retransmission request only. ERR files will need to be acknowledged via the ASF and ARC whether requested for delivery or not.

Table 4- 2. Instrument VCID and IM_PDU_ID Assignments

Source VCID (hex)

VCID (dec)

IM_PDUID

(hex)

IM_PDUID

(dec) AIA Port 1 Science Ka-Band, Normal IM_PDU, Primary Ka Comm Port 1 1 1 10 16 HMI Port 1 Science Ka-Band, Normal IM_PDU, Primary Ka Comm Port 2 2 2 20 32 EVE Port 1 Science Ka-Band, Normal IM_PDU, Primary Ka Comm Port 3 3 3 30 48 AIA Port 3 Science Ka-Band, Normal IM_PDU, Primary Ka Comm Port 4 4 4 12 18 HMI Port 3 Science Ka-Band, Normal IM_PDU, Primary Ka Comm Port 5 5 5 22 34 EVE Port 3 Science Ka-Band, Normal IM_PDU, Primary Ka Comm Port 6 6 6 32 50 AIA Port 2 Science Ka-Band, Normal IM_PDU, Redundant Ka Comm Port 1 9 9 11 17 HMI Port 2 Science Ka-Band, Normal IM_PDU, Redundant Ka Comm Port 2 0A 10 21 33 EVE Port 2 Science Ka-Band, Normal IM_PDU, Redundant Ka Comm Port 3 0B 11 31 49 AIA Port 4 Science Ka-Band, Normal IM_PDU, Redundant Ka Comm Port 4 0C 12 13 19 HMI Port 4 Science Ka-Band, Normal IM_PDU, Redundant Ka Comm Port 5 0D 13 23 35 EVE Port 4 Science Ka-Band, Normal IM_PDU, Redundant Ka Comm Port 6 0E 14 33 51 AIA Port 1 Science Ka-Band, Link Error IM_PDU, Primary Ka Comm Port 1 31 49 n/a n/a HMI Port 1 Science Ka-Band, Link Error IM_PDU, Primary Ka Comm Port 2 32 50 n/a n/a EVE Port 1 Science Ka-Band, Link Error IM_PDU, Primary Ka Comm Port 3 33 51 n/a n/a AIA Port 3 Science Ka-Band, Link Error IM_PDU, Primary Ka Comm Port 4 34 52 n/a n/a HMI Port 3 Science Ka-Band, Link Error IM_PDU, Primary Ka Comm Port 5 35 53 n/a n/a EVE Port 3 Science Ka-Band, Link Error IM_PDU, Primary Ka Comm Port 6 36 54 n/a n/a AIA Port 2 Science Ka-Band, Link Error IM_PDU, Redundant Ka Comm Port 1 39 57 n/a n/a HMI Port 2 Science Ka-Band, Link Error IM_PDU, Redundant Ka Comm Port 2 3A 58 n/a n/a EVE Port 2 Science Ka-Band, Link Error IM_PDU, Redundant Ka Comm Port 3 3B 59 n/a n/a AIA Port 4 Science Ka-Band, Link Error IM_PDU, Redundant Ka Comm Port 4 3C 60 n/a n/a HMI Port 4 Science Ka-Band, Link Error IM_PDU, Redundant Ka Comm Port 5 3D 61 n/a n/a EVE Port 4 Science Ka-Band, Link Error IM_PDU, Redundant Ka Comm Port 6 3E 62 n/a n/a Fill 3F 63 n/a n/a

See Reference #1 (SDO High Speed Bus (HSB) ICD) and Reference #2 (CCSDS Implementation Plan) for most up to date list of VC assignments.

Page 21: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

464-GS-ICD-0010 Revision A

As soon as a TLM file has been generated, the DDS stores it in its 30-day storage and does a file transfer “Push” to write it to the dds2soc/tlm directory. It is the responsibility of the receiving SOC to maintain this directory, and make sure it does not overflow its capacity by removing files that are no longer needed.

Figure 4-1 illustrates the structures of the SDO science Ka-Band telemetry stream. See Reference #1 (SDO High Speed Bus (HSB) ICD, 464-CDH-ICD-0012) for latest information.

Packet Identification

Version

(3)

Type

(1)

SecHdr Flag

(1)

Application ID

(11)

Packet Sequence Control

SegmentationFlag

(2)

Source Sequence Count

(14)

Secondary HeaderPacket Length

Pkt Length

(16)

Seconds

(32)

SubSeconds

(32)

Packet Data

Application Data

(n*8)

VCDU Insert Zone

IM_PDU_ID

(6)

IM_PDU Seq #

(42)

M_PDU Header

Spare

(5)

First Header Ptr

(11)

M_PDU Packet Zone

Source Packet Data1 or more packets

1768 Octets

CVCDU Primary Header

VersionAOS=1

(2)

VCDU IdentifierSpacecraft

ID

(8)

VC ID

(6)

VCDUCounter

(24)

ReplayFlagRT=0

(1)

Spare

(7)

VCDU

Data

1776 Octets

VCDU Trailer

VCDU CRC

(16)

Sync Marker

0x1ACFFC1D(32)

VCDU

1784 Octets(1784 x 8 = 14272)

CADU CADUCADUCADUCADU2044 Octets

CADU

Next Packet

Structures above this line are built by Instruments

Structures below this line are built by Ka-Comm Card

For Frames from the Ka-Comm Card this bit is always 0

VC ID reflects the Port number on the HSB Ka-Comm Card the Data was received on

This IM_PDU_ID can be used toidentify the source camera orInstrument

Used for pkts spanning VCDUs VCDU Counter remains unique for >10 years

Transferred on HSB

June 23, 2005 4-3

Figure 4-1. Science Data Packet, I-MPDU, and VCDU Structure

RS Check Symbols

Interleave Depth = 8256 Octets

Added by Ka-Comm Card

SDO Overview Ka-Band Science Telemetry CCSDS Data Structures

Added by Ka-Comm Card

Page 22: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

464-GS-ICD-0010 Revision A

June 23, 2005 4-4

4.1.1.1 TLM File Naming Convention The file naming convention for telemetry files is:

VCid_yyyy_ddd_hh_mm_ss_seq_mod_vers.tlm

Where:

id: 2 decimal characters, 01-62, VCID of the data contained in the file as specified in the primary header VCID field

yyyy: 4 decimal characters, 2000-9999, UTC converted Year of 1st complete packet in file

ddd: 3 decimal characters, 001-366, UTC converted day of year of 1st complete packet in file.

hh: 2 decimal characters, 00-23, UTC converted hour of day of 1st complete packet in file

mm: 2 decimal characters, 00-59, UTC converted minute of hour of 1st complete packet in file.

ss: 2 decimal characters, 00-59, UTC converted second of minute of 1st complete packet in file.

seq: 11 hex characters, 0x00000000000-0xFFFFFFFFFFF, First theoretical Insert Zone sequence number in the file. If an instrument resets, the current file will be padded and closed. The next file will begin at sequence #0.

mod: 5 hex characters, 0x00000-0xFFFFF theoretical number of VCDUs in the file.

vers: 2 decimal characters, 00-99, Monotonically increasing count of the number of times a file has been opened after close. All versions of a file will be archived. Initial value is 0.

.tlm: file extension for TLM file that contains only valid, RS error free VCDUs.

Example: VC02_2008_365_23_59_59_0123456789A_FFFFF_01.tlm

4.1.1.2 TLM File Format

Figure 4-2 illustrates the KA-band science telemetry format and its relationship to the TLM files.

Each file will contain a fixed number of 1788 Octet data Units. Each data unit will either be a VCDU or 1788 Octets of 0 fill. The number of data units in a file will be predefined and sized to equal approximately 1 minute’s worth of data. This modulus will be configurable by VCID, but will require FOT intervention to change. The process to change the modulus and will be described in the operational documentation. Each TLM file will have a fixed length and will contain a fixed number of 1788-octet data units. For VCDUs, the format will be as follows:

4 octet sync pattern (0x1ACFFC1D) 1784 octet VCDU Reed Solomon Symbols will be removed

For 0 fill, the format will be 1788 octets of binary 0’s.

The relative position within a file of a given VCDU will be based on the value of the 42 bit IM_PDU sequence number. VCDUs will be ordered sequentially in ascending order, (i.e. lowest sequence number

Page 23: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

464-GS-ICD-0010 Revision A

first). The position of missing or non-correctable VCDUs will be filled with 0s. The sequence number in the filename is the first theoretical IM_PDU sequence number of the first theoretical VCDU in the file.

Instrument resets may cause the 42 bit IM_PDU sequence number to jump backwards or to 0. If this occurs, (IM_PDU sequence number jumps backwards, secondary header time moves forwards), the currently open TLM file(s) will be closed and padded accordingly. New TLM files will be opened corresponding to the new IM_PDU sequence number.

Telemetry packet

M_PDU Header M_PDU Packet Zone (2 octets) (1768 octets)

Telemetry packet

Sync Marker VCDU=1ACFFC1D (4) (1784)

(1788 Octets)

TLM data file(Fixed number of frames)

Built by the instruments

Built by KA- Comm card

Sync VCDUSync VCDU

Sync VCDU

Sync VCDU0000 00 ÉÉÉÉ 00

0000 00 ÉÉÉÉ 00

Sync VCDUSync VCDU

Sync VCDU

VCDU Data VCDU RS Symbols Trailer (ID=8) (CRC) (1776 Octets) (2) (256)

VCDU Primary Header = 6 Octets

VersionAOS=1

(2)

VCDU Identifier

Spacecraft ID(8)

VC ID(6)

VCDUCounter

(24)

ReplayFlagRT=0

(1)

Spare(7)

Provided by InstrumentSee table 4-2

Provided by KA- comm cardReflects HSB port number

Insert Zone (6 Octets)IM_PDU

ID(6 bits)

IM_PDUCounter(42 bits)

Delivered to SOC

Figure 4-2. TLM File Format.

June 23, 2005 4-5

Page 24: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

464-GS-ICD-0010 Revision A

June 23, 2005 4-6

4.1.2 ERR File Naming Convention The ERR filename is the same as the associated TLM file, with extension .err instead of .tlm.

VCid_yyyy_ddd_hh_mm_ss_seq_mod_version.err

Where VCid_yyyy_ddd_hh_mm_ss_seq_mod_version is as described in Section 4.1.1.1

.err is the file extension.

Example: VC02_2008_365_23_59_59_0123456789A_224E8_00.err contains all ERR VCDUs associated with VC 02 (as defined in Table 4-2) received while VC02_2008_365_23_59_59_0123456789A_224E8_00.tlm was open.

If VCDUs defined as ERR are received while no associated TLM file is open, an ERR file will be opened with the above naming convention, but the time in the file name will be the ground receipt time of the first VCDU received. The sequence number will be that of the last TLM file opened. Such a file will remain open for 1 minute.

4.1.2.1 ERR File Format DDS will create an ERR file only when it receives VCDUs that have been identified by the spacecraft as being in error. When the high-speed bus receives an IM_PDU that was either too long, too short or had link errors, it creates VCDUs with specific VCIDs defined for that purpose (see Table 4-2). Each TLM VCID has an ERR VCID associated with it. ERR files will be variable in length and will contain all ERR VCDUs associated with a given TLM VCID in the order in which they were received. Duplicates VCDUs will not be removed. Given nominal operations, there will be 2 copies of each VCDU in the file. Each data unit in the file will be 1788 octet:

4 octet Sync pattern (0x1ACFFC1D) 1784 octet VCDU Reed Solomon Symbols will be removed

Nominally, the ERR files will cover the same time frame as the similarly named TLM file. If no TLM file is open, see section 4.1.2.

The delivery of ERR files is not automatic: DDS generates the ERR files and lists them in the DSF file but does not automatically attempt to transmit them to the SOCs. The SOCs may request the ERR files listed in the DSF and the ERR files will then be treated as retransmissions requests. ERR files must be acknowledged via ASF if no retransmission request is made. If an ERR file is requested for retransmission, it must be acknowledged by both ASF and ARC.

ERR files will nominally be retained for 30 days, however they may be deleted more frequently if disk utilization becomes critical.

.

4.1.3 Quality and Accounting (QAC) Files This ASCII text file accompanies each TLM science telemetry file and provides quality information about the contents of the TLM file.

DDS will store the QAC files in parallel with the TLM files (30 days nominal). After 30 day, QAC files will be moved to a permanent archive and stored for the life of the mission. Requests for these files will be a manual process to be coordinated with FOT personnel.

DDS does a file transfer “Push” to write the QAC file to the dds2soc/tlm directory. It is the responsibility of the receiving SOC to maintain this directory and remove files that are no longer needed. If the directory gets full, DDS will fail delivering of subsequent files.

Page 25: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

464-GS-ICD-0010 Revision A

June 23, 2005 4-7

4.1.3.1 QAC File Naming Convention The QAC filename is the same as the associated TLM file, with extension .qac instead of .tlm.

VCid_yyyy_ddd_hh_mm_ss_seq_mod_version.qac

Where VCid_yyyy_ddd_hh_mm_ss_seq_mod_version is as described in Section 4.1.1.1

.qac is the file extension.

Example: VC02_2008_365_23_59_59_0123456789A_224E8_00.qac describes VC02_2008_365_23_59_59_0123456789A_224E8_00.tlm

Page 26: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

464-GS-ICD-0010 Revision A

June 23, 2005 4-8

4.1.3.2 QAC File Format File contents are ASCII text in the form “keyword=value” with no white space. All variables are fixed length fields with leading zero padding as necessary. All arguments will be followed by a line termination character (LF, line feed or new line, ASCII 0x0A).

Table 4- 3 shows the key words to be used in the QAC file.

Table 4- 3. QAC File Key Words

Keyword Description Argument3

TLM_FILE_NAME= Name of corresponding TLM file ASCII text, 47 characters, new line

TLM_FILE_SIZE= Size in bytes of associated .tlm file. Unless mod is changed, this number should be constant for a given VCID

ASCII text, 9 decimal characters, new line

QAC_FILE_SIZE= Size in bytes of this .qac file ASCII text, 8 decimal characters, new line

ERR_FILE_SIZE= Size in bytes of associated .err file. If 0, no .err file exists

ASCII text, 9 decimal characters, new line

TOTAL_TLM_VCDU= Total number of valid VCDUs in TLM file (Not theoretical)

ASCII text, 9 decimal characters, new line

TOTAL_MISSING_VCDU= Total number of missing VCDUs in TLM file. This number is based on gaps in the 24 bit VCDU sequence number

ASCII text, 9 decimal characters, new line

TOTAL_MISSING_IM_PDU= Total number of missing IM_PDUs in TLM file. This number is based on gaps in the 42 bit IM_PDU sequence number.

ASCII text, 9 decimal characters, new line

TOTAL_ERROR_VCDU= Total number of ERR VCDUs in ERR file.

ASCII text, 9 decimal characters, new line

TOTAL_GAPS= Total gaps in file. This number is based on gaps in the IM_PDU sequence number.

ASCII text, 9 decimal characters, new line

FIRST_IM_PDU_SEQ= 42 bit IM_PDU Sequence number of first VCDU in TLM file (Not theoretical)

ASCII text, 11 hexadecimal characters, new line

FIRST_VCDU_SEQ= 24 bit VCDU Sequence number of first VCDU in TLM file (Not theoretical)

ASCII text, 6 hexadecimal characters, new line

FIRST_IM_PDU_TIME= Converted UTC Time of first packet in this file (from packet secondary header) in format yyyy_ddd_hh_mm_ss.sss

ASCII text, 21 decimal characters, new line

LAST_IM_PDU_SEQ= 42 bit IM_PDU Sequence number of last VCDU in TLM file (Not theoretical)

ASCII text, 11 hexadecimal characters, new line.

LAST_VCDU_SEQ= 24 bit VCDU Sequence number of last VCDU in TLM file (Not theoretical)

ASCII text, 6 hexadecimal characters, new line

LAST_IM_PDU_TIME= Time of last packet in this file ASCII text, 21 decimal characters, new line

GAP_START_SEQ(1,2)= 42 bit IM_PDU Sequence number of last VCDU before gap

ASCII text, 11 hexadecimal characters, new line

GAP_START_TIME(1,2) = Converted UTC Time of last packet ASCII text, 21 decimal

Page 27: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

464-GS-ICD-0010 Revision A

June 23, 2005 4-9

before gap (from packet secondary header) in format yyyy_ddd_hh_mm_ss.sss

characters, new line

DISCONTINUITY= Flag to indicate a discontinuity occurred in the 42 bit IM_PDU counter, but not the 24 bit VCDU counter. Argument = VC Seq gap – IM_PDU_SEQ gap

ASCII text, 9 decimal characters, new line

VCDU_ERROR_CNT= Number of error VCDUs received during gap

ASCII text, 9 decimal characters, new line

GAP_STOP_SEQ(1)= 42 bit IM_PDU Sequence number of first VCDU after gap.

ASCII text, 11, hexadecimal characters, new line

GAP_STOP_TIME(1)= Converted UTC Time of first packet after gap (from packet secondary header) in format yyyy_ddd_hh_mm_ss.sss

ASCII text, 21 characters, decimal digits, new line

EOF_MARKER= Constant and recognizable ASCII string. C5C5

ASCII text, 4 hexadecimal characters, new line

(1) Repeat pair as necessary for each gap in the VCDU sequence number

(2) If data is missing that spans files, the first GAP_START_SEQ and GAP_START_TIME will be from the last VCDU received before the start of the gap.

(3) All arguments will be fixed length fields with leading 0 padding as necessary.

Example:

TLM_FILE_NAME=VC03_2005_116_11_59_31_0000bdf129f_072b2_00.tlm TLM_FILE_SIZE=52495680 QAC_FILE_SIZE=00000777 ERR_FILE_SIZE=000000000 TOTAL_TLM_VCDU=000029360 TOTAL_MISSING_VCDU=000000002 TOTAL_ERROR_VCDU=000000000 TOTAL_GAPS=000000002 FIRST_IM_PDU_SEQ=0000bdf129f FIRST_VCDU_SEQ=df129f FIRST_IM_PDU_TIME=2005_116_11_59_31.000 LAST_IM_PDU_SEQ=0000bdf2f4b LAST_VCDU_SEQ=df2f4b LAST_IM_PDU_TIME=2005_116_12_00_31.000 GAP_START_SEQ=0000bdf16f0 GAP_START_TIME=2005_116_11_59_32.000 DISCONTINUITY=000000000 VCDU_ERROR_CNT=00000000 GAP_STOP_SEQ=0000bdf16f2 GAP_STOP_TIME=2005_116_11_59_32.004 GAP_START_SEQ=0000bdf1ada GAP_START_TIME=2005_116_11_59_34.000 DISCONTINUITY=000000000 VCDU_ERROR_CNT=00000000 GAP_STOP_SEQ=0000bdf1adc GAP_STOP_TIME=2005_116_11_59_34.004

Page 28: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

464-GS-ICD-0010 Revision A

June 23, 2005 4-10

EOF_MARKER=C5C5

4.1.4 Delivery Status File (DSF) DDS maintains a catalog of all the data files and their transmission status, and once per hour it sends a DSF to each SOC. The DSF lists all the files for a given SOC/Instrument that exists within DDS that are closed and available for delivery and have not yet been acknowledged by the SOC. File listed in the DSF will be in one of 4 categories:

a. Delivery has been attempted (successful or not)

b. Retransmission has been attempted (successful or not)

c. Reprocessed files

d. Backlogged files (files more than 5 minutes old, no delivery attempted)

Files will remain listed in the DSF until they have been acknowledged in an Acknowledgement Status File (ASF) or Archival Status File (ARC) as successfully received by the SOCs. Files requested for retransmission will be removed from the DSF while queued for delivery and put back into the DSF after the retransmission attempt has been made.

DDS does a file transfer “Push” to write the DSF file to the dds2soc/dsf directory every hour on the hour. As with TLM and QAC files, it is the responsibility of the SOCs to maintain this directory.

DDS stores the DSF files for 30 days.

4.1.4.1 DSF File Naming Convention The delivery status file name is as follows:

soc_yyyy_ddd_hh_mm.dsf

where:

soc is EVE, HMI or AIA, representing the receiving SOC

yyyy_ddd_hh_mm: is the time this file was created

yyyy: (2008-9999), 4 decimal digits, UTC year

ddd: (001-366) 3 decimal digits, UTC day of year beginning at 001

hh: (00-23) 2 decimal digits, UTC hour of day

mm: (00-59) 2 decimal digits, UTC minute of hour

.dsf is the file extension

Example: EVE_2007_222_05_23.dsf

4.1.4.2 DSF File Format File contents are ASCII text in the form “keyword=value”

The group of keywords FILE_NAME and STATUS shown in Table 4-4 will be repeated for each entry in the DSF file with no white space. All status values will be one decimal character followed by a line termination character (new line, ASCII 0x0a).The last keyword in the file will be EOF_MARKER= and only will appear once.

Table 4- 4. DSF File Keywords

Keyword Description FILE_NAME= Name of TLM file (Transmitted and Not acknowledged by SOC)

Page 29: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

464-GS-ICD-0010 Revision A

June 23, 2005 4-11

STATUS= Status of TLM file, Allowable values defined in Table 4- 5EOF_MARKER= Constant and recognizable ASCII string. C5C5 (ASCII Hex)

Table 4- 5. Status Values

Code Status Description 1 Active File available, not acknowledged by SOC (See section 4.1.4 for

full description). 2 Expunged Removed from active list, Not acknowledged, Only issued once

4.1.5 Acknowledgement status File (ASF) The SOC will generate an ASF file in answer to the hourly DSF file. It is identical in format to the DSF file but the valid values for the status field are limited to allow the SOC to either acknowledge reception of a file or request its retransmission. When a retransmission of a file is requested by the ASF, that file name will be removed from the DSF until the retransmission has been attempted. Files that have been acknowledged by the ASF will also be removed from subsequent DSFs.

DDS does a file transfer “Pull” to get the ASF file from the soc2dds/asf directory every hour on the half-hour. After the file is retrieved, the DDS will remove the contents of soc2dds/asf.

4.1.5.1 ASF File Naming Convention The name of the ASF file is the same as the name of the DSF file it answers.

SOC_yyyy_ddd_hh_mm.asf

where:

SOC: name of SOC, HMI, EVE, AIA

yyyy_ddd_hh_mm: is the time the DSF file was created. yyyy: (2008-9999), 4 decimal digits, UTC year

ddd: (001-366)3 decimal digits, UTC day of year beginning at 001

hh: (00-23) 2 decimal digits, UTC hour of day

mm: (00-59) 2 decimal digits, UTC minute of hour

.asf is the file extension

Example: EVE_2007_222_05_23_30.asf

4.1.5.2 ASF File Format File contents are ASCII text in the form “keyword=value”.

The group of keywords FILE_NAME and STATUS shown in Table 4- 6 will be repeated for each entry in the DSF file with no white space. All status values will be one decimal character followed by a line termination character (new line, ASCII 0x0a).The last keyword in the file will be EOF_MARKER= and only appear once.

Table 4- 6. ASF File Keywords

Keyword Description FILE_NAME= Name of a TLM or ERR file, present in the DSF file. STATUS= Status of TLM or ERR file, Allowable values are described in Table 4-

7EOF_MARKER= Constant and recognizable ASCII string. C5C5 (ASCII Hex)

Page 30: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

464-GS-ICD-0010 Revision A

June 23, 2005 4-12

Table 4- 7. ASF Status Values

Code Status Description 2 Retransmit SOC requested retransmit 3 Acknowledge SOC acknowledges receipt of this TLM file.

4.1.6 Archive Status File (ARC) Each SOC generates a daily archive acknowledgement file (ARC). The ARC file contains a list of all the files that the SOC has successfully archived during that day. Files listed in the ARC file will be removed from subsequent DSFs even if no ASF acknowledgement has been received. DDS does a file transfer “Pull” to get the ARC file from the soc2dds/arc directory every day at 00:00:15 UTC.

After the file is retrieved, the DDS will remove the contents of soc2dds/arc.

4.1.6.1 ARC File Naming Convention soc_yyyy_ddd.arc

where

• soc is EVE, HMI, or AIA which sends the ACK file

• yyyy_ddd is the UTC year and day of year to which this file applies.

• .arc is the file extension

4.1.6.2 ARC File Format The keyword FILE_NAME shown in Table 4- 8 will be repeated for each entry in the ARC file with no white space. All file name values will be followed by a line termination character (new line, ASCII 0x0a).The last keyword in the file will be EOF_MARKER= and only appear once.

Table 4- 8. ARC Keywords

Keyword Description FILE_NAME= Name of a TLM file. Or ERR File EOF_MARKER= Constant and recognizable ASCII string. C5C5 (ASCII Hex)

Page 31: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

464-GS-ICD-0010 Revision A

June 23, 2005 5-1

5.0 COMMUNICATIONS PROTOCOLS

5.1 DELIVERY ADDRESS The DDS will deliver files to a predefined (configurable) Host and Directory for each SOC. Real-time and retransmission deliveries may be sent to different locations. The delivery parameters will be kept in a database in the DDS and may be periodically changed, however it is not envisioned that this will be a real-time or automated process. Delivery parameter modifications will need to be coordinated with MOC personnel via phone or e-mail. Detailed procedures for this will be documented in the SDO MOC to SOC Operations Agreements.

5.2 FILE TRANSFER PROTOCOLS All TLM, ERR, and QAC, files will be transferred using Secure Copy version 2. ASF, DSF, and ARC files will be transferred using secure File Transfer Protocol. Both use SSH as the underlying protocol. The procedures for the exchange of encryption keys will be documented in the SDO MOC to SOC Operations Agreements.

5.3 DIRECTORY STRUCTURES When logging into the SOC machines, the DDS will expect to see a directory structure like that shown in Table 5- 1. The directory structure will be configurable. The tlm, err and retrans directories may all be the same directory. All directory names will be lower case. It is the responsibility of the SOC to maintain the tlm, err, retrans and dsf directories (if they exist) by removing old files as necessary. DDS will maintain the asf and arc directories by removing old files as necessary.

Table 5- 1. SOC Directory Structure

Directory Location Description dds2soc/tlm SOC DDS puts all TLM, and QAC files here.

dds2soc/err SOC DDS puts all ERR files here

dds2soc/retrans SOC DDS puts all retransmission TLM, QAC and ERR

files here (If requested by SOC).

dds2soc/dsf SOC DDS puts DSF file here.

soc2dds/asf SOC SOC puts ASF file here.

soc2dds/arc SOC SOC puts ARC file here.

5.4 SECURITY MEASURES Figure 5-1 depicts a diagram of the SDO-DDS routed network. The routers between the DDS and SOCs will be running firewall services and execute very restrictive access control lists (ACL). Only data flows

Page 32: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

464-GS-ICD-0010 Revision A

required for delivery and accounting of the DDS science data, and ancillary flows from the MOC will be permitted into the interface at each SOC. This of course includes all the flows described in this ICD.

The intent of this posture is to allow the SOCs considerable flexibility in their internal network configuration and security plan. All SDO project facilities must comply with NASA Policy Requirements.

June 23, 2005 5-2

Figure 5-1 SDO network diagram

EVE(Boulder)

sdo-lasp-r18a sdo-lasp-r18b

sdo-lasp-sr19a sdo-lasp-sr19b

JSOC (Stanford)

sdo-stanford-r16a sdo-stanford-r16b

HMI AIA

sdo-stanford-sr17a sdo-stanford-sr17b

SOC SDP SegmentScience Data Processing

FibreChannel

(WSGT)DDS

sdo-wsgt-sr1b sdo-wsgt-sr1a

FVSFEH FEHFVS

QCP

QCPspare

QCPspare

QCP QCP

DSIMbackup DSIM

30-day archiveprivate SAN

metadata LAN

sdo-wsgt-fr4a sdo-wsgt-fr4b

VMspare

spare

FOP/RM

FOP/RMVM

FOP/RM

sdo-wsgt-sr3a sdo-wsgt-sr3b

OC3OC3T3

ScienceTelemetry

SDOGS

STGTWSGT

MOC

Page 33: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

464-GS-ICD-0010 Revision A

June 23, 2005 A-1

APPENDIX A. LIST OF ACRONYMS Abbreviation Definition ACK Acknowledgement ACL Access Control Lists AIA Atmospheric Imaging Assembly ARC Archive Acknowledgement file ASCII American Standard Code for Information ASF Acknowledgement Status File B/W Bandwidth CADU Channel access data unit CCB Configuration Control Board CCSDS Consultative Committee for Space Data Standards CFDP CCSDS File Delivery Protocol CMO Configuration Management Office CRC Cyclic redundancy check DDS Data Distribution System DMR Detailed Mission Requirements DSF Delivery Status File DSIM DDS SDOGS Integrated Manager EOF End Of File ERR Error EVE Extreme Ultraviolet Variability Experiment FEP Front-end Processor FIFO First In First Out GHz Giga Hertz 1X109 Hz GS Ground System HMI Helioseismic and Vector Magnetic Imager HSB High Speed Bus Hz Hertz (Cycles Per Second) ICD Interface Control Document IM_PDU Instrument Multiplexing Protocol Data Unit IM_PDU_ID Instrument Multiplexing Protocol Data Unit ID IP Internet Protocol JSOC Joint Science Operations Center Ka 20 - 30 GHz Communications Band LASP Laboratory for Atmospheric and Space Physics LMSAL Lockheed Martin Solar & Astrophysics Laboratory MBps Mega Bytes Per Second Mbps Mega Bits Per Second Mbytes Mega Bytes MDP Multicast Dissemination Protocol MOC Mission Operations Center MPDU Multiplexing Protocol Data Unit MRD Mission Requirements Document NASA National Aeronautics and Space Administration NPR NASA Procedures and Requirements OC3 Optical Carrier 3 (3X51.84 MHz = 155.52 Mbps QAC Quality and Accounting

Page 34: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

464-GS-ICD-0010 Revision A

June 23, 2005 A-2

RS (R-S) Reed-Solomon RTC Real Time Critical SDO Solar Dynamics Observatory SDOGS SDO Ground Station SOC Science Operations Center STGT Second TDRS Ground Terminal T&C Telemetry and Command T1 1.544 Mbps data communications service T3 44.736 Mbps data communications service TBD To Be Defined Tbytes Tera Bytes (1X109 Bytes) TLM Telemetry UTC Universal Time Coordinated VCDU Virtual Channel Data Unit VCID Virtual Channel ID WSC White Sands Complex WSGT White Sands Ground Terminal

Page 35: Solar Dynamics Observatory (SDO) Interface Control Document (ICD) Between The SDO Data ...hmi.stanford.edu/NASA_documents/DOC_196_4_1_1.pdf · 2006-01-12 · Chad Salo Brett Sapper

464-GS-ICD-0010 Revision A

June 23, 2005 B-1

1.0 Appendix B – Configurable Parameters

The following items will be configurable by the SDO Operations staff:

Primary SOC file delivery host IP address

Additional SOC file delivery hosts IP addresses

Telemetry file modulus by VCID

predefined number of VCDUs in a given file, this number determines TLM and QAC delivery interval

Telemetry file creation timeout

Length of ime a TLM file may be open before forced closed

File delivery timeout

Length of time a file delivery wil be attempted before it is failed

File delivery latency timeout

How old a file can be and still be considered for real-time delivery

SOC file delivery host Telemetry directory

SOC file delivery host Error directory

SOC file delivery host retransmit directory

DSF/ASF delivery/retrieval interval

DSF/ASF delivery/retrieval time

Telemetry file retention time

How long TLM files reside on the DDS (30 days nominal)

Error file retention time

How long ERR files reside on the DDS (30 days nominal)

SOC email addresses for 10 and 25 day notifications