Download - ISUAL Data Formats & Science Data Processing S. Geller.

ISUAL Data Formats & Science Data Processing

S. Geller

PDR 31 August 2000 2NCKU UCB Tohoku Data Formats S. Geller

Astrium Specification on ISUAL T/M

VersionNumber

3 bits= 100

Type(TM or TC)

1 bit = 0

Data FieldHeader Flag

1 bit = 1

APID

11 bits

SegmentationFlag

2 bits = 11

SourceSequence

Count

14 bits

PACKET ID (16 bits)PACKET SEQUENCE

CONTROL (16 bits)

16 bits

PacketLength

PACKET HEADER (48 bits)

9 bytes

On Board (UTC) time

var. but fixed per APID

Source Data

PACKET DATA FIELD (var.)

Packettrailer

8 bits = AAh

Follows CCSDS Format:


ISUAL T/M Detailed Data Format

• Common format used for Science Data, SOH Data, and ISUAL communications with Spacecraft– Standard CCSDS Primary Header

• APID’s 700 to 74A are assigned to ISUAL

– Secondary Header Comprised of a 9 byte Time Code

– Variable number of data bytes• Total Packet Length limited to 256 bytes for SOH channel and 4096 bytes for

Science Data channel

– One byte of filler if needed to make total packet length even

– One byte of Check Sum

– One byte Trailer = hex AA

• Same packets can be sent over either SOH or Science Data channel provided length constraint is met

• UCB Drawing 8565-x7 lists all T/M data formats


Currently Defined T/M Packets

APID Packet Name 701 MBR 703 Command Echo 705 Ancillary Data 706 Housekeeping 709 Memory Dump 70B MM Bad Sections 70C Aurora Header 70D Aurora Data 70E Sprite Header 70F Sprite Data 710 Spectrophotometer Header 711 Spectrophotometer Data 712 SP Group-Average Header 713 SP Group-Average Data 714 Array Photometer Header 715 Array Photometer Data 716 AP Group-Average Header 717 AP Group-Average Data 7FF Idle Packet


Example Telemetry Data Source Packet (1)

ApID=701 MBR packet (Request to send Science Data)byte-offset element size no. elements

-6 1 1 100 0 1 111-5 1 1 Ap ID LSB's = 00000001-4 1 1 11xxxxxx = source seq cnt MSB's-3 1 1 Source Sequence Count LSB's-2 1 1 Packet Length MSB-1 1 1 Packet Length LSB0 2 1 year2 2 1 day of year Time4 1 1 hour of day When5 1 1 minute of hour Packet6 1 1 second of minute was7 1 1 1/256 second made8 1 1 1/256/2569 2 1 number of PVCFs

11 1 1 filler (to have even packet length)12 1 1 checksum13 1 1 Trailer = 0xAA

Total length of Packet Data Field in bytes: 14Total length of Packet (incl. Primary Header): 20


Example Telemetry Data Source Packet (2)

ApID=711 Spectrophometer Data packetbyte-offset element size no. elements

-6 1 1 100 0 1 111-5 1 1 Ap ID LSB's = 00010001-4 1 1 11xxxxxx = source seq cnt MSB's-3 1 1 Source Sequence Count LSB's-2 1 1 Packet Length MSB-1 1 1 Packet Length LSB0 9 1 Secondary Header (Time when exposure was started)9 2 1 data packet sequence number

11 1 1 number of valid bytes in this packet12 1 4076 Data Bytes (4076 maximum)

4088 1 1 checksum4089 1 1 Trailer = 0xAA

Total length of Packet Data Field in bytes: 4090 Total length of Packet (incl. Primary Header): 4096

max


Astrium Specification on ISUAL Commands

VersionNumber

3 bits= 000

Type(TM or TC)

1 bit = 1

Data FieldHeader Flag

1 bit = 1

APID

11 bits

SegmentationFlag

2 bits = 11

SourceSequence

Count

14 bits

PACKET ID (16 bits)PACKET SEQUENCE

CONTROL (16 bits)

16 bits

PacketLength

PACKET HEADER (48 bits)

var.

Application data

PACKET DATA FIELD (var.)

Packettrailer

8 bits= AAh

• Again CCSDS Format is Followed– Type bit now a 1

• Includes Secondary Header consisting of 9 byte Time Code


ISUAL Detailed Command Format

• Common format used for Ground Commands and On-Board Commands

– Standard CCSDS Primary Header• APID’s 700 to 70F are assigned to ISUAL

– No Secondary Header used

– Variable number of data bytes• Total Packet Length limited to 240 bytes

– One byte of filler if needed to make total packet length even

– One byte of Check Sum

– One byte Trailer = hex AA

• APID used to differentiate between on-board and ground commands

• Presence on check sum in on-board commands subject to negotiation with Astrium

• UCB Drawing 8564-x7 shows detailed structure of all commands


ISUAL Science Data Types

• Sprite Mode– Sprite Event: 8 or more Images

– Spectrophotometer: N Sample Sets

– Array Photometer: N Sample Sets

• Aurora Mode– Aurora/Airglow Image

– Spectrophotometer: MS average groups

– Array Photometer: MA average groups


Ancilliary Data

• Spacecraft position and attitude are received as TPA messages.

• Packets are generated with this information, interleaved with the science data.

• In ground processing, position and attitude will be averaged from two TPA messages bracketing the data time.


Data Compression

• Normally, all science data will be compressed.

• For diagnostic and perhaps other purposes, some uncompressed data will be sent.

• Data compression may be turned on and off by command; the default is on - compressed.


Science Data Flow

• Readout raw images and/or photometer samples.

• Compress the data arrays

• Generate CCSDS source packets

• Pack packets in to PVCFs

• Pack PVCFs into Spacecraft Data Block

• Send Data Block to spacecraft


R aw S c ien c eD ata

DCM

C C S D SP ac k ets

TL M G e n e ra to r

P VC F

S c ien c eD ata Blo c k

S pa c e c ra ftD a ta B uffe r

( u n c o m p r es s ed )

( c o m p r es s ed )

S cience Telem etry F low


Level-Zero Data

• This is the first level of ground processing.

• Extract CCSDS Source Packets

• Eliminate duplicates; put into ascending time order

• The result is “Level-zero” data: CCSDS source packets made available as a binary file or network socket data stream.


Corrupted Data

• If CCSDS header is corrupted, packet is lost.– This can cause loss of multi-packet compressed data.

• If part of data is corrupted, the “bad bytes” will be filled with zeroes. – Prompts use of internal checksum in ISUAL packets.


Stages of Data Generation

• Raw readout within ISUAL– into Mass Memory areas

– DCM compresses raw data, stores output in other Mass memory areas

• Data Packets– Uncompressed

– Compressed


Raw Camera Data

• A Camera image is read out as 128 rows of 512 pixels (may be commanded to be smaller)

• Each pixel is 16 bits, of which 12 bits are significant.

• Storage in memory is “big endian” -- high order first.


Sprite Image Arrays

• A single image is 128 * 512 * 2 = 131,072 bytes.

• A Sprite Event generates 8 or more images– Trigger - when the event happened (1)

– Before - taken before the event (usually 1)

– After - taken after the event (usually 6)


Aurora / Airglow Images

• One 131,072-byte image per exposure

• There is no Event trigger.

• Exposures are taken at regular intervals.


Raw Spectrophotometer Data

• 6 channels

• 12-bit sample values

• Sample values are stored in memory as 16-bit words (low 12 are significant) “big endian” - high order first

• Sample size is 6 * 2 = 12 bytes


Raw Array Photometer Data

• 16 “A” channels (short wavelength)

• 16 “B” channels (long wavelength)

• Samples are stored in memory as 16-bit words (low 12 are significant) “big endian” - high order first

• ample size is 32 * 2 = 64 bytes

Preliminary Plans for Processing ISUAL Telemetry


What the Flight Software does

• Reads out raw data

• Generates compressed data

• Generates CCSDS source packets– stamps them with time of generation (secondary header)

• Packs packets into PVCFs and sends them to the spacecraft.


What the Spacecraft does

• Receives data from ISUAL as “files” of PVCFs.

• Stores data in on-board bulk memory

• Encapsulates data as transfer frames, and sends it all to the ground.


What the Ground Software does

• Decodes CCSDS packets from the transfer frames.

• Sorts packets in ascending time order, using the time-stamp in the secondary header

• Eliminates duplicates

• Deals with corrupted data– if header corrupted, drops packet

– otherwise does zero fill


Level 0 Data

• CCSDS packets, in time order, duplicates removed.

• Except for possible dropped packets, these are the CCSDS packets that were generated by ISUAL, and in the same order.

• ISUAL should include a checksum in each packet, in case of corruption.


Raw Data Products

• The CCSDS packets are used to construct raw data products

• Raw Products are the original data, with CCSDS header and other telemetry artifacts removed.

• In raw products, compressed data is converted to its original uncompressed form.


Sprite Data Product

• Number of images

• Image arrays (512 cols, 128 rows)

• index of event image

• Time of Sprite Event

• number of post-trigger images

• Exposure duration

• milliseconds after event before 1st exposure


Sprite Data Product (cont.)

• Which was triggering photometer (1..6)

• Trigger level

• MCP HV; Phosphor HV

• Filter code

• First TPA; Second TPA


Aurora Data Product

• Time of start of exposure

• duration of exposure (millisecs)

• MCP HV; Phosphor HV

• Filter code


• One image array


Spectrophotometer Data Product

• Total number of samples

• Index number of time-stamped sample

• Time-stamp

• Sampling rate (millisec/sample)

• HV setting


• Up to N groups of 6 photometer values


Spectrophotometer Averages Product

• Total number of groups

• Index number of time-stamped group

• Time-stamp

• Number of samples averaged per group


• HV setting


• Up to N groups of 6 averages


Array Photometer Data Product

• Total number of samples

• Index number of time-stamped sample

• Time-stamp

• Sampling rate (millisecs/sample)

• HV setting


• Sample Array (N x 32)


Array Photometer Averages Product

• Total number of groups

• Index number of time-stamped group

• Time-stamp

• Number of samples/group


• HV setting


• Array of averages (N x 32)