Lossless Data Compression as a Spacecraft Service

Mark Reid, JHU/APL Eric McGinnis, University of Delaware

Lossless Data Compression as a Spacecraft Service 2 20 October 2011

Agenda

 Parties Involved in this Work  Problem Statement  Work Performed and Options Explored  Results  Implications and Future Applications

Parties Involved in this Work

 This work was performed as a summer intern project by Eric McGinnis from the University of Delaware

 I mentored Eric and provided ideas and guidance for the work he performed

Lossless Data Compression as a Spacecraft Service 3 20 October 2011

PROBLEM STATEMENT:

4

Efficient Data Storage and Transmission

Packetized Data Sent from Instruments and Subsystems on Software Bus

Packets Recorded to Solid State Recorder (SSR)

Satellite Downlinks Data How can we improve

this process?

Lossless Data Compression as a Spacecraft Service 20 October 2011

Compression Options Considered

Lossless Data Compression as a Spacecraft Service 5

Background File vs. Real-time Packet Compression

20 October 2011

Background File Compression

  Read files of data from SSR, compress the files and put them back

  Similar to compression performed on previous APL missions (MESSENGER, New Horizons)

  Only supports compression of recorded file data

  Does not work well with CFDP class 1 file transactions (lost frames may make file unrecoverable)

Real-time Packet Compression

  Compresses “user selectable” streams of packets

  Supports compression of both recorded file data and real-time data (housekeeping, dumps, etc.)

  Ideal for operations with CFDP class 1 file transactions (lost frames only cause the loss of a single compressed packet)

Compression Algorithms Considered

Lossless Data Compression as a Spacecraft Service 6

LZMA vs. Zlib (ZIP)

20 October 2011

LZMA

  Lempel-Ziv-Markov chain Algorithm (LZMA)

  Uses a variant of LZ77 (Lempel-Ziv 1977) with the output encoded by a range encoder

  Public Domain version created by Igor Pavlov, 2008

  Used in 7-zip file archiver   Decoder currently used on

RBSP to decompress FSW applications at boot time

Zlib

  Written by Jean-Loup Gailly and Mark Adler

  Uses DEFLATE algorithm which uses a combination of LZ77 and Huffman coding

  Used in gzip file compression program

  Provides control of processor and memory use

  Supports tunable compression levels (0-9)

Compression Performance

Lossless Data Compression as a Spacecraft Service 7

-

50,000,000

100,000,000

150,000,000

200,000,000

250,000,000

300,000,000

350,000,000

Original Data Volume

zLib Packet (level 6)

LZMA Packet zLib File (level 6)

LZMA File

Bytes 336,168,174 217,338,971 211,295,647 173,201,656 144,386,878

Using RBSP Mission Simulation #2 Data

20 October 2011

8

0%

10%

20%

30%

40%

50%

60%

Packet File

Z L I B

Z L I B

L Z MA

L Z MA

Dat

a R

educ

tion 35%

48%

37%

57%

Lossless Data Compression as a Spacecraft Service 20 October 2011

Compression Performance Same data viewed as % data reduction

RECOMMENDATION:

9

Real-time Packet-Level Compression with zLib

Packetized Data Sent on Software Bus

Satellite Downlinks Data

Larger, uncompressed

packets

Smaller, compressed

packets ZIP (Real-time Data Compression using zLib)

Lossless Data Compression as a Spacecraft Service 20 October 2011

Real-time downlink stream

Playback stream

Packets Recorded to Solid State Recorder (SSR)

10

• Packet Compression • Compression of both real-time and recorded data • More compatible with CFDP class 1 (Unacknowledged) operations

• zLib Characteristics • zLib is fast • Low Memory Footprint • Stable, open source, tested • Highly Flexible • Good compression ratio

Why Choose Packet Compression & zLib?

Lossless Data Compression as a Spacecraft Service 20 October 2011

11

• Highly portable, useful on almost any mission running cFE

• Library allows other applications to compress or deflate other data (macros, parameter tables, etc.)

• Low priority, compresses what it can in otherwise unused CPU cycles

• To compress all RBSP data (~75 Kbps) ≈ 15% CPU (RAD750 @ 33 MHz) < 265KB RAM

ZIP (Packet

Compression Application)

Lossless Data Compression as a Spacecraft Service 20 October 2011

Implemented as a cFE Library and Application The zLib Library and ZIP Application

zLib (Data

Compression Library)

CFE_SB_RcvMsg CFE_SB_SendMsg

ZLIB_Compress

Possible Applications

12

• Amount of data returned is determined by downlink rate and contact schedule • Maintaining the same contact schedule and downlink rates the satellite can collect more data

• Average downlink: 1 mbps • Effective compressed downlink: 1.546 mbps • Sample mission cost: $500 Million • Value of additional data: $273 Million

Data return before compression

Data return after compression

Lossless Data Compression as a Spacecraft Service 20 October 2011

Increased data return

Possible Applications

13

Decreased Operating Costs

• Amount of data returned is determined by amount of data collected

• If downlink frequency is reduced, costs are reduced • 2500hrs downlink @ $1250/hr: $3.125 million • Downlink cost with compression: $2.031 million • Savings: $1.094 million • Mission Extensions More Likely

Downlink cost after compression

Downlink cost before compression

Lossless Data Compression as a Spacecraft Service 20 October 2011

Possible Applications

14

Freedom in Design

• Data return dictates observatory design

• Power requirements for RF System • Non-volatile Memory requirements for Avionics • Solid State Recorder (SSR) memory requirements

• More space/mass/power/money for instruments

Lossless Data Compression as a Spacecraft Service 20 October 2011

Flight Software Architectural Modeling with SAVE 15 06 November 2007

Questions?