Lossless Layered Residual System

1Gary Demos Lossless Residual System, 24Feb2006

Lossless Layered Residual System

Gary Demos

HPA

24Feb2006

Palm Springs, CA


Features

• Arbitrary Precision

• Direct Support of Integers of Arbitrary Precision (8bits up to 16 bits tested)

• Floating via Direct Lossless Layered Support of OpenExr 16-bit Half-Float

• Wide Dynamic Range Half-Float

• Integrated With Floating-Point Layered Codec

(as presented Oct 2004 SMPTE Pasadena)


Use of Bins

• Uses Bins Like the OpenExr 16-bit Half-Float “Piz” Compressor

• Bin List Is Sorted, and Contains All Pixel Values From Original Image

• Bins Can Contain Integers or Floating-Point of Arbitrary Precision

• Floating Point Values Can Have Any Arbitrary Dynamic Range

• Negative Values Are Fully Supported

• Currently Optimized For Integers from 8 to 16 bits, and 16-bit Half-Float

• Bins Are Referenced Solely By Integer Index (“Bin Number”)

• Unused Codes In Wide Range Representations (e.g. 16-bit Half-Float) Automatically Eliminated From Bin List


Lossy CodecFloating Value

Bin List

Nearest Bin

Desired (Lossless) BinIntegerBinDelta


Floating Lossy ValueOn A Cusp(nearly equidistant)

Floating PointOriginal Values

If on a Cusp, Flag This Pixel’s PrimaryIndicate Whether to Go Up or DownorCode The Final Bin Directly

Only a Small Proportion of Pixels Will Be on a Cusp

Solely a Function of Lossy Floating-Point Precision (e.g. 32-bit Floating)

Necessary Because of Processor and Compiler Inconsistencies (e.g. P4 vs G5)


Noise-Reduced-Driven-Intermediate

• How To Find A Useful Intermediate Bin On the Way To Lossless?

• A Proportion Of the Distance

• And / Or

• Driven By A Noise-Reduced Original Image

• Constrain To Range Between Lossy Nearest Bin and Lossless Bin


Lossy CodecFloating Value

Bin List

Nearest Bin

Desired (Lossless) BinTwoBinDeltas

Noise-ReducedOriginal Intermediate Bin


Noise-ReducedOriginal Lossy Codec

Bin-DeltaIntermediate(s)

Bin-DeltaLossless

(bit exact)

Floating Point

Floating Point

Integeror

Floating Point


Bin List Subset(s)

• Reduce Intermediate Bin Deltas By Shifting or Scaling

• And / Or

• Create a Subset of the Bin List From The Original

• More Than One Subset Can Support More Than One Intermediate

• Subsets Can Be Matched To Noise Floor

• Or Can Take Ever-Deeper “Bites” Out of the Noise Floor

• Values Can Be Added To Subset (If Not Present In Original) For Useful Step(s)


Bin Subset Value Why Included or Not in Subset, or Why Added0 Yes/1 -.00038 Useful for averaging black at 0.01 Yes/1 -.0002 Useful for averaging black at 0.02 Yes/1 0.0 Useful for representing extreme black3 No/0 .000001 Value below both noise floor and extreme black4 No/0 .0000024 Value below both noise floor and extreme black5 No/0 .000013 Value below both noise floor and extreme black6 Yes/1 .0001 Useful as 10,000:1 dynamic range floor7 No/0 .000101 Below noise floor at this brightness level8 Yes/1 .0002 Useful stepA0 Only .0003 Useful step, not present in original9 Yes/1 .0004 Useful step10 No/0 .00041 Below noise floor at this brightness levelA1 Only .00055 Useful step, not present in originalA2 Only .0007 Useful step, not present in original11 Yes/1 .0009 Useful step12 Yes/1 .0012 Useful stepA3 Only .0014 Useful step, not present in original13 Yes/1 .0017 Useful step14 No/0 .0018 Below noise floor at this brightness level15 Yes/1 .0019 Useful stepetc. etc. etc. etc.… … … …921 Yes/1 0.994 Useful step922 Yes/1 1.0001 Useful step (logical default white level at 1.0)923 Yes/1 1.07 Useful step (values above reference white

preserved up to highest value present in original)924 No/0 1.071 Below noise floor at this brightness level925 Yes/1 1.094 Useful stepA134 Only 1.13 Useful step, not present in original926 Yes/1 1.18 Useful step927 Yes/1 1.213 Useful step928 No/0 1.217 Below noise floor at this brightness level929 Yes/1 1.241 Useful step (highest value present in original)

Floating Point Bin List Subset For Intermediate(s):


Integer Bin List Subset For Intermediate(s):

Bin Subset Value Why Included or Not in Subset, or Why Added0 Yes/1 76 Lowest value used in original (Dmin)1 No/0 77 Below noise floor at this density level2 No/0 78 Below noise floor at this density level3 Yes/1 79 Useful value slightly below noise floor4 No/0 81 Below noise floor at this density level5 Yes/1 82 Useful value slightly below noise flooretc. etc. etc. etc.… … … …517 Yes/1 756 Useful value slightly below noise floor518 Yes/1 759 Useful value slightly below noise floorA0 Only 762 Useful step, not present in original519 Yes/1 764 Useful value slightly below noise floor520 Yes/1 768 Useful value at noise floor521 No/0 769 Below noise floor at this density level522 No/0 770 Below noise floor at this density level523 Yes/1 771 Useful value at noise floorA1 Only 775 Useful step, not present in originalA2 Only 779 Useful step, not present in originalA3 Only 783 Useful step, not present in original523 Yes/1 786 Useful value524 Yes/1 790 Useful value at noise floor525 Yes/1 793 Useful value (highest value in original, Dmax)


Results

• Stem MMR4 4096x1714 12-bit RGB packed within 16-bit Tiff16 (Film)

• Patuxent River Air Ops 1920x1080 @30fps 10bit Viper Filmstream to 16-bit Half-Float

• Garden Music 1280x720 @59.94fps 10-bit Viper Filmstream to 8-bit DPX (8bit in 10bit)

• Sphere 4096 x 2160 16-bit RGB as Tiff16 (noiseless CGI)


Layer of STEM MMR4 bitrate (24fps) compression ratio1024 x 429 Layer(motion compensated)

39.18 mbps 9.68 : 1 vs. 1024 x 429 at 12-bits

2048 x 857 Layer 61.99 mbps 24.46 : 1 vs. 2048 x 857 at 12bits2048 x 857 Layer + low-band SNR (at 1k)

75.00 mbps 20.22 : 1 vs. 2048 x 857 at 12bits

4096 x 1714 Layer 75.88 mbps 79.94 : 1 vs. 4096 x 1714 12bits4096 x 1714 Layer + 2klow-band SNR (at 1k)

88.89 mbps 68.24 : 1 vs. 4096 x 1714 12bits

4096 x 1714 Layer + 2klow-band SNR (at 1k) + 4klow-band SNR (at 2k)

110.88 mbps 54.71 : 1 vs. 4096 x 1714 12bits

4096 x 1714 Layer + low-band SNR’s + all-bandwavelet SNR

119.44 mbps 50.79 : 1 vs. 4096 x 1714 12bits

4096 x 1714 Layer above +all-band wavelet SNR withzero deadband

130.59 mbps 46.45 : 1 vs. 4096 x 1714 12bits

4096 x 1714 Layer above +dark-targeted all-bandwavelet SNR

175.68 mbps 34.53 : 1 vs. 4096 x 1714 12bits

4096 x 1714 Layer above +direct delta SNR

179.36 mbps 33.82 : 1 vs. 4096 x 1714 12bits

4096 x 1714 Layer above +dark-targeted direct deltaSNR

227.24 mbps 26.69 : 1 vs. 4096 x 1714 12bits

4096 x 1714 Layer above +noise-reduced-driven bin-delta intermediate

2.049 gbps 2.96 : 1 vs. 4096 x 1714 12bits

4096 x 1714 Layer above +lossless bin-delta (this isbit-exact lossless coding ofthe original)

4.460 gbps 1.36 : 1 vs. 4096 x 1714 12bits

Original, 12-bits RGB(packed in Tiff16)

6.066 gbps 1.0 vs. 4096 x 1714 original at12-bits

Original, 16-bits RGB Tiff16 8.088 gbps 0.75 vs. 4096 x 1714 original at12-bits

Stem MMR4:


10-bit RGB source OpenExr 16-bit gzip of OpenExr 16PaxRiver 150-4477 1.0 1.8748 vs. 10bit 1.9506 vs. 10bit

PaxRiver performance of OpenExr “piz” and gzip of “piz”:


PaxRiver 150-4477 mbps compression ratiovs. 10-bit

compression ratiovs. 16-bit

960x540 layer(motion comp layer)

23.52 mbps 19.83 : 1 vs. 10-bit(960 x 540)

31.73 vs. 16-bit(960 x 540)

1920x1080 layer 42.42 mbps 43.99 : 1 vs. 10-bit 70.38 vs. 16-bit1920x1080 layer +low-low SNR band

57.07 mbps 31.77 : 1 vs. 10-bit 50.83 vs. 16-bit

1920x1080 layer +low-low SNR band +lossless residual

759.7 mbps 2.46 : 1 vs. 10-bit 3.94 vs. 16-bit

PaxRiver 1920 x 1080 10-bit Filmstream color adjusted using conversion to linearwith rgb scale factors into OpenExr 16-bit half-float:


[email protected] 0-4799

8-bit RGB source(in 10-bit dpx file)

gzip(of 10bit dpx file)

lossless

Garden Music 1.0(for 8-bit source)

1.53 : 1(vs. 8-bit original)

2.88 : 1(vs. 8-bit original)

Garden Music (8-bit) Comparison with gzip:


Garden [email protected] 0-4799

bit-rate compression ratio

[email protected] layer(motion comp layer)

24.17mbps 14.7 : 1 (vs. 640x360 8-bit RGB)

[email protected] layer

51.64mbps 27.45 : 1 (vs. 1280x720 8-bit RGB)

[email protected] layer + low-bandSNR layer

65.05mbps 21.79 : 1 (vs. 1280x720 8-bit RGB)

[email protected] layer + low-bandSNR layer + noise-reduced intermediate

361.5mbps 3.92 : 1 (vs. 1280x720 8-bit RGB)

[email protected] layer + low-bandSNR layer + noise-reduced intermediate +lossless residual

551.4mbps 2.57 : 1 (vs. 1280x720 8-bit RGB)

Garden Music (8-bit):


Sphere Test Dataset (16-bit):

4096x2160Tiff16 sphere, frame 0000(intra coded)

compression ratio(vs. 16-bit source)

4k layer 701.98 : 1 vs. Tiff16 source

4k layer + 2SNR layers 655.89 : 1 vs. Tiff16 source


4k layer + 7SNR +intermediate

17.16 : 1 vs. Tiff16 source

4k layer + 7SNR +noise-reduced-driven-intermediate + lossless

6.05 : 1 vs. Tiff16 source(exact to all 16 bits)(48k of 64k bins used)

gzip of Tiff16 source 3.11 : 1 vs. Tiff16 source


Sphere Test Dataset (8-bit):

4096x2160Tiff8 sphere, frame 0000(intra coded)

compression ratio(vs. 8-bit source)

4k layer 345.23 : 1 vs. Tiff8 source



4k layer + 7SNR +intermediate

26.6 : 1 vs. Tiff8 source

4k layer + 7SNR +noise-reduced-driven-intermediate + lossless

15.21 : 1 vs. Tiff8 source(exact to all 8 bits)(~192 of 256 bins used)

gzip of Tiff8 source 10.48 : 1 vs. Tiff8 source


Realtime Software Decoder (Quad G5):

Garden Music mc half resolution [email protected] realtimeGarden Music mc full resolution [email protected] half realtime

Stem MMR4 intra quarter resolution 1024x429@24 realtimeStem MMR4 intra half resolution 2048x857@24 1/2.2x realtimeStem MMR4 intra full resolution (1st layer) 4096x1714@24 1/8.0x realtime


Conclusions:

• Lossless bin-delta-style residual can be integraded with floating-point wavelet-style codec

• Efficiency is good, rivalling or exceeding other lossless codecs

• Supports arbitrarily wide dynamic range (including negative numbers)

• Noise-reduction integrated with codec, including integration with lossless bin-delta system

• Noise-reduced-driven intermediate layer forms a highly useful (perhaps optimal) step on the way to the lossless original

• Intermediates can be as finely layered as desired


Contact SMPTE or see me if you don’t have a copy of myOctober 2004 Pasadena SMPTE codec paper and would like one

Questions?

Lossless Layered Residual System

Documents

Transcript of Lossless Layered Residual System