Outline - Columbia Universitysfchang/course/vis/SLIDE/lecture12-handout.pdf · EE 6850: Visual...
Transcript of Outline - Columbia Universitysfchang/course/vis/SLIDE/lecture12-handout.pdf · EE 6850: Visual...
EE 6850: Visual Information System
Lecture # 12: Multimedia Security II
Professor Shih-Fu ChangGuest Speaker: Dr. Ching-Yung Lin
Columbia University /IBM T. J. Watson Research Center
Nov 27, 2002
Outline§ Watermarking
-- Introduction-- Digital Communication and Spread Spectrum
Techniques-- Robustness of Watermarking Techniques -- Other Issues and References
Watermarking
Tx Rx
Watermark Verify thewatermark
PIL or content- based feature codes
• Embedding Visible/Invisible Codes in Multimedia Data for (or not for) Security Purpose
Visible Watermark
♦ Purpose: – Claim the ownership and prevent content piracy.
♦ Properties:– Robust: Watermarks must be very difficult, if not impossible,
to be removed.
– Non-obtrusive: Watermarks must not affect the audiovisual contents too much.
– Visible: It must be visible, but it had better to be insensible.
Visible Watermark Example: IBM Method
♦ Description:– Pixel brightness were altered pixel by pixel.
Depending on the brightness of each mask pixel, if it is larger than a threshold, then we add a value to the corresponding pixel of the original image to make it brighter. Otherwise, if it is smaller than a threshold, then we subtract a value to make the corresponding pixel darker.
♦ Alternation Function:
Watermark mask
♦ Robustness: – Randomly set the adjustment factor.– Randomly locate the mask.– The mask pixels may depend on the
image contents.
Invisible Watermark
♦ Purpose: – Protect ownership and trace illegal use.
♦ Properties -- Transmit a bitstream through a very noisy channel, i.e. the
original picture.
– Robust: The watermark must be very difficult, if not impossible, to
remove. It must be able to survive manipulations to the images, such
as: lossy compression, format transformation, shifting, scaling,
cropping, quantization, filtering, xeroxing, printing, and scanning.
– Invisible: The watermark should not visually affect the image/video
content.
original image
add watermark
manipulation
image after crop-and-replacement and JPEG lossy compression
watermarked SARI image
authentication
authentication& recovery
Invisible Watermark Example: Self Authentication-and-Recovery Images (SARI)
What is Watermarking ? –Multimedia as a Communication Channel
Information
W Encoder Decoder W
§ Analog Communication --§ Encoder/ Decoder:
§ Amplitude Modulation (AM), § Frequency Modulation (FM). èMultiplexing: use different carrier frequencies.
§ Channel: air, wire, water, space, …
Channel
Information
W Encoder Decoder WImage/Video/Audio
§ Watermarking:
§ Basic communication system:
Digital Communication
Information
W Encoder
S
Decoder W
Z ≤ N
SwX ≤ P Sw
Encoding Keys Decoding Keys
§ Encoder may include two stages: Coding and Modulation.§ Coding:§ Error Correction Codes, Scrambling (use cryptographic keys).
§ Modulation: § Time Division Multiple Access (TDMA), Frequency Division Multiple Access
(FDMA), Code Division Multiple Access (CDMA).§ Spread Spectrum is a CDMA technique, which needs modulation keys for
Frequency Hopping or other specific codes.
Spread Spectrum Communication
Information
W Encoder
S
Decoder W
Z ≤ N
SwX ≤ P Sw
Encoding Keys Decoding Keys
• Spread Spectrum Communication:• Orthogonal codebooks, E [ fi · fj] = 0• e.g.:
• f1 = 1 1 –1 1 1 –1 –1 –1 –1 1• f2 = -1 1 –1 1 –1 1 1 –1 1 1
• Detection:• arg (n) max correlation coefficient ( Sw – S , f n )
• Examples
Watermarking on Multimedia Content
Information
S: Source Image (Side Information)W: Embedded InformationX: Watermark (Power/Magnitude Constraint: P)Z: Noise (Power/Magnitude Constraint: N )
W Encoder
S
Decoder W
Z ≤ N
Source Image
SwX ≤ P Sw
PerceptualModel
Private/ Public
DistortionModel
Frequency-Domain Analysis: DCT and DFT
Spectrum
Original Image
DCT: (Discrete Cosine Transform)
DFT: (Discrete Fourier Transform)DFT
Spread Spectrum Watermarking (Cox et. al. 1997)
• Spread Spectrum: T( Sw ) = T( S ) + T(X)• T can be any spatial-frequency transforms. • E.g. Fourier Transforms (DFT, DCT), Wavelet Transforms
• Objectives:• Detect the existence of a specific code, which is served as thecopyright information.• Watermark detection needs the original source.
• Implementation:• Add a specific code on the 1000 largest or the 1000 lowest frequency DCT coefficients of the image.• E.g. T(X) = 1 1 -1 1 1 –1 –1 –1 –1 1 …..
• Detection:• correlation coefficient ( T(Sw ) – T (S) , T(X) )
Considerations for Watermarking System
§ Security – Kerchoff’s assumption
§ Robustness§ Hiding Payload§ Application Scenario
Three Metrics of Watermarking
Amount of embedded information
Robustness
Invisibility
RobustWatermark
FragileWatermark
Semi-FragileWatermark
Robustness of Watermarking
§ What spread spectrum methods can do -- Pixel Value Distortion
§ What spread spectrum methods cannot do -- Geometric Distortion:-- Rotation, Scale,-- Translation, Cropping
Change of Boundary, Padding
original printing printed documentscanning
rescannedimage
• An example of application scenario: Print-and-Scan
Designing Robust Watermarking against Geometric Distortion
rotation
cropping
scaling
R+S+C
S+C
General CroppingStrict Cropping
♦ Some solutions:– 2nd watermark (self-registration template): Univ. of Geneva, 1998.– Recognizable structure: Kutter, 1998. – Invariant coefficients: O’Ruanaidh, 1998; Lin et. al., 2000
♦ Example of Rotation, Scaling, and Cropping
Continuous Fourier coefficients of continuous images after RSC
♦ Rotation in time => Rotation in frequency
♦ Scaling in time => Scaling in frequency
♦ Translation in time => Phase shift in frequency
♦ Information Loss in time => Noise in frequency
♦ Change of Image Size in time => No definition in frequency
),()cossin,sincos(
)cossin,sincos(),(
212121
212121
ffXffffX
ttttxttx
R
FR
=+−→←+−=
θθθθθθθθ
),(),(),(),( 21221121 2
2
1
1 ffXffXxttx SFtt
S =→←= λλλλ
Difference between continuous Fourier Transform and Discrete Fourier Transform
♦ DFT: Samples of the Fourier coefficients of the repeated discrete image.
FT Sampling DFT coefficients
O p e r a t i o n s i n t h e d i s c r e t e i m a g ed o m a i n
D F T S i z e S c a l i n g C r o p p i n g R o t a t i o nI m a g e s i z e A l m o s t n o
e f f e c t *S c a l i n g +P h a s e s h i f t +( I n f o r m a t i o nl o s s )
R o t a t i o n
F i x e d l a r g e s i z e S c a l i n g P h a s e s h i f t +( I n f o r m a t i o nl o s s )
R o t a t i o n
S m a l l e s tr e c t a n g l e w i t hr a d i x - 2w i d t h / h e i g h t
S c a l i n g P h a s e s h i f t +( I n f o r m a t i o nl o s s ) +( S c a l i n g )
R o t a t i o n
S m a l l e s t s q u a r ei n c l u d i n g t h ew h o l e i m a g e
S c a l i n g i n o n ed i m e n s i o n a n dn o e f f e c t * i nt h e o t h e rd i m e n s i o n
S c a l i n g +P h a s e s h i f t +( I n f o r m a t i o nl o s s )
R o t a t i o n
♦ Compare to the continuous domain
DFT coefficients after scaling or zero-padding
t
F.T.f
F.T.(c)
TS/2 f0
t
F.T.f
t
F.T.f
The continuous original signal
(b)T0
TS
TB
f0 fS
fB
2fST0
t f(d)
2T0 f0/2fSTS
discretization
up-sampled (or scaled)
zero-padded
DFT coefficients after rotation
♦ Characteristics: “cross” effect, Cartesian sampling points=> Solutions: Estimate the cross positions from boundary/
larger values
Spectrum
After Rotation After Rotation and CroppingOriginal Image
The Log-Polar Map of Fourier Coefficients
♦ Log-Polar Map
θ
log r
For RST (uniform scaling)
§ Rotation: shift in the θ axis§ Scale without boundary change:
shift in the log r axis. § Translation: no effect on the
magnitudes.§ Scale with boundary change,
cropping: noise.
Projection along the log r axis:§ Cyclic shift for rotation,§ Invariant to scaling.
Algorithm for generating feature vector
Calculate the magnitudes of log-polar coefficients, |Fm|, from DFT magnitudes
Summation of the log of the Fourier-Mellin magnitudes along log r axis
Combine values in orthogonal directionsg1(θ) = g0(θ) + g0(θ+90°)
Subtract g(θ) by its global mean, (whitening filter)
The Feature Vectorfv = g(θl, …, θu)
Zero-padded to double image size
feature vector watermark vectormodified feature vector
Embedding Public Watermark : Feature Vector Shaping
§ Extract a Noise-Like Feature Vector and iteratively shape it to a watermark pattern
§ Estimation differences in the log-polar domain and distribute them in the 2-D DFT domain.
feature vector watermark vectormodified feature vector
(mixed signal)
Spread Spectrum:T( Sw ) = T( S ) + T(X)
Feature Vector Shaping:T( Sw ) ≈ T(X)
Test Example -- Print-and-Scan
After PS, Crop to 360x240 & JPEG CR: 95:1 => ρ=0.64, Z=4.30
After Print & Scan, Crop to 402x266 => ρ=0.80, Z=6.46
Original Image [384x256] Watermarked Image, PSNR 43.8dB, ρ=0.84, Z=7.02
Measure Metric I: False Positive (10,000 images from Corel Image Library, 10 different watermarks)
§ What are False Positive (false alarm) and False Negative (miss)?§ What are ROC curves?
Detection Result
Fact
A BC
Measure Metric II: Robustness (ROC curves of 2,000 images)
Rotation(4°, 8 °,
30 °, 45 °)
Scaledown
(5%, 10%, 15%, 20%)
Scale Up
(5%, 10%15%, 30%)
Trans-Lation
(5%, 10%,15%, 20%)
Information Hiding Capacity
Information
S: Source Image (Side Information)W: Embedded InformationX: Watermark (Power Constraint: P)Z: Noise (Power Constraint: N )
• Channel Capacity: Shannon (1948) (private watermarking) , Costa (1983) (public watermarking)
C = ½ log2 ( 1+ P/N) bit/sample
W Encoder
S
Decoder W
Z ≤ N
Source Image
SwX ≤ P Sw
PerceptualModel
Private/ Public
DistortionModel
-- Is this the final answer or just a beginning?
• Assumptions: • Uniform Power Constraints on Watermark and Noise• An image is a channel
Masking Effects on Human Vision System Models
e.g. JND modelPSNR = 32 dB
Specific Domains: •Watson’s DCT masking (1993)• Watson’s Wavelet masking (1997)• Chou and Li’s JND (1995)• JPEG, QF =50
General models:• Lubin’s HVS model (1993)• Daly’s HVS model (1993)
Some properties of HVS models:
-- Amplitude nonlinearity, Inta-eye blurring, Re-sampling, Contrast sensitivity function, Subband decomposition, Masking, Pooling
How many channels within an image?
Previous Propositions:• Image as parallel channels?
• Parallel Gaussian Channels – Akansu (1999), Servetto (1998), Kundur(2000) -- Possible Drawback: A channel needs infinite codeword length !!
• Image as one channel•Arbitrary Varying Channel (AVC, Csiszar 1989 ) – Possible Drawback: arbitrary varying
New research directions on watermarking capacity:• Image as a variant-state channel• Image coefficient values are discrete• Human vision models
If not power-constraint, but amplitude constraints on noises,è Zero-Error Watermarking Capacity
Watermarking capacity of power-constrained noisy environments
σnoise = 5
WW: 102490 bitsWD: 84675 bitsJPG: 37086 bitsChou: 33542 bits
Reference:Zero-error capacity
JPG: 28672 bits
Resources: Copyright Protection Forum§ The Copyright Protection Technical Working Group
(CPTWG) : http://cptwg.org
§ The Intellectual Property Management and Protection Group of the Moving Picture Experts Group (ISO/IEC JTC1/SC/29/WG11, IPMP group at MPEG) : http://www.cselt.it/mpeg and http://www.mpeg.org
§ TV Anytime Forum: http://www.tv-anytime.org
§ Digital Versatile Disk Forum: http://www.dvdforum.org
§ Secure Digital Music Initiative’s charter: http://www.sdmi.org
§ Open Platform Initiative for Multimedia Access: http://www.cselt.it/ufv/leonardo/opima
§ Digital Audio-Visual Council: http://www.davic.org
Resources: Books
§ Digital Watermarkingby Ingemar Cox, Jeffrey Bloom, Matthew Miller (Oct. 2001)
§ Information Hiding Techniques for Steganography and Digital Watermarkingby Stefan Katzenbeisser and Fabien A. P. Petitcolas (Jan. 2000)
§ Image and Video Database: Restoration, Watermarking and Retrieval
by Hanjalic, Langelaar, Van Roosmalen and Biemond (July 2000)
Resources: Papers§ H. Yu, D. Kundur, and C.-Y. Lin, “Spies, Thieves, and Lies: The Battle for
Multimedia in the Digital Era,” IEEE Multimedia, Vol.8, No. 3, July 2001.
§ I. J. Cox, J. Kilian, F. T. Leighton and T. Shamoon, “Secure Spread Spectrum Watermarking for Multimedia,” IEEE Trans. on Image Processing, Vol. 6, No. 12, Dec. 1997.
§ G. W. Braudaway, K. A. Magerlein, and F. Mintzer, “Protecting Publicly Available Images with a Visible Image Watermark,” SPIE Optical Security and Counterfeit Deterrence Techniques, Vol. 2659, Jan. 1996.
§ C.-Y. Lin and S.-F. Chang, “Semi-Fragile Watermarking for Authenticating JPEG Visual Content,” SPIE Security and Watermarking in Multimedia Contents II, Vol. 3971, Jan. 2000.
§ F. A. P. Petitcolas, R. J. Anderson, and M. G. Kuhn, “Information Hiding: A Survey,” Proceedings of the IEEE, Vol. 87, No. 7, July 1999.
§ J. A. Bloom, I. J. Cox, T. Kalker, J.-P. Linnartz, M. L. Miller and B. Traw, “Copy Protection for DVD Video,” Proceedings of the IEEE, Vol. 87, No. 7, July 1999.
Resources: Links
§ Multimedia Authentication Resources: http://www.ctr.columbia.edu/~cylin/auth
§ Watermarking mailing list: http://www.watermarkingworld.org
§ Information Hiding and Digital Watermarking:
http://www.cl.cam.ac.uk/~fapp2/steganography
Issues in Multimedia Security§ copyright protection, authentication, fingerprinting: system,
theory and techniques § public watermarking techniques, watermarking attacks, quality
evaluations and benchmarks § perceptual models, noise models, information theoretical models § conditional access § legal aspects § watermarking protocols§ security in JPEG2000, MPEG-4, MPEG-7 or MPEG21 § biometrics and multimedia security § watermarking/ information hiding applications