1How

Rea

listic

is P

hoto

real

istic

?

2

How Realistic is Photorealistic?

Yaniv Lefel

Hagay Pollak

Based on the work of - Siwei Lyu and Hany Farid

3

Introduction

• Among the set of all possible images, natural images only occupy a tiny subspace.

• For instance, there are totally 256^(n^2) different 8-bit grayscale images of size nxn pixels. Natural images are sparsely distributed in the space of all possible images.

4

Image space

•e.g. when n = 10 pixels, it results in 1.3x10^154 different images !!!

5

Introduction (cont’)

• The regularities within natural images can be modeled statistically.

• Image statistical models are already in use by applications such as:Compression, de-noising, segmentation, texture synthesis, content-based retrieval and object/scene categorization.

6

Motivation 1 Identify Computer Graphics

• Sophisticated computer graphics software can generate highly convincing photorealistic images able to deceive the human eye.

• Differentiating these two types of images is an important task to ensure the authenticity and integrity of photographs.

7

Computer graphics example

8

Motivation 2Identify Steg Images

• Image steganography hides messages in digital images in a non-intrusive way that is hard to detect visually.

• The task of generic steganalysis is to detect the presence of such hidden messages without the detailed knowledge of the embedding methods.

9

Steganography example

• Steg is the message image embedded into the original image.• The rightmost image is the absolute value of the difference

between the original and steg image, normalized into 8 bit for display purposes.

Original

message

Steg |Original-Stego|

10

How ? Example

11

Motivation 3Identify Re-broadcasting

• Biometrics-based (e.g., face, iris, or voice) authentication and identification systems are vulnerable to the “rebroadcast” attacks. (e.g. using a high-resolution photograph of a human face).

• We need to differentiate a “live” image (captured in real time by a camera) and a “rebroadcast” one (a photograph).

12

How to distinuish images ?

• Image properties ? – Image intensity histogram– Image frequency

Other method ?

13

Using known methods

14

Why wavelets

• Image representations based on multi-scale image decomposition (e.g., wavelets) decompose an image with basis functions partially localized in both space and frequency - a compromise between these representations.

17

QMF - Quadrature Mirror Filter

• The QMF pyramid decomposition splits the image frequency space into three different scales, and within each scale, into three orientation subbands (Vertical, Horizontal and Diagonal).

18

QMF diagram

19

QMF

• The vertical, horizontal and diagonal subbands at scale i are denoted by Vi(x; y), Hi(x; y), and Di(x; y), respectively.

• Can be generated by convolving the image, I(x, y), with low-pass and high-pass filters.

23

QMF decomposition – Example 1

24

QMF decomposition – Example 2

25

Example – QMF statistics

28

Add some magic …• QMF

coefficients

• Magic Box

• Error coefficients

Simple but long (and out of scope)

mathematical procedure

32

Technique Diagram

Feature vector

33

Computing the Feature Vector

• 3 – Sub-bands (vertical, horizontal, diagonal).• 3 – Scales (levels of decompositions).• 4 – First order statistics (mean, variance, skewness

– asymmetry measure, kurtosis).• 3 – Colors (RGB)• 2 – marginal statistics (wavelet coefficients),

error statistics.• 216 = 3*3*4*3*2

34

Image examples

35

Feature vectors projected on 3D space

Natural image – Blue.Synthetic images - noise (Green), fractal (Black), and discs (Red)

36

Learning and Testing CG\Steg\rebroadcast

• CG\steg\rebroadcast images are prepared.• Statistics is collected over natural images and

CG\steg\rebroadcast images (not using color).• A Machine learning system (e.g. FLD, LDA,

SVM) is then trained on some of the natural and some of the CG\steg\rebroadcast images.

• The remaining natural and CG\steg\rebroadcast images are used for testing.

37

Natural vs. CG results (SVM)

All images

Train Succ [%]

Test Succ [%]

Natural 40000 32000 70.9 8000 66.8

CG 6000 4800 99.1 1200 98.8

38

Training the system

39

Photorealistic (CG) images

41

The Impact of Color

42

Correctly Classified Photorealistic

43

Incorrectly Classified Photographic

44

Natural vs. Steganography images

• A message consists of a 64x64 pixel region of a random image chosen from the same image database.

45

Natural vs. Steganography results

All images

Train Succ [%]

Test Succ [%]

Natural 1000 750 99.5 250 98.9

Steg 1000 750 98.3 250 97.6

46

Live vs. rebroadcast

• We collect statistics from natural images and the same images after having been printed on a laser printer and re-scanned with a scanner (printing and scanning are done at 72 dpi).

47

Live vs. rebroadcast results

All images

Train Succ [%]

Test Succ [%]

Natural 1000 750 99.5 250 99.5

rebroadcast

200 150 100 50 99.8

48

Live vs. rebroadcast (cont’)

• Remark: It is not surprising that printing significantly disturbs the image statistics. Detecting a rebroadcast image will become more difficult with printers improvement.

49

Rebroadcasting example

Shown is the original iris images (top row) and the images after being printed and scanned (bottom row).

50

Feature vectors projected on 3D space

Results from a four-way classifier of 1000 natural, 1000 steg, 500 graphic, and 200 rebroadcast images.

51

More Applications

how many different artists ?

52

More Applications

Forgery detection.

54

Finally

• Statistical model.

• capture regularities that are inherent to photographic images.

• Distinguish tampered \ CG images and natural images.