Attribution Attributing and Authenticating Evidencenflaw/EIE4114Sem12019-20/part4s.pdfCopy-move...

Attributing and Authenticating Evidence

Attribution Forensic source identification

Link multimedia content to the acquisition device

2

Authentication

Computer generated images? http://www.businessinsider.com/photorealistic

-3d-images-2013-2 The scientists found that 97% of test

subjects were fooled into believing that the digital renderings were real photographs and that real photos were CGI.

3

Attribution

4

Authentication Fake photo?

Tampering detection

5 6

Multimedia Forensics

Application of scientific methods to the investigation and prosecution of a crime Outcomes of a forensic analysis may

serve as probative facts in court Detect: source of multimedia data Detect forgeries

Copy-move forgery Hide undesired objects/replicate

similar objects Copying another region of the same image

Cell Phone Camera

Standalone Camera

Scanner

Computer Generated

7

Examples

A tampered image appeared in press in July 2008

4 Iranian missiles: 3 are real Red/purple: copy-move forgeries

2007: Fars News Agency, Tehran copy-move forgeries

Recapturing problem

10

Applications

https://www.youtube.com/watch?v=3bZvtWA7qGQ

11

Hash function An algorithm

Input: files (word document, pdf, image, …) Output: a pre-fixed length string

Purpose: ensure data integrity Property

Hashed result unique One way function

Good for authenticating word/pdfdocuments 12

Hash function Example:

http://www.fileformat.info/tool/hash.htm

13

d41d8cd98f00b204e9800998ecf8427e

7dbc9f235835a899880f3e9a7ae1f393

Hashing function To see if images are modified

Compare hash values Too strict for multimedia data

Images: transmitted through sharing platform Compression content / meaning: doesn’t change

Video: wireless loss

14

Example Images taken by smart phone send

through whatsapp compression

15 16

Feature-based Image Hashing

Feature: invariant under perceptually insignificant distortion corners? (Harris

corner detection) For each corner:

find average brightness feature 17

bitmap JPEG

rotate Local scale change

Illustration

Resizing Color Featureextraction

Hash GenerationImage hash

original whatsapp

Resize -> 8x8

Grayscale

originalwhatsapp

Color feature extraction

Average color (mean)93.42 92.39

If (intensity)> average 1Otherwise 0


Features: color features For RGB and YCbCr color spaces: 6 color

components For each color component, calculate the

statistical information Mean, variance, Moment values http://www.naturalspublishing.com/files/published/

54515x71g3omq1.pdf Concatenate moment values of the six color

components to form a feature vector

21


Form signature based on the extracted features concatenate all features together Different methods to form the hash

Represent them by using certain number of bits Take Fourier Transform, consider the magnitude and the

phase as features and represented using certain number of bits

http://www.brainflux.org/java/classes/FFT2DApplet.html

22


23

Feature: invariant under perceptually insignificant distortion

Evaluation

Hash length Robustness towards different

changes Brightness adjustment Contrast adjustment JPEG compression Addition of noise Lowpass filtering …

24

Evaluation

Large degree of compression? Share through social media

Miss detection?25 26

Active approach for data authentication

27

Digital watermark

10011010 …

© Copyright …


Active approach Digital watermark

Epson PhotoPC 3000Z, 700/750Z, 800/800Z (discontinued) Watermark is invisible Requires optional software to embed and

view watermark Kodak DC-200, 260, 290

(discontinued) Watermark is invisible Watermark capabilities built into

camera


Active approach Kodak DC-200, 260, 290

(discontinued) Watermark is invisible Watermark capabilities built into

camera


Active vs Passive Active approach

Addition of extra data More powerful, end-to-end protocol Not popular

Passive approach: Detect intrinsic image regularity or tampering

artifacts Wider application, less powerful

31

Forgery detection techniques http://www.izitru.com/ Three levels of assumption

Rules and models of the physics of the scene Inconsistency a basis for forgery detection Size inconsistency, lighting directions, shadow

inconsistencies, reflection inconsistencies Inherent characteristics of the acquisition system

(camera components, imaging pipeline) Statistics of natural images

32

Demonstration Web platform

https://29a.ch/photo-forensics/ Python:

http://www.sourcecodeonline.com/details/copy-move_forgery_detection_in_images.html

Purchase: http://belkasoft.com/forgery-detection

33 34

Example: JPEG Forensics Quantization tables:

Transform to frequency domain (Discrete cosine transform), divide each F[u,v] by a constant q[u,v]

Eye: more sensitive to low frequencies Most software use standard quantization Some software (Photoshop)

Have their own quantization table Camera manufacturers have their own table Clue for manipulation 35

Example: JPEG Forensics Quantization tables:

www.dfrws.org/sites/default/files/session-files/paper-using_jpeg_quantization_tables_to_identify_imagery_processed_by_software.pdf

36

Examples : Quantization

37

Photo1_SamsungA7

Examples: Quantization

38

Photo1_SamsungA7 Standard JPEG table, quality=96


39

Photo1_SamsungA7_modified (software) Standard JPEG table, quality=90


40

Photo1_SamsungNote Non Standard JPEG table, quality=97


41

Photo1_Nikkon Standard JPEG table, quality=80

Example: Clone detection Samsung A7 photos: combine

42

Example: Clone detection Clone: Copied regions in an image Similarity: the similarity between the

copied regions and the original Minimal detail:

Blocks with less detail are not considered in searching for copied regions

Cluster size: how many copied regions need to be found in order for them to show up as results

43

Example: Clone detection Increase “Minimum similarity”

44

http://www.imageforensic.org/

45

http://www.imageforensic.org/

46

Forgery detection techniques General two classes of techniques

Non-source identification related Lighting direction, shape of the light source specific tampering anomalies

Source identification related Features:

sensor noise pattern, dust patterns, demosaicingregularity, statistical regularities, chromatic aberration

47

Non-source identification methods

Tampering characteristics Different tampering methods different

characteristics Copy-move forgery

Highly correlated regions

Splicing Sharp discontinuity boundary

Double JPEG compression Periodicity in DCT coefficient histogram Uneven JPEG blocking artifacts 48

Copy-move ForgeryCopying regions of the original image and pasting into other areas.The yellow area has been copied and moved to conceal the truck.2 types of techniques

Block-basedKeypoint-based 49 50

Detection of Copy-move ForgeryBlock-based

Feature extraction

Find Similar blocks

N

B

B

N

B

B

Generate

(N-B+1)(N-B+1) Blocks

52

155 155 155 158 158 156 158 159

155 155 155 158 158 156 158 159

155 155 155 158 158 156 158 159

155 155 155 158 158 156 158 159

155 155 155 158 158 156 158 159

151 151 151 154 157 156 156 156

155 155 155 156 157 158 156 153

149 149 149 153 155 154 153 154

Original image

155 155 155 158

155 155 155 158

155 155 155 158

155 155 155 158

155 155 158 158

155 155 158 158

155 155 158 158

155 155 158 158

…158 156 158 159

157 156 156 156

157 158 156 153

155 154 153 154

Block size : 4 × 4

Detection of Copy-move Forgery: Features: DCT

Detection of Copy-move Forgery: DCT

Discrete cosine transform From spatial domain to frequency domain

53

155 155 155 158

155 155 155 158

155 155 155 158

155 155 155 158

DCT Transform

Original blockDCT coefficient block

420.8 37.7 -3.3 4.2

-3.0 0.9 2.2 -0.3

-0.3 -5.4 0.8 -0.7

2.6 0.7 -0.6 0.6

Features: coefficients or histogram

Detection of Copy-move Forgery

Block-based

Feature extraction

Find Similar blocks

[05, 0.6, …]

[08, 0.7, …]Similar condition ： 4

2

1_ ( , ) ( )k k

i i j i i j similark

m match A A v v D

2 2( , )i i j i i j i i j dd V V x x y y N

Results

55 56

Detection of Copy-move Forgery: Block-based

High computational complexity Lots of blocks compute features, find

matching blocks Geometric manipulation

Scaling, rotation

Detection of Copy-move Forgery: keypoints

Keypoint-based

Descriptors for each keypoint

Associate similar keypoints

[05, 0.6, …]

[08, 0.7, …]

Review of SIFT-based approach Steps:

Scale-space extreme detection Search over multiple scales DoG: difference of Gaussian

Gaussian filtering

Downsampling & Gaussian filtering

Downsampling & Gaussian filtering

Difference

Difference

Review of SIFT-based approach

Steps: Scale-space extreme detection Keypoint localization

Local extrema in the DoG pyramid Cleaning: remove low contrast points

Orientation assignment Compute best orientation for each

keypoint Achieve rotation invariance


Steps: Scale-space extreme detection Keypoint localization Orientation assignment

Find orientation of intensity gradients

36 bins (10 degrees) histogram Keypoint orientation = histogram peak

, , , , ,L x y G x y I x y

1 , 1 ( , 1), tan

( 1, ) ( 1, )L x y L x y

x yL x y L x y


Steps: keypoint descriptors

16x16 image patch descriptors Center: keypoint, origin axis: orientation

Form 4x4 sub-patches Sub-patch: histogram (8 bin) of

gradient orientation Local image gradients: 4x4x8 = 128

values

Copy-move Forgery Detection Keypoint matching

1 2

,1 ,2 ,

, ,,

, ,

j j

j

j m

j j j n

x x x xx

x x

d d d

F F F FD X

F F

Small distance: similarity of keypoints

similar objects matching

Forgery detection techniques General two classes of techniques

Non-source identification related Lighting direction, specific tampering anomalies

Source identification related Legal system:

Accepts the forensic analysis of digital image evidence if the attribution techniques are unbiased, reliable, non-destructive and widely accepted by experts in the field

Features: Hardware defects (lens distortion) Sensor defects (sensor noise pattern, dust patterns) Processing regularities (CFA, JPEG) 65

Forensic work flow

66

Image authentication Two-step process

Exam for the reliability of the evidence (image tampering and forgery detection)

Analysis to determine its probative value regarding to source camera and image metadata

67

Example Prosecuting attorneys claim:

Series of images discovered on a suspect’s computer are potentially an evidence of a crime

Possible that a third party has access to the suspect’s computer, but no evidence of such access

Desirable if forensic evidence examiner provides info about: The consistency of these images with a specific

digital camera discovered in the suspect’s house68

Digital Image Generation

69

Example: Image Acquisition

70

Example: Image Acquisition Lens: focus the light of scene on sensor Filters: filter out invisible part of light (infra-red,

ultraviolet) CFA: color filter array (on top of the sensor)

Common: only one sensor for detecting all three colors (red, green blue)

71

CFA / Demosaicing

72

Example: Image Acquisition Sensor: CCD/CMOS

Photosensitive pixels capture photons and convert them into charge

CFA interpolation To generate image with full resolution for all

colors At each sensor pixel, only one color is

measured The other two colors have to be estimated from

neighboring pixels 73

Example: Image Acquisition Post processing:

Apply enhancement technique to eliminate unwanted artifacts, degradations or noise

Color-artifact removal (introduced during CFA interpolation), edge enhancement

Storing EXIF JPEG format

74

Source-based forgery detection or model identification Discover traces left by hardware

component or software process during image generation process Image artifacts: 2 types Hardware-related

Caused by lens, sensor imperfections (noise) software-related

Introduced through camera processing

75

Image artifacts

76

hardware

software

Opticalaberrations

Sensor

Processing statistics

Lens radial distortionChromatic aberration

Processing regularities

Sensor noiseSensor dust pattern

Model statisticsHigh order statistics

CFA arrayJPEG compression

Hardware: Optical defects Optical aberrations

Radial lens distortion Straight lines appear curved in an image

Serious in low-cost wide-angle lenses The degree of distortion changes with focal length

77

Hardware: Optical defects Order-2 model

(xD,yD): distorted image coordinate (x,y): undistorted coordinate (a,b): optical centre r = sqrt((x-a)^2 + (y-b)^2)

Find distorted lines to estimate k1 and k278

2 41 2

2 41 2

( )(1 )

( )(1 )

x xD a kr k r ay yD b kr k r b

Hardware: Optical defects More likely to be used for forgery

detection Less likely to be used for source camera

attribution Built with the same/similar lenses similar

characteristics Scene content dependency: difficult to

estimate distortion in images with flat scene content

79

Hardware: Optical defects Less likely to be used for source camera

attribution Camera setting dependency

Change with focal length, focal distance, aperture size, illumination, etc

Images captured with one device but different zooming different distortions

80

Hardware: sensor defects Sensor imperfections:

Sensor defects, sensor pattern noise, sensor dust

Sensor defects / pixel defects Dead pixels:

not responding to light, appear as a black spot Rarely exist in new manufactured camera or be

removed during post-processing

81

Hardware: sensor defects Sensor pattern noise

Most sever type of sensor artifacts Photo-response non-uniformity: generated

based upon the sensitivity of pixels Sensitivity: measured by determining the

light intensity Effect of inhomogeneity of silicon wafer and the

imperfection of the sensor manufacturing process

82

Hardware: sensor defects

83

Output image Original

image

PRNU

= + +

Noisy Output Noise free input PRNU Noise Other Noise

84

Photo response non-uniformity noise (PRNU)

This pattern noise will survive for every image that taken by the same camera.

Unique for each individual

deviceDevice 1 Device 2

Hardware: sensor defects PRNU:

Can be used to identify individual device used for taking the image

Is able to distinguish cameras from same model and brand

Has been used to solve court cases when the query image was tested to verify the claimed camera device Device linking

85

Hardware: sensor defects Dust pattern on lens

Cameras with interchangeable lens Dust particles remain in front of the imaging

sensor Produce a constant pattern in all captured

images Results:

High classification accuracy Problem: user cleaned the lens?

Positive result is conclusive, but negative result is inconclusive 86

Software: processing statistics

Identify statistical artifacts left by different cameras https://www.dpreview.com/reviews/studioco

mpare.asp Color characteristics

Color reproduction of the camera with respect to each color band

image quality Measure quality of the scene reproduction by the

optical system 87 88


Example statistical features Average pixel value per RGB and RGB pairs correlation Pixel difference Use filters to decompose RGB band to three

sub-bands determine mean, variance etc Discrete cosine transform, wavelet transform,

ridgelet, contourlet, …

89


Challenges Difficult to achieve large inter-model similarity

for devices of the same brand sharing similar hardware and processing components

Camera setting dependency: focal length, indoor/outdoor illumination/flash

Scene content dependency: Images captured by 2 cameras in different environments

90

Software: processing regularities

Examine processing artifacts CFA configuration: specific arrangement of

color filters across the sensor plane

91 92

93


Examine processing artifacts CFA interpolation algorithms

Used to estimate missing color from surrounding samples of the raw pixel

Use different size for interpolation (number of surrounding samples)

Adopt different methods to estimate the missing color Simple averaging, weighted averaging, image content

dependent averaging

94


Bilinear interpolation

95

GA

GB

GL GR

)(41

ABRLI GGGGG

GI


Bilinear interpolation

96

97original interpolated

98

artifacts Appear at edges / regions with high freq

99

Features: Study the relationship among neighboring pixels

100

Image artifacts + machine learning

101

hardware

software

Opticalaberrations

Sensor

Processing statistics

Lens radial distortionChromatic aberration

Processing regularities

Sensor noiseSensor dust pattern

Model statisticsHigh order statistics

CFA arrayJPEG compression

Machine learning approach Used to analyze large amounts of data Black Box Approach:

Collect all features from a large number of multimedia data

Use the machine learning approach for grouping / classifying these features

102

103

Machine learning approach 2 types

Supervised Make predictions based on a given

set of features Unsupervised

Learn the data and organize the data by the algorithm

104

Machine learning approach Examples:

Support vector machine clustering algorithm artificial neural networks nearest neighbors Deep learning algorithm

105

Example: Tampering detection using demosaicing regularity

106

Tools for source camera attribution

Amped software: authenticate https://ampedsoftware.com/authenticate Qualified government/law enforcement

agencies Software package for forensic image

authentication and tamper detection on digital photos

107

Error level analysis Multiple JPEG compression

PRNU identification Create PRNU

PRNU tampering Find inconsistencies in PRNU noise

Clone Blocks 108

109Multiple compression 110

Inconsistence of sensor noise

111Multiple compression

Applications Insurance companies

Use forensics to cut fraud and abuse (save time)

Car crash: minor dents and scratches Upload a picture/video to the insurance company

to save time Findings: use photo editing software to create

fake photo evidence

112

Applications: insurance

113

Forensic Image analyser http://www.forensic-

pathways.com/forensic-image-analyser/

Identifies if the image was taken by a suspected device

identifies which images in a set were taken by the same device and which were taken by other devices 114

Read about the real court case in the web site

Other tools: Fourandsix Technologies http://www.fourandsix.com/ 115

Other approaches Photos: mostly come with EXIF header

Consistency between information (ISO Speed rating, exposure time, focal length) with the image content? Estimate camera setting from the image content

and compare with the data found in the EXIF header

116

Consistency checking

117

Attribution Attributing and Authenticating Evidencenflaw/EIE4114Sem12019-20/part4s.pdfCopy-move...

Documents

Transcript of Attribution Attributing and Authenticating Evidencenflaw/EIE4114Sem12019-20/part4s.pdfCopy-move...