My Research Experience My Research Experience

Cheng Qian

Outline Outline • 3D Reconstruction Based on Range Images

• Color Engineering

• Thermal Image Restoration

3D – Overview 3D – Overview To reconstruct the geometry and texture of a scene in a virtual environment.

--- 3D scanning

Create an arbitrary view by interpolation

Method 1： 2D color-image-based reconstruction

3D – Overview3D – OverviewMethod 2： Range-image-based reconstruction

Range image -----Depth

Intensity image --Reflectance

CCD Image--RGB

3D – Overview3D – Overview Range Image Intensity Image CCD Image

A digital model

Geometric Structure Materials Texture

3D – Overview3D – Overview

Preprocessing

Registration

Mesh

Texture Mapping

Lighting

Shading …..

System architecture

Raw Data (3D coordinates, Intensity, RGB)

Visual Information (Geometry, Texture)

Knowledge from Object Recognition

Modeling

3D – Range Image Registration 3D – Range Image Registration

Image of a left view

Images registered

Image of a right view

Objective

3D – Range Image Registration3D – Range Image RegistrationScheme

Range image I1 Range image I2

Description based on geometric features and their interrelationships

……

Construct feature correspondence M and measure the Similarity S between the two images ------ Find the Ｍ maximizing S

Feature Extraction: surface, curve, corner point ……

Noise filtering, outlier removing ……

3D – Range Image Registration3D – Range Image RegistrationNoise filtering, outlier removing

Before After

• Polar window filtering,

• Pseudo-median filtering

• Isolated point filtering

3D – Range Image Registration3D – Range Image RegistrationFeature extraction

Surface: adaptive-shape window

Curve, corners: edge evolution

3D – Range Image Registration3D – Range Image RegistrationDescriptions of the geometric features

Interrelationship contained in nested geometric features

Related geometric features are nested

3D – Range Image Registration3D – Range Image RegistrationCorrespondence and similarity measure

Virtual features

Virtual features

3D – Range Image Registration3D – Range Image RegistrationImprovement of the registration results

Before After

Global optimization

3D – Range Image Registration3D – Range Image RegistrationWhat was left: Texture

Color Engineering Color Engineering

-60

-30

0

30

60

90

-90-60

-300

3060

901

1.05

1.1

1.15

1.2

1.25

1.3

1.35

1.4

• Proposed a method for measuring luminance distribution of indoor scenes using a digital camera rather than an expensive luminance meter.

• Proposed a novel radiometric model for CCD sensors and a color self-calibration algorithm based on this model. The objective of this project is to calibrate the color performance of 100 CCDs in a lightfield-rendering system for 3D scene reconstruction.

CCD 1 Fake CCD 2 CCD2

Transformed to be To approximate

Color Engineering Color Engineering • Calibrated a line CCD sensor with poor color performance. The radiometric correlation between r, g, b channels is considered.

11 12 13 14

21 22 23 24

31 32 33 34 1

rr m m m m

gMin g m m m m

bb m m m m

12 13 21 23 31 32, , , , , 0m m m m m m

Thermal Imaging Thermal Imaging •Proposed a radiometric model for infrared cameras and developed relevant model reconstruction methods, which resulted in obtaining a very precise forward function for thermal image restoration. Regularization techniques, such as Tikhonov, Total Variation, and Lasso, were applied to the restoration procedure and their performances were compared.

Thermal camera model

Thermal ImagingThermal ImagingThermal image restoration

Original image Image restored by Tikhonov regularization, Edges are strongly penalized

Image restored by Discontinuity-Adaptive model regularization, Edges are adaptively penalized

Noise is suppressed

Convexity of energy function is well controlled.

Image restored by Total Variation regularization, Edges are preserved

Thermal ImagingThermal Imaging• With the adjustment of the camera setting, the point spread function (PSF) of the camera system can be changed. Therefore we try to develop a semi-blind image restoration algorithm that can recover the original image and the PSF simultaneously.

Iteration 1

(a)Original image Iteration 1 Iteration 2 Iteration 3 Iteration 4

Iteration 5 Iteration 6 Iteration 7 Iteration 8 Iteration 9 Final restoration results

                           

 

• Thanks