Buildings Recognition and Camera Localization Using Image Texture Description

28
1 Buildings Recognition and Camera Localization Using Image Texture Description SULEIMAN Wassim 1 , JOLIVEAU Thierry 1 , FAVIER Eric 2 1 ISTHME-ISIG CNRS/UMR EVS, Université Jean Monnet - Saint-Etienne. 2 DIPI EA 3719 École Nationale d'Ingénieurs de Saint-Etienne [email protected] [email protected] [email protected] 25th International Cartographic Conference (Sageo) – 8 july 2011 – Palais de congrès Paris

description

3D GIS model/2D image registration called much attention in the recent years because of its vast range of potential applications in real and virtual navigation. However, automatic registration remains until now a challenge. This paper presents a methodology for enhancing and complementing a GIS database of buildings with a descriptor of their texture by using information extracted from a reference images. This descriptor is used to locate any other image by searching similar texture in the image. Then the absolute position and orientation of the camera of the new image can be computed if the camera parameters (like focal length) are known. The paper proposes a technique that can be used for achieving the identification of the facade in the photograph, calibrated camera geolocation and discusses the quality of the results.

Transcript of Buildings Recognition and Camera Localization Using Image Texture Description

Page 1: Buildings Recognition and Camera Localization Using Image Texture Description

1

Buildings Recognition and Camera Localization Using Image Texture

Description SULEIMAN Wassim1, JOLIVEAU Thierry1,

FAVIER Eric2

1ISTHME-ISIG CNRS/UMR EVS, Université Jean Monnet - Saint-Etienne. 2DIPI EA 3719 École Nationale d'Ingénieurs de Saint-Etienne

[email protected] [email protected]

[email protected]

25th International Cartographic Conference (Sageo) – 8 july 2011 – Palais de congrès Paris

Page 2: Buildings Recognition and Camera Localization Using Image Texture Description

Objective

2

Find a building in an image

SIG 3D3D GIS

Locate the camera that took the image by using the location of the building

Page 3: Buildings Recognition and Camera Localization Using Image Texture Description

Methodology

Enhancing the GIS database with information which can describe the building

unique information quantifiable information

■ The texture signature

3

Texture signature

Page 4: Buildings Recognition and Camera Localization Using Image Texture Description

How to isolate the building facade in the image?

Manual method (long)

4

Sourimant 2009

Automatic method : (3D SIG model/2D image) registration (simple building)

Page 5: Buildings Recognition and Camera Localization Using Image Texture Description

Work plan

1. GIS database enhancement with building texture information

2. Facade recognition3. Camera geolocation 4. Possible applications5. Limits

5

Page 6: Buildings Recognition and Camera Localization Using Image Texture Description

Work plan

q Enhancing GIS databases with building texture information

q Facade recognitionq Camera Geolocation q Possible Applicationsq Limits

6

Page 7: Buildings Recognition and Camera Localization Using Image Texture Description

Enhancing GIS databases with building texture information

7

Texture analyses(SIFT)

Finding the interest points with their local descriptor

Page 8: Buildings Recognition and Camera Localization Using Image Texture Description

Enhancing GIS databases with building texture information

8

Finding the (x,y,z) of the interest points

Homography constraints

3D GIS model

Page 9: Buildings Recognition and Camera Localization Using Image Texture Description

Enhancing GIS databases with building texture information

9

The texture descriptor :

list of interest points with their local descriptor and their 3D position

Page 10: Buildings Recognition and Camera Localization Using Image Texture Description

Work plan

q Enhancing GIS databases with building texture information

q Facade recognitionq Camera Geolocation q Possible Applicationsq Limits

10

Page 11: Buildings Recognition and Camera Localization Using Image Texture Description

Facade recognition

11

False matching because of the locality of the descriptor

Page 12: Buildings Recognition and Camera Localization Using Image Texture Description

Facade recognition

12

Eliminate the false matching using the homography constraints

Select the best matching score between the current image and the stored descriptor in the databases

Page 13: Buildings Recognition and Camera Localization Using Image Texture Description

Facade recognition

13

The facade in the 3D GIS

Page 14: Buildings Recognition and Camera Localization Using Image Texture Description

Work plan

q Enhancing the GIS database with building texture information

q Facade recognition

q Camera geolocation q Possible applicationsq Limits

14

Page 15: Buildings Recognition and Camera Localization Using Image Texture Description

Camera Geolocation

15

Association of the interest points with the 3D position of the matched points in the GIS databases

Page 16: Buildings Recognition and Camera Localization Using Image Texture Description

Camera geolocation

16

4 points non-collinear (Yang & al. 2009)

Real position Measured position

Page 17: Buildings Recognition and Camera Localization Using Image Texture Description

Camera Geolocation

Error for distance (20-100)m and angle (0-30°) camera direction and facade normal :■ Position : 1 - 3 m■ Orientation : 5 - 10°

17

Page 18: Buildings Recognition and Camera Localization Using Image Texture Description

Error because the texture description is not an affine function 18

Real positionMeasured position

4 points non-coplanaires SOFTPOSIT (David et al. 2004)

Camera Geolocation

Page 19: Buildings Recognition and Camera Localization Using Image Texture Description

Work plan

q Enhancing the GIS database with building texture information

q Facade recognitionq Camera geolocation

q Possible applicationsq Limits

19

Page 20: Buildings Recognition and Camera Localization Using Image Texture Description

Possible Applications

Management of photos taken in urban areas

20

Link

GIS

Page 21: Buildings Recognition and Camera Localization Using Image Texture Description

Possible Applications Navigation systems support in an urban

environment

21

Satellites visibility Multipath

Page 22: Buildings Recognition and Camera Localization Using Image Texture Description

Possible applications

Initial phase in the (2D/3D) registration

22

Sourimant 2009

Page 23: Buildings Recognition and Camera Localization Using Image Texture Description

Work plan

q Enhancing the GIS database with building texture information

q Facade recognitionq Camera geolocation q Possible applications

q Limits

23

Page 24: Buildings Recognition and Camera Localization Using Image Texture Description

Limits Angle between camera direction and facade

normal has to be less than 30°

24

Page 25: Buildings Recognition and Camera Localization Using Image Texture Description

Limits

Distance between camera and facade has to be less than 200 m

25

Page 26: Buildings Recognition and Camera Localization Using Image Texture Description

Limits

Identical facades :

26

Page 27: Buildings Recognition and Camera Localization Using Image Texture Description

Limits

Glass facade which reflects the sky and other buildings

27

Page 28: Buildings Recognition and Camera Localization Using Image Texture Description

28

Thank youFor your attention

Suleiman [email protected]