Generation of Virtual Image from Multiple View Point Image Database

Haruki Kawanaka, Nobuaki Sado and Yuji Iwahori

Nagoya Institute of Technology, Japan

・

Background

The soccer playing game has become popular in Japan since the World Cup Soccer of Japan and South Korea cosponsorship was held in 2002. It is desired to see the game from various view points.– setting cameras at the reverse side of the goal– at the ceiling to view down. – many cameras are set at various locations– a camera has the function of pan−tilt−zoom

But, these are not free viewpoints.

・

・

Previous method to generate a virtual image

large scale environment with many camera settings and installations– It takes much cost.– the application is restricted at only that stadium.

using a few cameras and a motion capture– at only the indoor space– the special wear and several markers

It is difficult to use in actual games.

・・

・・

・・ ・

・

Present approach

The labels of the back number are generated as the virtual image.

The pose of each player is not considered.

・・

Study Purpose

generate a virtual image at another view point – from a real image – without the special environment

・・

Proposed Method

The appropriate pose image of each player is determined from using multiple viewpoint image database of a player’s CG model.

Each pose image is synthesized at the position to the virtual scene.

The position of each player is assumed to be provided by the trajectory system.

Trajectory Recording System

We have developed Trajectory Recording System.

Trajectory of two players

・・・・・

Flow of Proposed Method

1. Creation of database by CG model

2. Generation of virtual image for each player from image database

I. Recognize the pose of a player

II. Generate the corresponding virtual image from another view point

3. Synthesis of another view point image・

Creation of database by CG model ～ Images for Database ～

Image Database (Human model) is created by CG.– Various motions (run, walk, shot, pass, heading, trap

etc) 280 poses– From 8 view direction (rotation with every 45°)– total 2240 images

・

・ ・

Creation of database by CG model ～ Processing of Each Image ～

CG Image of human model Silhouette

To eliminate of many factors such as the condition of light source, skin color, hair and uniform…– It is necessary to save the data size and the search

time of the image database.

Image database is created using the silhouette. – This depends on the difference of pose but does not

depend on such factor of each player.

・

・

Creation of database by CG model ～ Processing of Each Image ～

N

N

Normalize the image size[00011000 ･･･ ]

N×N

1-dimensional data

The image size for each pose is normalize.– The rectangle region which surrounds the silhouette

of each pose, is extracted.– The extracted region just touches to the square with

keeping its aspect ratio.

By the raster scan, one dimensional expansion of the normalized image is made.

・

Creation of database by CG model ～ Principal Component Analysis ～

Set of image data(all poses & all view)

eigen values, eigen vectors

Covariance matrix

Pj j

i

1

21

eseigen valu

of total

λλ･･･λλ

Compress & Projection onto eigen space

If the sum of eigen values becomes over 90%

effective

Image data

size image:eeigen valu: Pi 　λ

[00011000 ･･･ ][01011010 ･･･ ]

[10001001 ･･･ ]・・・

・

・

・・

・

・

Generation of virtual image for each player ～ Recognize the pose of a player(1) ～

Detection of player

N

N

[0000111 ･･･ ]

by eigen vector

Normarize Projection onto eigen space

1-D

Each silhouette is normalized, changed to one dimensional vector and projected to a point in the eigen space. ・・・

Generation of virtual image for each player ～ Recognize the pose of a player(2) ～

Eigen Space Result of search

minimumdistance

e1

e3

e2

CG imageReal image

AB

When A is given, B is selected as the most similar sample. The pose of image A is recognized as B.

given image

・・・

Generation of virtual image ～ Generate the corresponding virtual image

from another view point ～

Coordinates are acquired from trajectory recording system

Virtual stadium created using the OpenGL

＋

Another view image is made from result of search according to view point & view direction of virtual image.

＋Generated

virtual scene

(x, y)

・

・

・・

Original Silhouette Searched Similarity : s

90.0%

87.1%

94.0%

Experiments

1or 0 valuesilhouette:,

(%)100)(

1 1

ii

N

i ii

yx

N

yxs

Actual original image

Virtual image from the same view direction as original

The experiment of pose recognition・・・・

・・

Experiments

Original imageVirtual image from different view point

Player’s position is fixed.Viewpoint is moved.Texture is used as soccer field.

It is also possible to generate an animation of movie by connecting each frame image sequentially.

・

・

Conclusion

A new approach to generate a virtual image from another view point is proposed.– Multi-image database to apply the eigen s

pace method for the pose recognition.

– This approach is simple but generates the reasonable virtual scene.

・

・・

Future Works

It is difficult to discriminate the absolute pose of each player.

It is also difficult to treat the overlapped case in which two or more players cross.

Investigation of more effective matching approach is required to reduce the cost of time and memory.

・