A Prototype Software System for Multi-object Recognition ...
Transcript of A Prototype Software System for Multi-object Recognition ...
A Prototype Software System for Multi-object Recognition and itsFPGA Implementation
H. Ando, N. Fuchigami, M. Sasaki and A. Iwata
Graduate School of Advanced Sciences of Matter, Hiroshima University1-3-1 Kagamiyama, Higashi-hiroshima, 739-8530, Japan
Phone: +81-824-22-7358, E-mail: ando, futigami, sasaki, [email protected]
1. IntroductionIn order to recognize various objects such humans, an-
imals, and so on, we have to develop highly intelligentinformation processing systems. Robust image process-ing techniques are also required under variation in size,orientation, lighting condition, etc. Moreover, it is nec-essary to develop VLSI hardware systems with the latestintegrated circuit technologies.
For the aim of achieving highly information processing,we have been proposed a concept of multi-object recog-nition system based on 3D custom stack system (Fig. 1)presented in the 21st century COE of Hiroshima Uni-versity [1]. The preliminary experimental results of facedetection/recognition have already been obtained [2].
In this paper, as a first step toward realizing a fu-ture multi-object recognition system, we propose a realtime human face detection/recognition software systembased on eigenfaces methods. Moreover, we design objectrecognition LSI circuits including eigen-space generationcircuits and DB matching circuits, and implement thesecircuits in FPGA.
Vision
Detection
RecognitionDB
1DB
2DB
N
Normalization
Local Wireless Interconnection
Global Wireless Interconnection
Input Image
Power Supply
Figure 1: A concept of multi-object recognition system based
on 3DCSS.
2. Eigenfaces for face detection and recognitionOne of the well-known face recognition methods is
“Eigenfaces” based on PCA (Principal Component Anal-ysis) [3, 4]. Let us explain the algorithm of this methodbriefly. An i-th face image consists of M pixels is repre-sented as a vector Γi. A preprocessed face Φi is definedby Φi = Γi − Ψ, where Ψ represents the average faceof N images in DB(database), that is Ψ = 1
N
∑Nn=1 Γn.
The “eigenfaces” can be calculated as the eigenvectors ak
of the covariance matrix C of DB, C = 1M
∑Mn=1 ΦnΦt
n,where Φt is a transposed matrix.
A face image is transformed into so-called “eigen-space” ωk by a simple operation: ωk = at
kΦi. The eigen-space ωk forms a vector Ω = [ω1 ω2 . . . ωm] that de-scribes the contribution of each eigenface for face image.
Face detection is performed by generally used thresh-olding methods. A reconstructed image Φr, defined byΦr =
∑mk=1 ωkak, is used as an input of evaluation func-
tion for thresholding. For example, Euclidean distanceε = ∥Φin − Φr∥ is often used as evaluation function,where Φin is a preprocessed unknown input image. Ifthe value ε is lower than a threshold, an unknown inputimage is classified as a human face.
Face recognition is also achieved with the similarmethod. If the face space vector ΩDBi of i-th faceimage in DB leads to the minimum distance εmin =∥Ω − ΩDBi∥, we can know that the input face is thesame as i-th face.
By the eigenfaces methods, only an object of the samekind as DB can be recognized. Therefore, we can recog-nize the various kinds of object by preparing DB corre-sponding with them.
3. Development of prototype software systembased on eigenfaces methodsWe developed the prototype software systems for
multi-object recognition (Fig. 2). Note that the systemis not necessarily limited to the application to humanface. Our proposed algorithm in the case of human facerecognition shown in Fig. 2(b) is as follows: First, aninput image with RGB color spaces is resized to about100×100 pixels in order to improve the processing time.Second, the candidate of face area is determined by skincolor detection. Finally, at the inside area of the candi-date region, face detection and recognition are achievedby using the methods explained in Sec. 2. After face de-tection/recognition was done, the candidate of face areais limited to a small area around the detected face regionfor the speed-up of the movie picture processing.
In the proposed algorithm, we used Euclidean distanceand Correlation coefficient for evaluation functions. Onlyif both of the evaluation values are satisfied, a face can bedetected and the detected face can be recognized. Whenthis condition is not satisfied, the system answers thatthere is not a face or the detected face does not exist inDB.
We achieved the correct face detection and recognitionfrom natural scenes with the developed software systemas shown in Fig. 2(c) and (d). The processing time wasabout 10∼30fps for the captured VGA image by usingthe latest personal computer.
detected skin color area
detected face area
computertarget
USB camera result view
(c) Appearance of software systems
(d) Screen shot of human-face detection/recognition
identified person
face candidate area
Face detection
Skin color detection
Face recognition
Image capture
Image scale-down
(b) Flow chart
Computer
Image capture device
Operating system
Programming language
Xeon3.06GHz x 2CPU
USB Camera
Windows XP
Visual C++7.0 & DirectShow
(a) System configuration
Figure 2: A prototype software system of multi-object recog-
nition.
4. Hardware implementation of object recogni-tion processingWe designed LSI circuits corresponding to object
recognition. As mentioned in Sec. 2, object recognitionneeds calculation of eigen-space, evaluation, and search-ing for minimum value. Therefore, the object recognitioncircuit blocks can be realized by using the eigen-spacegeneration circuits and DB matching circuits as shownin Fig. 3. The eigen-space ωk = at
kΦi are calculatedby using subtracter, multiplier, accumulator and mem-ory storing average face and eigenfaces. If one use theEuclidean distance for an evaluation function, the DBmatching circuits consist of evaluation block includingsubtracter and multiplier, Winner-take-all (WTA) circuitblock for searching for minimum distance, and memorystoring eigen-spaces. In this way, the eigenfaces meth-ods can be easily realized by conventional simple digitalcircuits.
The designed LSI circuits were implemented to theFPGA evaluation board (Fig. 4 (a)). The used FPGAdevice model was “Xilinx Virtex-II Pro(XC2VP7)”. Thespecifications of the implemented circuits are shown inFig. 4 (b), where the evaluation function was the Eu-clidean distance. As a result, the operation time of oneobject recognition was 212µs, where the object image sizeand the clock frequency were 20×20 and 100MHz, respec-tively. Therefore, we can perform 100 objects recognitionper 1ms by employing line-parallel processing.
Subtracter Multiplier AccumulatorΓin
aDBΨDB
Memory
Input image
Average face Eigenfaces
Preprocessed imageEigen-space
ΦinΩin
aDBΨDB
(a) Eigen-space generation circuits
Evaluation block
Eigen-space
Ωin
ΩDB
Memory
Eigen-space
ΩDB
E(Ωin, ΩDB)
Values of evaluation function
WTARecognition
results
(b) DB matching circuits
Figure 3: Object recognition circuits based on PCA.
Face-space generation circuit
Bit width of eigenface
Bit width of input face
Rate of gate use
Processing time
7bit
8bit
1.7%
200µs
DB matching circuit
Bit width of eigen-space
Rate of gate use
Processing time
18bit
16%
12µs
(a) FPGA evaluation board (b) Specifications
FPGA
Figure 4: FPGA board and detail of designed circuits.
5. ConclusionThe prototype software systems for multi-object recog-
nition system were developed and real time human facedetection and recognition were achieved. The process-ing time was about 10∼30fps by using latest personalcomputer. Moreover, we designed the object recognitionLSI circuits and implemented these circuits to FPGA.The operation time of one object recognition was about200µs.
We are scheduled to extend the software so that vari-ous objects can be recognized and to complete the hard-ware system including an image input device and objectdetection circuits.
References
[1] A. Iwata, et al., “Targets and Research Plan of 21stCentury Center of Excellence(COE) on Nanoelectronicsfor Tera-bit Information Processing”, Proc. First Interna-tional Workshop on NTIP, pp. 2–3, 2003.
[2] H. Ando, et al., “Human Face Detection and Recogni-tion using Principal Component Analysis”, Proc. SecondInternational Workshop on NTIP, pp. 78-80, 2004.
[3] W. Zhao, et al., “Face Recognition: A Literature Survey”,ACM CSUR archive, vol. 35, pp. 399-458, 2003.
[4] M. A. Turk, et al., “Eigenfaces for Recognition”,CVPR’91, pp. 586–591, 1991.