TRACKING MOVING OBJECTS ACROSS DISTRIBUTED CAMERAS …

i

TRACKING MOVING OBJECTS ACROSS

DISTRIBUTED CAMERAS

BY

NOR NADIRAH BINTI ABDUL AZIZ

A thesis submitted in fulfillment of the requirement for

the degree of Master of Science (Mechatronics

Engineering)

Kuliyyah of Engineering

International Islamic University Malaysia

JANUARY 2016

ii

ABSTRACT

The rise of crime has led to the increasing of demand on automated video surveillance

system due to the limitations in the ability of humans to vigilantly monitor the video

surveillance footage. Video surveillance system is an important tool used in detection

of snatch theft crime that includes detection and tracking of the objects which can

provide detailed information about the objects' appearance and their biometric

information. Tracking moving objects in multi-cameras environment is more

challenging than a single camera view due to variation in illumination conditions,

poses and viewing angles. Besides, there is no spatial continuity between cameras

with non-overlapping view, thus is more challenging. Most of existing tracking

methods perform well for single camera, but not for multiple cameras. Some of

available trackers that work well for multi-cameras environment have high

computational time. This thesis builds on prior studies to select the optimal features

from the object's appearance and to develop tracking algorithm for multiple non-

overlapping cameras view that can provide the optimal trade-off between accuracy

and speed. In this thesis, the method based on an adaptive Gaussian Mixture Model

and background subtraction to extract the foreground object is presented. The

proposed tracking algorithm is formulated based on visual appearance including Hue

colour, YCbCr colour, texture, shape and edge features extracted from the upper and

lower parts of body for correspondence management. Position cue is used in single-

camera tracking to reduce the computational cost. The comparison between the

effectiveness of the features is presented in the result section. The accuracy of the

proposed framework for tracking the moving objects based on frame-based

performance is very good, that is 95.97 percents with a speed of 43.967 frames per

second (fps) for single camera. For two and three non-overlapping cameras, the

overall accuracy based on frame-based performance is 99.29 percents and 99.73

percents with a speed of 26.30 fps and 17.54 fps respectively. The proposed algorithm

is reliable for real-time performance based on the experimental results.

iii

البحث ملخّص

يعُدُّ التتبع البصري في البيئة ذات الكاميرات الدتعددة من أكثر البحوث تطوراً في مجال الدتوفرة الدتتبعينللكائناتوحتي الآن لا يزال معظم .الحاسوب ، لاسيما في السنوات الأخيرة

؛وىذه الدراسة مبنية على دراسات سابقة .. مفاضلة بين الدقة والسرعةيسعون لتقديم ومعلوم أن تغطية . أمثللتحديد الدلاح الدثالية من الدظهر الدرئي للكائن من أجل توفير بديل

وموضع الاىتمام موجّو للمشاة من . وقد يحدث في التغطية تداخل،الكاميرات محدود للغاية-G-Mفهذا البحث تبني طريقة . ة جريمة السرقةحجات النارية، لدكافا والدر،الناس

Model والصورة الثنائية . حذف الخلفية للحصول على الكائن موضع العناية الرامية إلى. لإزالة الضوضاء أو الأصوات غير الدرغوب فيها ؛(مورفولوجية )يتم صقلها بعملية شكلية

والخطة ،سرعة الأداءوولعملية الدتابعة أو التتبع يتم اختيار الدلامح الدثلى بناء على الدقة ، والدلمس،ycbcrاللوغرثمية الدقترحو تعتمد على مزج الدلاح مع لون

تعقبويتمّ . وىي محسوبة من قبل لوغرثمية حسابيّة . وحدود الدلامح ،وحوافأوالشكلإدخال ،خروج ،عادي ،ودمج ،وفصل : وىي،الأجسام الدتحركة بناءً على خمس حالات

باستخراج الجزء العلوي والسفلي من متابعة الكائنات الدتحركةه الدتقدم يمكناللوغرثميبع توالت.الجسم لإجراء عملية مطابقة الدلامح، وفي قسم النتائج تتمّ الدقارنة بين فعالية الدلامح

الدستخدمة في التتبع اللوغريثمي وبين دقّة الإطار الدقترج لتتبع الأجسام الدتحركة اعتماداً على في الثانية 43.967 في الدائة،مع سرعة 95.29الأداء بدرجة جيد جداً إذا كان نسبتو

للكاميرا الواحدة ولكاميرتين أو ثلاث كاميرات غير متداخلة،فإنّ الدقّة الشاملة بناء على على 17.54FPSو FPS 26.30 بسرعة99.73في الدائة، و 99.29الأداء ىي

.التوالي ، وقد أظهرت النتائج التجربية فعاليّة ىذه الطريقة على أرض الواقع

iv

APPROVAL PAGE

I certify that I have supervised and read this study and that in my opinion; it conforms

to acceptable standards of scholarly presentation and is fully adequate, in scope and

quality, as a thesis for the degree of Master of Science (Mechatronics Engineering).

............................................................

Yasir Mohd Mustafah

Supervisor

............................................................

Amir Akramin Shafie

Co-supervisor

I certify that I have read this study and that in my opinion it conforms to acceptable

standards of scholarly presentation and is fully adequate, in scope and quality, as a

thesis for the degree of Master of Science (Mechatronics Engineering).

............................................................

Salmiah binti Ahmad

Internal Examiner

............................................................

Shazmin Aniza binti Abdul Shukor

External Examiner

This thesis was submitted to the Department of Mechatronics Engineering and is

accepted as a fulfillment of the requirement for the degree of Master of Science

(Mechatronics Engineering).

............................................................

Tanveer Saleh

Head, Department of Mechatronics

Engineering

This thesis was submitted to the Department of Mechatronics Engineering and is

accepted as a fulfillment of the requirement for the degree of Master of Science

(Mechatronics Engineering).

............................................................

Md Noor Hj Salleh

Dean, Kuliyyah of Engineering

v

DECLARATION

I hereby declare that this thesis is the result of my own investigations, except

otherwise stated. I also declare that it has not been previously or concurrently

submitted as a whole for any other degrees at IIUM or other institutions.

Nor Nadirah Abdul Aziz

Signature................................................... Date.............................................

vi

INTERNATIONAL ISLAMIC UNIVERSITY MALAYSIA

DECLARATION OF COPYRIGHT AND AFFIRMATION

OF FAIR USE OF UNPUBLISHED RESEARCH

Copyright © 2016 by International Islamic University Malaysia. All rights reserved.

DEVELOPMENT OF ALGORITHM FOR TRACKING MOVING

OBJECTS ACROSS DISTRIBUTED CAMERAS

No part of this unpublished research may be reproduced, stored in a retrieval system,

or transmitted, in any form or by any means, electronic, mechanical, photocopying,

recording, or otherwise without prior written permission of the copyright holder

except as provided below.

1. Any material contained in or derived from this unpublished research may

only be used by others in their writing with due acknowledgement.

2. IIUM or its library will have the right to make and transmit copies (print

or electronic) for institutional and academic purposes.

3. The IIUM library will have the right to make, store in a retrieval system

and supply copies of this unpublished research if requested by other

university and research libraries.

Affirmed by Nor Nadirah Abdul Aziz

........................................... .................................

Signature Date

vii

ACKNOWLEDGEMENT

Praise to the Almighty One, Allah (subhanahu wa ta'ala) for giving me the

opportunity to live in this world and giving me a good health that I, now capable to do

my daily job with peaceful mind. Besides, thanks to Allah that I able to fulfill my

responsibility to be servant of God. Alhamdulillah, praise to Allah for his guidance

and blessings in completing this thesis.

First of all, I would like to thank Assoc. Prof. Dr. Yasir Mohd Mustafah, my

project main supervisor for his valuable guidance and advice. Without his help, I

would not be able to complete this project successfully. I also would like to thank

another member of my supervisory committee, Prof. Dr. Amir Akramin Shafie for his

guidance to my thesis. There is no doubt that this work could not be successfully

completed without observations, comments and time consuming discussion

contributed by them. Besides, I also would like to thank all the lecturers who

indirectly helped me to complete this thesis. These special thanks are dedicated to

Deputy Dean of Postgraduate and Research, Prof. Dr. Erry Yulian Triblas Adesta,

Head of Mechatronics Engineering Department, Assoc. Prof. Dr. Tanveer Saleh, and

Postgraduate coordinator, Assoc. Prof. Dr. Nahrul Khair who has been giving

comments and guidance from time to time that made me able to write my thesis on

time.

Finally, I would like to express my deepest gratitude to my husband,

Muhammad Izad bin Yusoff, my parents, family, friends and lecturers for their

support, constructive suggestion and criticism. Without them, I would face difficulties

while doing this project.

viii

TABLE OF CONTENTS

Abstract ............................................................................................................................ i

Abstract in Arabic ..........................................................................................................iii

Approval Page ................................................................................................................iii

Declaration Page ............................................................................................................. v

Copyright Page ............................................................................................................... vi

Acknowledgement ........................................................................................................ vii

List of Figures ................................................................................................................ xi

List of Tables ................................................................................................................. xi

List of Abbreviations .................................................................................................xviii

List of Symbols ............................................................................................................ xix

CHAPTER 1: INTRODUCTION ................................................................................ 1

1.0 Introduction .................................................................................................. 1

1.1 Background of the Thesis............................................................................. 3

1.2 Problem Statement ....................................................................................... 5

1.3 Research Objectives ..................................................................................... 7

1.4 Research Methodology................................................................................. 7

1.5 Research Scope .......................................................................................... 10

1.6 Thesis Organization ................................................................................... 10

CHAPTER 2: LITERATURE REVIEW .................................................................. 11

2.0 Background ................................................................................................ 11

2.1 Related Approaches used for Object Detection ......................................... 12

2.1.1 Pre-processing .................................................................................. 12

2.1.2 Object segmentation ......................................................................... 12

2.1.2.1 Background subtraction ........................................................ 13

2.1.2.2 Optical flow .......................................................................... 15

2.1.3 Mathematical morphology ................................................................ 16

2.1.3.1 Fundamental operations in morphological operation ........... 16

2.1.3.2 Combination of fundamental operations in morphological

operation ............................................................................................. 17

2.1.4 Blob extraction ................................................................................. 18

2.2 Visual tracking ........................................................................................... 19

2.2.1 Related works on modeling .............................................................. 19

2.2.1.1 Related features used for single-camera view ..................... 19

2.2.1.2 Related features used for multiple-camera views ................ 31

2.2.1.3 Summary for related features and tracking algorithms ......... 36

2.2.1.4 Related methods used for modeling ...................................... 40

2.2.2 Related algorithms used for tracking ................................................ 44

2.2.2.1 Kalman Filter ....................................................................... 45

2.2.2.2 Particle Filter ........................................................................ 45

ix

2.2.2.3 Mean Shift Algorithm .......................................................... 46

2.2.2.4 CamShift Algorithm ............................................................ 47

2.2.2.5 Euclidean distance ............................................................... 48

2.2.2.6 Bhattacharyya coefficient .................................................... 48

2.2.2.7 Bhattacharyya distance ........................................................ 49

2.2.2.8 Homography transformation ................................................ 49

2.2.2.9 CSK Tracker ........................................................................ 50

2.2.2.10 Support Vector Machines (SVMs) ...................................... 50 2.2.2.11 Adaptive Boosting (Adaboost) ............................................ 51

2.2.2 Summary ........................................................................................... 55

CHAPTER 3: METHODOLOGY ............................................................................. 57

3.0 Background ................................................................................................ 57

3.1 Object Detection......................................................................................... 57

3.1.1 Background Modeling and Object Segmentation ............................. 58

3.1.2 Morphological Operation ................................................................. 60

3.1.3 Blob Extraction ................................................................................. 63

3.2 Object Tracking .......................................................................................... 67

3.2.1 Types of Features used in Modeling ................................................. 74

3.2.1.1 Position ................................................................................. 75

3.2.1.2 Colour (RGB) ....................................................................... 76

3.2.1.3 Colour (Hue) ......................................................................... 77

3.2.1.4 Texture ................................................................................... 78

3.2.1.5 Edge ....................................................................................... 80

3.2.1.6 Colour (YCbCr) .................................................................... 81

3.2.1.7 Shape ..................................................................................... 82

3.2.1.8 Distance Results for Various Type of Features .................... 82

3.2.2 Tracking within Single-Camera View .............................................. 88

3.2.2.1 Entering Case ........................................................................ 88

3.2.2.2 Leaving Case ......................................................................... 90

3.2.2.3 Normal Case ......................................................................... 91 3.2.2.4 Merging Case ........................................................................ 91

3.2.2.5 Splitting Case ........................................................................ 91 3.2.3 Tracking across Multiple-camera Views .......................................... 93

3.2.3.1 Entering Case ........................................................................ 94

3.2.3.2 Leaving Case ......................................................................... 96

3.2.3.3 Normal Case ......................................................................... 97

3.2.3.4 Merging Case ........................................................................ 98 3.2.3.5 Splitting Case ........................................................................ 98

3.3 Full Tracking Process of Proposed Algorithm for Multiple Cameras ...... 99

3.4 System Architecture ................................................................................. 102

3.5 Summary .................................................................................................. 103

CHAPTER 4: RESULTS AND DISCUSSION ...................................................... 104

4.0 Introduction .............................................................................................. 104

4.1 Performance of Objects Tracking Based on Accuracy and Speed ........... 104

4.1.1 Ground Truth Generation ............................................................... 106

x

4.1.2 Performance Metrics for Accuracy Performance ........................... 108

4.1.2.1 Frame-based Measurement ................................................. 108 4.1.2.2 Object-based Measurement ................................................. 111

4.1.3 Tracking within Single Camera View ............................................ 111

4.1.4 Tracking across Two Non-Overlapping Camera Views ................. 116

4.1.5 Tracking across Three Non-Overlapping Camera Views ............... 122

4.2 Real-Time Performance Evaluation for The Proposed System ............... 127

4.3 Comparison of Results with Existing Works ........................................... 129

4.3.1 Tracking within Single Camera ...................................................... 129

4.3.2 Tracking across Two Non-Overlapping Camera Views ................. 132

4.3.3 Tracking across Three Non-Overlapping Camera Views ............... 139

4.4 Summary .................................................................................................. 142

CHAPTER 5: CONCLUSION AND RECOMMENDATIONS ........................... 143

5.1 Conclusion ............................................................................................... 143

5.2 Recommendations .................................................................................... 148

REFERENCES .......................................................................................................... 150

PUBLICATIONS ...................................................................................................... 160

APPENDIX A: FIVE CASES FOR TRACKING WITHIN SINGLE-CAMERA

VIEW AND MULTIPLE-CAMERA VIEWS ............................................................ 162

APPENDIX B: RESULTS FOR TRACKING MOVING OBJECTS USING

PROPOSED ALGORITHM ....................................................................................... 171

APPENDIX C: GRAPHICAL USER INTERFACE (GUI) OF THE INTELLIGENT

SURVEILLANCE SYSTEM ...................................................................................... 184

APPENDIX D: TRACKING CODE FOR THREE NON-OVERLAPPING

CAMERAS ................................................................................................................. 185

APPENDIX E: COMPUTATION OF ACCURACY ................................................. 211

xi

LIST OF FIGURES

Figure No. Page No.

1.1 General framework for tracking across distributed camera views 5

1.2 Research methodology for the proposed system 8

2.1 Bag-of-Features method (Huang, Yang, & Qiao, 2012) 33

2.2 Edge Orientation Histograms as in (Liu & Zhang, 2007) 41

2.3 Illustration for block 1 and block 2 with 50% overlap 43

(Dalal & Triggs, 2005b)

2.4 Result from colour clustering as in (Lin & Huang, 2011) 44

2.5 Mean Shift algorithm unable to adapt with changes in scale 46

(Alexander & Abid, 2013)

2.6 Mean Shift algorithm unable to adapt with changes in orientation 47

(Snekha, Sachdeva, & Birok, 2013)

2.7 Person tracking using CamShift algorithm in outdoors environment 47

(Snekha, Sachdeva, & Birok, 2013)

3.1 Diagram for overall proposed algorithm for tracking algorithm for 57

distributed cameras

3.2 Diagram for Proposed Object Detection 58

3.3 Results for Background Subtraction 60

3.4 Examples for Morphological Operation using various operation 61

on large objects


on small objects


motorcyclist

3.7 Flowchart for Blob Extraction (Azhar Ibrahim, 2013) 64

3.8 Computation of a new bounding box for a broken mask 64

xii

3.9 Flowchart for computation of broken mask bounding box 65

3.10 Flowchart for computation of bounding box for removal for 66

shadow area

3.11 Examples for computation of bounding box for broken mask 66

and removal of shadow area

3.12 Top middle vertex position of the extracted human 68

(Azhar Ibrahim, 2013)

3.13 Draw bounding box for features selection 70

3.14 Flowchart for computation of small bounding box for feature 71

extraction

3.15 Centroid position of extracted objects 75

3.16 Steps for texture feature extraction 79

3.17 Steps for edge feature extraction 80

3.18 Full tracking process for the proposed Tracking Algorithm 100

3.19 Compare Object Position Diagram based on Figure 3.18 101

3.20 Compare Object Features Diagram based on Figure 3.18 102

3.21 CCTV Surveillance System Architecture 103

4.1 Results for generation of ground truth using ViPER-GT marking tool 107

4.2 Three matching cases (Nascimento & Marques, 2006) 110

4.3 Results for Tracking within Single Camera from Camera1 116

Video 3 (EPFL Datasets)

4.4 Results for Tracking within Single Camera from Camera2 Video 10 116

IIUM

4.5 Results for Tracking across Two Non-Overlapping Cameras for 121

Video 1 IIUM

4.6 Results for Tracking across Two Non-Overlapping Cameras for 121


4.7 Results for Tracking across Three Non-Overlapping Cameras 126

for Video 3 (EPFL Datasets)

xiii

4.8 Results for Single-camera Object Tracking using PETS2001 131

Dataset 1 Camera 1

4.9 Results of Tracking for Two Non-Overlapping Views using 135

Dataset Video E Lin and Huang (2011)

4.10 Results of Tracking for Two Non-Overlapping Views using 138

Dataset Chen, Huang and Tan (2011)

4.11 Results of Tracking for Three Non-Overlapping Views using Dataset 141

Chen, Huang and Tan (2012) and Chen, Huang and Tan (2014)

A.1 Entering case based on condition (a) in Section 3.2.2.1 162

A.2 Entering case based on condition condition (b) in Section 3.2.2.1 162

A.3 Entering case based on condition (c) in Section 3.2.2.1 164


A.5 Leaving case based on condition (a) in Section 3.2.2.2 165

A.6 Leaving case based on condition (a) in Section 3.2.2.2 165

A.7 Normal case based on condition (a) in Section 3.2.2.3 165

A.8 Normal case based on condition (b) in Section 3.2.2.3 166

A.9 Result of merging case 166

A.10 Result of splitting case 166

A.11 Entering case based on condition (a) in Section 3.2.3.1 167

A.12 Entering case based on condition (b) in Section 3.2.3.2 167


A.14 Entering case based on condition (d) in Section 3.2.3.1 168

A.15 Entering case based on condition (d) in Section 3.2.3.1 168

A.16 Entering case based on condition (d) Section 3.2.3.1 169

A.17 Leaving case based on condition (a) Section 3.2.3.2 169

A.18 Leaving case based on condition (b) in Section 3.2.3.2 169

A.19 Normal case based on condition (a) in Section 3.2.3.3 170

xiv

B.1 Results for Tracking within Single Camera from Camera1 Video 8 171

IIUM


IIUM

B.3 Results for Tracking from Camera2 Video 3 (EPFL Datasets) 171

B.4 Results for Tracking from Camera2 Video 6 (EPFL Datasets) 171


IIUM

B.6 Results for Tracking within Single Camera from Atrium Video 172

B.7 Results for Tracking within Single Camera from Video 1 PDRM 172








B.15 Results for Tracking across Two Non-Overlapping Cameras for 174

Video 2 Public




Video 4 Public


Video 5 Public


Video 7 IIUM


Video 8 IIUM

xv


Video 9 IIUM


Video 10 IIUM

B.23 Results for Tracking across Three Non-Overlapping Cameras 180

for Video 1 IIUM


for Video 6 (EPFL Datasets)


for Video 9 IIUM


for Video 8 IIUM


for Video 10 IIUM

xvi

LIST OF TABLES

Table No. Page No.

1.1 Gantt Chart 9

2.1 Features and tracking algorithms used for tracking objects within 29

single-camera view

2.2 Features and tracking algorithms used for tracking objects across 37

multiple-camera views

2.3 Features with their advantages and limitations 39

2.4 Tracking algorithms with their advantages and limitations 51

2.5 Summary of Selected Methods and Improvements of Methods 55

3.1 Examples for Extracted Upper Part of Body and Lower Part of Body 72

3.2 Examples for Results of Computation of Distance from Upper Part of 83

Body and Lower Part of Body

4.1 Tracking Performance Based on Individual Feature and Fusion of 112

Features with Position Feature

4.2 Overall Tracking Performance based on Hue Colour, YCbCr Colour, 113

Edge, Shape, Position and Texture Features for Single Camera

4.3 Frame-based Performance Measures for Human Pedestrians and 114

Motorcyclists Tracking using Proposed Algorithm for Single Camera

4.4 Object-based Performance Measures for Human Pedestrians and 116

Motorcyclists Tracking using Proposed Algorithm for Single Camera


Edge, Shape, Position and Texture Features for Two Non-Overlapping

Camera Views


Motorcyclists Tracking using Proposed Algorithm for Two

Non-Overlapping Camera Views


Motorcyclists Tracking using Proposed Algorithm for Two


xvii


Edge, Shape, Position and Texture Features for Three Non-Overlapping

Camera Views


Motorcyclists Tracking using Proposed Algorithm for Three



Motorcyclists Tracking using Proposed Algorithm for Three


4.11 Average Frame Rate for Single-camera Object Tracking using 128

Proposed Algorithm

4.12 Average Frame Rate for Two Cameras Object Tracking using 128

Proposed Algorithm

4.13 Average Frame Rate for Three Cameras Object Tracking using 128

Proposed Algorithm

4.14 Performance Comparison with Previous Works on Video PETS2001 130

Dataset 1 Camera 1

4.15 Performance Comparison on Video for Non-Overlapping Cases using 134

Dataset Video E Lin and Huang (2011)

4.16 Performance Comparison on Video for Two Non-Overlapping Cases 137

using Dataset Chen, Huang and Tan (2011)

4.17 Performance Comparison on Video for Three Non-Overlapping Cases 140

using Dataset in in Chen, Huang and Tan (2012) and Chen, Huang

and Tan (2014)

xviii

LIST OF ABBREVIATIONS

CCTV Closed-Circuit Television MSCR Maximally Stable Colour

Regions

ATM Automated Teller Machine HKM Hierarchical K-means

PDRM Polis Di Raja Malaysia SVMs Support Vector Machines

NKRA National Key Result Areas BTF Bidirectional Texture

Functions

GTP Government Transformation

Programme

CSK Circulant Structure of

Kernels

CPU Central Processing Unit LoG Laplacian of a Gaussian

FOV Field of View MHKF Multi-Hypothesis

Kalman Filter

LBP Local Binary Pattern DVR Digital Video Recorder

YCbCr Luminance, Chroma Blue,

Chroma Red

EM Expectation

Maximization

Blob Binary Large Object RAM Random Access Memory

RGB Red, Green, Blue IIUM International Islamic

University Malaysia

L*a*b Lightness, colour-opponent

dimensions

QVGA Quarter Video Graphics

Array

L*u*v CIE 1976 colour space CMC Cumulative Match

Characteristics

HSV Hue, Saturation, Value et al. (et alia): and others

CN Colour Names etc (et cetera): and so forth

pages that follow

CamShift Continuous Adaptive Mean

Shift Algorithm

e.g (exempligratia); for

example

2-D Two Dimensional n.d no date

DCT Discrete Cosine Transform no. number

EOH Edge Oriented Histogram vol. volume

PCA Principal Component

Analysis

i.e (idest): in other word

HOG Histogram Oriented Gradient BoF Bag-of-features

xix

LIST OF SYMBOLS

B(x) Binary value at pixel x 𝐿𝐵𝑃𝑏𝑛 LBP feature (b =1,...,5)

for target blob

𝐶𝑙𝑖 YCbCr colour property for

candidate blob 𝑂𝑡 Target blob

𝐶𝑙𝑛 YCbCr colour property for

target blob 𝑂𝑐 Candidate blob

𝐶𝑏𝑖 Chroma Blue for candidate

blob 𝑝𝑖 Centroid position for

candidate blob

𝐶𝑟𝑖 Chroma Red for candidate

blob 𝑝𝑛 Centroid position for

target blob

𝐶𝑏𝑛 Chroma Blue for target blob 𝑇𝑖 LBP texture model for

candidate blob

𝐶𝑟𝑛 Chroma Red for target blob 𝑇𝑛 LBP texture model for

target blob

d Distance for position, Hue,

Edge, Texture, YCbCr feature 𝑇𝑑 Threshold value for

position feature

𝐸𝑖 Edge property for candidate

blob 𝑇𝐻𝑢𝑒 Threshold value for Hue

colour feature

𝐸𝑛 Edge property for target blob 𝑇𝑇𝑇𝐸 Threshold value for

hybrid of Texture and

Edge features

𝑓𝑐 Current frame 𝑇𝐶𝑙_𝑡𝑜𝑝 Threshold value for

YCbCr colour feature for

upper part of body

𝑓𝑐−𝑚 Previous frame (m = 1,..., m) 𝑇𝐶𝑙_𝑏𝑜𝑡𝑡𝑜𝑚 Threshold value for

YCbCr colour feature for

lower part of body

𝑓𝑐−1 Previous frame (m =1) 𝑥𝑖 x-position for candidate

blob

𝐻𝑢𝑒𝑖 Hue colour for candidate blob 𝑦𝑖 y-position for candidate

blob

𝐻𝑢𝑒𝑛 Hue colour for target blob 𝑌𝑖 Luminance for candidate

blob

I(x) Intensity value at pixel x 𝑌𝑛 Luminance for target

blob

𝐿𝐵𝑃𝑏𝑖 LBP feature (b =1,...,5) for

candidate blob

1

CHAPTER 1

INTRODUCTION

1.0 INTRODUCTION

The Closed-Circuit Television (CCTV) camera or known as video surveillance system

is an important tool used in many applications including security purposes. CCTVs

are used in the verification of alarms, particularly in the detection of criminal

offences, such as snatch theft. Snatch theft employs rob-and-run tactics to steal

valuable items from a pedestrian. The offenders may work in a pair and ride a

motorcycle or, they may work alone or do not use a motorcycle. Snatch theft crime

has become more prevalent in this country and a measure must be taken to prevent

this crime from progressing. Some of snatch theft cases have caused fatalities in which

the victims have been dragged by the motorcycle while holding onto their handbag,

or, have been exposed to violence when they want to take justice by their own selves.

The rise of snatch theft crime and security threats in private or public areas has

led to the increasing demand of video surveillance system. Plus, the system is getting

cheaper nowadays. The videos that are directly taken from CCTVs are transmitted to a

video monitor or several monitors and recorded for further analyses. In many

premises, human operator is assigned to monitor the video continuously to detect any

suspicious activity. For snatch theft crime, the person's appearance that related to the

crime itself is captured by the CCTVs. Thus, it can provide important information

about the criminals including a detailed view on the perpetrators' appearance and their

size which can help the investigator to complete a thorough and fair investigation

process. CCTVs are deployed to monitor many areas such as the vicinity of banks,

2

schools, shops or pedestrian streets. Most of snatch theft cases occur on the street

which involves human pedestrian and motorcyclist. Meanwhile, the victims are adults

since they usually carry valuable items which are desired by the perpetrators.

Until now, the process of analyzing the videos is done manually by human

operator. The conventional method of utilizing human operator can be expensive and

inefficient. The process of analyzing series of videos can be time consuming. Besides

that, the short concentration span of a normal human operator during reviewing linear

video data content can result in disregarding important crime details. Thus, an

automated video surveillance system is essential to ease the burden on human

operators.

Automated video surveillance system intends to exploit the videos captured

from the CCTVs using software to automatically identify the objects. The automated

system involves processing methods on the visual images such as computer vision

techniques that give output without human interruption. Computer vision techniques

include method of acquiring, processing and analyzing visual images from a real

world and converting the raw data into a form that can be processed by the machine.

Applications for surveillance system in snatch theft crime detection cover a large

scope in machine vision, including detection and tracking of the object that may be

related to the crime itself. The aim of machine vision research is to provide computers

with humanlike perception ability. The application has the ability to detect the moving

objects that can reduce the amount of data needed for processing and applicable for

real-time performance. Meanwhile, tracking of moving objects can give a unique

identity to each of the object detected in the video consistently. From the tracking

results, it can provide important information about the objects such as the object's

appearance, and biometric information including the object's height.

3

1.1 BACKGROUND OF THE THESIS

Research on snatch robbery detection and tracking from video footage has scarcely

been addressed before. The only work on snatch theft detection found was by Ibrahim,

Mokri, Siong, Mustafa and Hussain (2010). However, the proposed algorithm is not

applicable in real-time and only a single camera is used. However, in this thesis, the

main concern is not to detect the behavior of the object, but to track the objects related

to the crimes. Each of the objects will be assigned to a different label based on the

objects' appearance.

Recent works on tracking moving objects using multiple cameras is

increasingly popular. The multi-cameras environment covers more physical space than

a single camera view which provides a more comprehensive view about the crime

scene. The coverage of multi-cameras surveillance is defined by the cameras' field of

view (FOV) which is either overlap or non-overlap depending on the direction of the

camera being installed. Based on the literature review, several approaches have been

proposed to track objects across disjoint camera views. This thesis intends to improve

the existing techniques for tracking moving objects across distributed cameras in order

to enhance the accurateness and robustness of the video surveillance system that is

applicable for real-time performance.

In general, the framework of a multi-camera surveillance system encompasses

a large scope in machine vision that includes background modeling, objects detection,

moving objects tracking, and requires fusion of information from the camera

networks. The foremost task in many surveillance applications is to detect the objects

of interest as they appear within the camera view. The detection of moving objects is

important for target tracking. Object detection is a challenging task especially for

distributed cameras as the objects belonging to the same class, such as human, might

4

significantly differ in appearance due to clothing, illumination, pose and camera

parameters factor (Chellappa, Sankaranarayanan, Veeraraghavan, & Turaga, 2010).

Thus, it is important to have a robust object detection technique that can differentiate

the foreground object from the background image even in cluttered backgrounds

under challenging illumination conditions (Azhar Ibrahim, 2013).

When the moving object is successfully detected in the video surveillance

system, the next step is to track the objects within the disjoint views. It is important to

accurately track the moving objects across the different views as it can help the

enforcers to get the visual tag of the criminals from various sources and track them

within a short concentration span. This is done so as to eliminate disregarding

important crime details. However, tracking moving objects in multi-cameras

environments is more challenging compared to single camera view. This is due to

different illumination conditions, viewing angles and poses between different camera

views (Chen, Huang, & Tan, 2011). Besides, there is no spatial continuity between

cameras that have non-overlapping views. Although there have been many research on

visual tracking to improve the existing techniques for multi-cameras environment,

most of them fail to encounter trade-off problems between accuracy and speed. This

area of study needs to be further investigated especially in several areas such as

background modeling and object segmentation; moving objects feature extraction, and

selection of optimal features that can optimize the tracking performance. This thesis

focuses on object detection and tracking for multiple non-overlapping cameras

especially for human pedestrian and motorcyclist that is applicable for real-time

application and to solve the trade-off problem. Figure 1.1 shows the general

framework for tracking objects across distributed cameras view.

5

Figure 1.1: General framework for tracking across distributed camera views.

1.2 PROBLEM STATEMENT

Tracking moving objects in multi-cameras environment is more challenging than a

single camera view due to different illumination conditions, viewing angles and poses

(Chen, Huang, & Tan, 2011). Most of available trackers work well for single camera,

but not for multiple cameras. Some algorithms that work well for multi-cameras

environment have high computational time. Cameras are located at different places,

thus, have different coverage area which is overlapping or non-overlapping. Tracking

moving objects in a non-overlapping view is more challenging than overlapping view

because there is no spatial continuity between the cameras. The object that is tracked

in the current camera might be wrongly tracked in the next camera (Hsu, Yang, &

Shih, 2013).

Although there have been many research on visual tracking to improve the

existing techniques, most of them fail to encounter trade-off problems between

accuracy and speed (Zhang, Dibeklio˘glu and Matted, 2014). Although their trackers

provide very good levels of accuracy, they are often impractical because of their high

computational requirements. Tracking objects in a video is computational costly due

to the amount of data contained in the video (Sangam, Lal, & Diwakar, 2013).

Selection of features from the object's appearance can affect the tracking

accuracy result. Some of features are sensitive to illumination variation. For example,

TRACKING MOVING OBJECTS ACROSS DISTRIBUTED CAMERAS …

Documents

Transcript of TRACKING MOVING OBJECTS ACROSS DISTRIBUTED CAMERAS …