Comparison of complex background subtraction algorithms using a fixed camera

Comparison of complex background

subtraction algorithms using a fixed camera

Geoffrey Samuel

PhD ResearcherIntelligent Systems and Robotics Research Group (ISR)

Creative TechnologiesUniversity of Portsmouth

Geoffrey Samuel www.GeoffSamuel.com

Comparison of complex background subtraction algorithms using a fixed camera

Intro Background subtraction is a important

and vital step for computers to understand and interpreter a real-world scene

It allows a computer to ignore a background so to concentrate on a foreground object



Hypothesis Each background subtraction algorithm

will have its advantages and disadvantages, and that looking and comparing these with a real-world situation, it would be possible to pick one algorithm or a method of combining algorithms to produce a algorithm capable of balancing speed with quality.



The Goal Test and evaluate the quality and speed

of existing background subtraction algorithms on a complex background with different everyday motions, and to compare the results with those of the extracted “Ground Truth”



Complex BackgroundStatic Background:-Background does not contain any secondary “unwanted” motion. Controlled environment.

Complex Background:-Background contains secondary “unwanted” motion such as the winds effect on trees or blinds.Real-world data.



Synthetic Test DataAdvantages:

• Automatically got the “Ground Truth”.• More control over each test clip.

Disadvantages:• Manual frame by frame “Ground Truth”

extraction.• Added artefacts from the Chroma keying

and compositing.



The Experiment To Create a set of synthetic data with

the “Ground Truth”

To test different motions with each background subtraction algorithm

To Compare the results of each algorithm with that of the “Ground Truth”



The Motions 7 everyday motions were chosen:

DrinkingJoggingPicking up walletScratching headSitting downStanding upWalking

Each motion started on the left of the screen and concluded on the right.


Comparison of complex background subtraction algorithms using a fixed cameraCreating the test scenarios

Green Screen

Back Ground

Green Screen with actor

Final Composite “Ground Truth”



Back Plate Difference

│framei – backplate│>Ts

The Algorithms

50



Frame Difference

│framei – framei-1│>Ts

The Algorithms

50



Approximate median

(x = ( framei - framei-1 – framei-2 . . .framei-n ) >

Ts )→ {background += 1}→ {background -= 1}

The Algorithms



Mixture of Gaussians

frame(it = μ) = Σi=1 ωi,t .ț(μ,o)

The Algorithms

k



Measuring the QualityCompare the Per-Pixel value ofeach frame with the “Ground Truth”

(0,0) (768,0)

(768,576)(0,576)

(0,0) (768,0)

(768,576)(0,576)



Results - Quality

Test MotionsBackplate Difference Frame Difference Approximate Median Mixture of Gaussian% of image # of pixels % of image # of pixels % of image # of pixels % of image # of pixels

Drinking 90.78% 401577.3019 82.12% 363282.5031 89.52% 396024.7107 83.78% 370625.2327

Jogging 88.24% 390349.3529 88.88% 393194.9412 92.14% 407602.3824 88.20% 390146.7941

Picking up Wallet 91.26% 403717.114 88.22% 390256.9035 83.40% 368940.5088 90.19% 398979.9737

Scratch head 88.18% 390065.7255 84.87% 375422.2549 90.56% 400599.9216 86.15% 381117.049

Sitting down 88.51% 391528.6796 80.07% 354204.932 82.28% 363994.2039 81.68% 361327.3981

Standing up 89.40% 395491.6311 83.82% 370787.165 80.99% 358290.4563 83.78% 370631.6893

Walking 88.47% 391373.5094 89.81% 397309.3396 94.22% 416820.1321 90.01% 398195.3396

Most correct pixels Most incorrect pixels



Results - Quality

Drinking Jogging Picking up Wallet

Scratch head Sitting down Standing up Walking70.00%

75.00%

80.00%

85.00%

90.00%

95.00%

100.00%

Percent of correctly identified pixels

Backplate DifferenceFrame DifferenceApproximate MedianMixture of Gaussian

Test Motions

Aver

age

Perc

ent o

f cor

rect

ly id

entif

ied

pixe

ls p

er fr

ame



Results - Speed

Test MotionsBackplate Difference(Average of 100 times)

Frame Difference(Average of 100 times)

Approximate Median(Average of 100 times) Mixture of Gaussian

Drinking 0.0507 0.0004 0.3301 10.6954

Jogging 0.0507 0.0025 0.0691 10.8219

Picking up Wallet 0.0492 0.0819 0.0730 12.2895

Scratch head 0.0450 0.0850 0.0718 10.6132

Sitting down 0.0420 0.0692 0.0662 10.8503

Standing up 0.0416 0.0747 0.0529 12.7196

Walking 0.0319 0.0129 0.0541 10.5202

“Fastest” Algorithm “Slowest “Algorithm



Results - Speed


Scratch head Sitting down Standing up Walking0.0000

2.0000

4.0000

6.0000

8.0000

10.0000

12.0000

14.0000

Average time to process per frame

Backplate DifferenceFrame DifferenceApproximate MedianMixture of Gaussian

Test Motions

Aver

age

proc

essi

ng ti

me

per f

ram

e in

Sec

onds

(run

100

tim

es)



Results - Speed


Scratch head Sitting down Standing up Walking0.0000

0.0500

0.1000

0.1500

0.2000

0.2500

0.3000

0.3500

Average time to process per frame

Backplate DifferenceFrame DifferenceApproximate Median

Test Motions

Aver

age

proc

essi

ng ti

me

per f

ram

e in

Sec

onds

(run

100

tim

es)

...now ignoring the Mixture of Gaussian speed results



Conclusion Backplate difference was the fastest and

produce the highest results in 4 out of 7 tests.

Frame difference was the ONLY algorithm to correctly remove the complex background, but couldn't correctly identify the foreground element.



ConclusionFrame Difference :-

Correctly Removed Complex BackgroundIncorrectly Removed inside of Subject

Backplate Difference :-Correctly Identified SubjectIncorrectly kept Complex Background



Taking it furtherA new method that incorporated both thespeed of updating to remove thebackground and yet the knowledge of thebackground to properly extract the wantedforeground element.

Theory Framework idea:

Frame Difference Backplate Difference

ƒComplex background removed



Where can this lead? Application of this technology could be

used in:

GamesSurveillanceMesh reconstruction and silhouette

extractionVarious computer vision tasks



Any Questions?



Acknowledgments

UK Engineering and Physical Science Research Council

Seth Benton for his Matlab code



Thank you for your time

[email protected]

www.GeoffSamuel.com

Comparison of complex background subtraction algorithms using a fixed camera

Documents

Transcript of Comparison of complex background subtraction algorithms using a fixed camera