(Reading Group) Saliency Detection: A Boolean Map Approach

B31XM Advanced Image Analysis 1

Saliency Detection: A Boolean Map Approach IEEE International Conference on Computer

Vision (ICCV), 2013Jianming Zhang , Stan Sclaroff

Department of Computer Science, Boston University

Team Members : H.Kidane, I.Sadek, M.Elawady Heriot Watt University

School of Electrical and Physical Sciences


Outline

• Introduction• Related work• Methodology• Experiments• Conclusion• Future Work


Outline



• Goal– Proposing a Boolean map based saliency . An image is represented in a set

of binary images by randomly thresholding the image’s color channel. based on figure ground segregation

• What is saliency !– Saliency at a given location = how different this location is from its

surround color, orientation, motion, depth etc (Koch an Ullman, 1985, Itti et al. 1998). This is usually called (Bottom up saliency)

• Applications– Image segmentation – Object recognition– Visual tracking

Introduction


Introduction

Feature Search

Fast & Effortless

Conjunction Search

Slow& Effortful

Vis

ual

In

pu

t

Sal

ien

cy M

ap

Visual Saliency Map


Reaction time vs. number of distractors

Introduction

Feature Search

Conjunction Search

Rea

ctio

n t

ime

to f

ind

th

e ta

rget

# of distractors

(Koch & Ullman 1985, Wolfe et al 1989, Itti & Koch 2000)


• Figure ground segregation:

– It is known as identifying the figure

from the background

• This image can be perceived as:– a vase shape in front of a black

background– two black faces on

a white background

Introduction

Rubin's Vase


Outlines



Related Work

Method Limitation

Center surround difference

Cannot detect large salient region efficiently

Scale variant

The negative logarithm of probability

Hierarchical decomposition

Spectral domain analysis

Machine learning

Methods based on topological structure information

Strong influence on visual attentionScale invariant


Outline



Methodology

Input Image

Boolean Maps

Attention Maps

Saliency

Mean of Attention Maps


It is generated by randomly thresholding an input image I

Where

donates feature map of I

Randomly generated threshold in the range [0, 255]

CIE lab color space (perceptual uniformity)

Boolean MAP

)),(( ITHRESHBi

)(I

Methodology


• the attention map A(B) is computed based on Gestalt Principle for figure-ground segregation from B

• Gestalt Principle: surrounded regions are more likely to be perceived as figure

Attention Map

Methodology


• Given a Boolean map B, and attention Map A(B), the saliency is modeled by the mean attention map given by

A

dBIBpBAA

)/()(

Methodology


Methodology


Outline



Experiments


ExperimentsEye Fixation Prediction

Implementationdetails

Width of image resizing 600

Width of kernel for opening operation 5

Sampling step size 8

Width of kernel for dilation operation 7

Width of kernel for opening operation(before Gaussian blurring)

23

Standard deviation of Gaussian blurring

20

Removing Small peaks on mean attention map

http://mentormate.com/

http://mentormate.com/


Datasets

MIT 1003

Toronto 120

Kootstra 100

Cerf 181

ImgSal 235

http://www.cse.cuhk.edu.hk

Evaluation Metric

AUC

Shuffled AUC

Border CutCenter-Bias

Sampling


http://www.cse.cuhk.edu.hk/


Original

GT

BMS

ΔMQDCT

SigSal

LG AWS

HFT

CAS

Judd

AIMGBVS

Itti



Optimal average shuffled-AUC with

corresponding Gaussian blur STD

2012 2012 2012 2011 2013 2012 2009 2009 2007 1998

Less Background Distraction

No Multi-Scale Processing



Runtime performance

Programming Language

C++

Image Size 600x400

Processor 2.5 GHz

Running OS Windows

Memory 2 GB

http://runtime.bordeaux.inria.fr

CAS 78 LG 13

AWS 10 Judd 6.5

AIM 4.8 GBVS 1.1

ΔQDCT 0.49 Itti 0.43

HFT 0.27 SigSal 0.12

BMS 0.38


http://runtime.bordeaux.inria.fr/

ExperimentsSalient Object Detection

High Blurred

BinaryImage

Width of kernel for opening operation for boolean maps is modified to 13

Turning off the dilation operation for attention maps

Post-processing for mean attention map using (opening, closing) operations with kernel size (5)

Binarizing saliency map at a fixed threshold

Object Level Segmentation

30B31XM Advanced Image Analysis


Original

GT

BMS

GSSP

HSal

RC

FT

CAS

HFT




ASDDataset


Outlines



Conclusion

BMS has powerful advantage in surroundence

aspect•Helpful in figure-ground segregation

BMS is only model that consistently achieves the state-of-art performance

•Best results in different five eye-tracking datasets

•Proper results in salient object detection


Outlines



Future Work

• Not only color channels• Feature channels (i.e. orientation, depth,

and motion)

Improve the effectiveness of BMS

• Integrating more saliency cues (i.e. convexity, symmetry, and familiarity) instead of current one (eliminating regions that touch image borders)

Improve the attention map computation

(Reading Group) Saliency Detection: A Boolean Map Approach

Engineering

Transcript of (Reading Group) Saliency Detection: A Boolean Map Approach