Object-based Image Representation

Dr. B.S. Manjunath

Sitaram Bhagavathy

Shawn Newsam

Baris Sumengen

Vision Research Lab

University of California, Santa Barbara

Object-based Image Representation 2

Outline

• Context and Objective

• Introduction

• Object Extraction and Description

• Time series object coding

• Future research ideas

• Conclusion

Object-based Image Representation 3

Context

Large-scale Image Database

user

Query

Retrieved images

Query example: “Give me all images similar to image X.”

Object-based Image Representation 4

Objective

To develop an object-based image representation

scheme in order to facilitate the following:

• Faster access of data in the context of object-based querying

• Reducing required storage space for images

• Relating maps and geographical aerial images

• Study of spatio-temporal relationships in/among aerial images

Note: Although our dataset consists of aerial images, we expect the scheme to be useful for other image datasets as well.

Object-based Image Representation 5

The Object-based Approach

Assumptions:

• Useful information (for searching) in images is concentrated in smaller regions termed objects.

• Objects are mostly homogeneous in color and texture and can be characterized thus.

• Most queries on image databases are in terms of objects; e.g. “Give me all images having a brown field.”

Objects from the imageAn aerial image

Examples of objects

Object-based Image Representation 7

Why Object-based approach?

• Uses semantic information for querying (user friendly)

• Efficient description of images– We ignore portions of images that would not be used for querying

• Potential reduction of storage space– Store images as collections of objects– Redundancy removal in time-series of objects

Object-based Image Representation 8

Past Research

• Object Extraction: Identify and extract semantic objects from aerial images. – So far, we have done this manually

– Working on automatic segmentation using semantic models (Sumengen)

• Object Description: Find efficient descriptors for the objects (Bhagavathy)– Shape: binary alpha plane

– Dominant Colors (Deng, Manjunath)

– Dominant textures

9

RGB to LUV conversion

Take means of clusters

K-means clustering

Convert means to RGB

object

Dominant color feature

24 Gabor filters

K-means clustering

Take means and

percentages

object

Dominant texture feature

24-dim outputs

10

Time-series object codingObjective: To apply object-based video coding (based on MPEG-4) techniques for coding time-series of objects.

Object-based Image Representation 11

Future Research Possibilities

• Storage space issues

– describe image as a collection of objects

– reconstruct image from its objects

• Relation between maps and aerial images

– maps have information, images have data

• Spatial and temporal relationships among objects

– variation of objects with time

– spatial querying (Newsam)

• Application in wireless networks

Object-based Image Representation 12

Conclusion

• The object-based approach enables semantic querying which is more user-friendly

• Time-series compression of objects reduces required storage space for large images

• Potential to reduce required bandwidth for wireless transmission of image information

• Enables the study of temporal change in images

• Enables spatial querying