Introduction Image Taxonomy Conclusion Bibliography

Image taxonomy in Olena

Christophe Berger

LRDE – EPITA Research and Development Laboratory

May 24, 2006

Image taxonomy in Olena 1 / 30

Introduction Image Taxonomy Conclusion Bibliography

Outline

1 Introduction

2 Image TaxonomyTypesHierarchiesPluging rulesMorphers

3 Conclusion

Image taxonomy in Olena 2 / 30

Introduction Image Taxonomy Conclusion Bibliography

Olena

About Olena:

Generic image processing library.

Developed at the LRDE [ole].

Latest release, april 2004.

Based on SCOOPv1 paradigm [BDLG+03].

Image taxonomy in Olena 3 / 30

Introduction Image Taxonomy Conclusion Bibliography

Image Type Taxonomy

Taxonomy

The classification of organisms in an ordered system that indicatesnatural relationships.

In our context, the organisms are image types.

Image taxonomy in Olena 4 / 30

Introduction Image Taxonomy Conclusion Bibliography

Our Goals

Our goals are:

Having many and efficient types in the next release.

Enhancing type coherance within the hierarchy.

Enforcing code factoring through a high level of genericity.

Achieving to properly implement image abstractions.

Image taxonomy in Olena 5 / 30

Introduction Image Taxonomy Conclusion Bibliography

From SCOOP1...

SCOOP1

Figure: SCOOP inheritance model [Ger06]

masked image could be an image 2d, but we don’t know.

Image taxonomy in Olena 6 / 30

Introduction Image Taxonomy Conclusion Bibliography

...to SCOOP2

The SCOOP2 paradigm [Ger06].

Figure: SCOOP2 inheritance model [Ger06]

Now masked image inherits from image entry which is plugged onthe proper hierarchies.

Image taxonomy in Olena 7 / 30

Introduction Image Taxonomy Conclusion Bibliography Types Hierarchies Pluging rules Morphers

Modus operandi

Bring together interesting and relevant image types.

Find out appropriate abstract hierarchies.

Bring forth general image properties.

Define the pluging rules to link the image types with theabstractions.

Image taxonomy in Olena 8 / 30

Introduction Image Taxonomy Conclusion Bibliography Types Hierarchies Pluging rules Morphers

Primitive Image Types

We want to define some primitive image types,

to test the hierarchies, SCOOP2 and“properties”.

to give the user the basic types he needs.

They have some particularities that make them special andinteresting to work on.

Image taxonomy in Olena 9 / 30

Introduction Image Taxonomy Conclusion Bibliography Types Hierarchies Pluging rules Morphers

Basic 2D image

Basic 2D image: image 2d<T>

Definition

T is the pixel value type.

Bounding box.

Representation

ima = {data = [...2D buffer of T ...], bbox}

Example

2D Bounding box example

from (8, 8) with dimension (4, 6)

from (8, 8) to (12, 18)

Image taxonomy in Olena 10 / 30

Introduction Image Taxonomy Conclusion Bibliography Types Hierarchies Pluging rules Morphers

Functional image

Functional image: fun image<T, G>

Definition

G is the image grid type.

T is the pixel value type.

Data given by the function f : point → value.

f is defined on a domain Df , and bbox ⊆ Df .

A specific point in image cannot be modified.

Representation

ima = {f : p → v , bbox}

Example

ima = {f = (r , c)→ r + c, bbox = {(3, 3) to (5, 7)}}

Image taxonomy in Olena 11 / 30

Introduction Image Taxonomy Conclusion Bibliography Types Hierarchies Pluging rules Morphers

point, value set image

Point, value set image: pvset image<T, G>

Definition

G is the image grid type.

T is the pixel value type.

Data stored in a vector< pair<point, value> >

Bounding box.

Representation

ima = {vec = [...(p, v)...], bbox}

Example

ima = {vec = [(0, 1), 2); (0, 2), 4); (1, 2), 3)], bbox ={(0, 0) to (2, 3)}}

Image taxonomy in Olena 12 / 30

Introduction Image Taxonomy Conclusion Bibliography Types Hierarchies Pluging rules Morphers

Constant image

Constant image: cst image<T, G>

Definition

G is the image grid type.

T is the pixel value type.

One unique pixel value but value wise mutable.

Bounding box.

Example

ima = {val = 3, bbox = {(3, 5) to (8, 7)}}We can write: ima.value(3) ← 5

Image taxonomy in Olena 13 / 30

Introduction Image Taxonomy Conclusion Bibliography Types Hierarchies Pluging rules Morphers

Abstract hierarchies

We specified basic types, we now have to characterise them.

We will use abstractions from image processing area.

Finer are the abstraction hierarchies, finer are the characterisation.

We are looking for orthogonal hierarchies.

Image taxonomy in Olena 14 / 30

Introduction Image Taxonomy Conclusion Bibliography Types Hierarchies Pluging rules Morphers

Type of support / domain

Type of support / domain:

image

regular_image graph_image

image_1d Image_2d image_3d

image_2d_rec image_2d_hex

Figure: Image defined by its support type

Example

image 2d<int> is an image 2d rec.

Image taxonomy in Olena 15 / 30

Introduction Image Taxonomy Conclusion Bibliography Types Hierarchies Pluging rules Morphers

Image contents

Image contents:

image

greylevel_image labelled_imagecolor_image field_image data_image

...

Figure: Image defined by its contents

Example

image 2d<rgb3x8> is a color image.

Image taxonomy in Olena 16 / 30

Introduction Image Taxonomy Conclusion Bibliography Types Hierarchies Pluging rules Morphers

Nature of values

Nature of values:

image

scalar_image

boolean_imagevectorial_image

structured_image string_image

...

Figure: Image defined by its values

Example

image 2d<rgb3x8> is a vectorial imagewhile:

image 2d<color name> is a string image (ima[p] = “red ′′).

image 2d<color wavelen> is a scalar image (ima[p] = 355).

Image taxonomy in Olena 17 / 30

Introduction Image Taxonomy Conclusion Bibliography Types Hierarchies Pluging rules Morphers

Point wise accessibility

image

pw_accessible_image not_pw_accessible_image

Figure: Point wise accessibility

Example

image 2d ima[p] works because it provides a buffer access.

pvset image ima[p] does not work because we do not want tobrowse the entire set of (p, v)!

Image taxonomy in Olena 18 / 30

Introduction Image Taxonomy Conclusion Bibliography Types Hierarchies Pluging rules Morphers

Value wise accessibility

image

vw_accessible_image not_vw_accessible_image

Figure: value wise accessibility

Consequences

ima.value(value&) ← valueima.value(value&) const → {values}

Example: ima.value(3) = 5

1 1 22 3 3

→ 1 1 22 5 5

This is not availlable with image 2d because we do not store the function f : v → {p}/ima[p] = v .It works with cst image.

Image taxonomy in Olena 19 / 30

Introduction Image Taxonomy Conclusion Bibliography Types Hierarchies Pluging rules Morphers

Image constness

image

const_image mutable_image

Figure: Constness

Consequences

mutable image

lvalue& op[](const site& s)const rvalue& op[](const site& s) const

const image

const rvalue& op[](const site& s) const

Image taxonomy in Olena 20 / 30

Introduction Image Taxonomy Conclusion Bibliography Types Hierarchies Pluging rules Morphers

Properties

What is a property?

A property is a notion, an abstraction.

Properties are seen in implementation as concrete types.

Example

image → value

image2d<int> → value=int

Image taxonomy in Olena 21 / 30

Introduction Image Taxonomy Conclusion Bibliography Types Hierarchies Pluging rules Morphers

Pluging to the appropriate hierarchies

Properties examples

Properties are deduced from types.

image → value

image → grid

grid → point

grid → iterator

We need:

to list and define relevant properties for our types.

to express the rules to allow the abstraction of a givenhierarchy to be deduced from a set of image properties.

a static mechanism that plugs to the appropriate hierarchies.

Image taxonomy in Olena 22 / 30

Introduction Image Taxonomy Conclusion Bibliography Types Hierarchies Pluging rules Morphers

First example

Properties of image 2d<T>value T

grid Grid2d

point Point2d

site Point2d

size BBox2d

mutable true

pwra true

vwra false

Abstract hierarchies describing image 2d<T>

image 2d rec

pw accessible image

mutable image

case of T is int: data image and scalar image

case of T is rgb3x8: color image and vectorial image

Image taxonomy in Olena 23 / 30

Introduction Image Taxonomy Conclusion Bibliography Types Hierarchies Pluging rules Morphers

Second example

Properties of cst image<T, G>value T

grid G

site G::site

point G::point

mutable true

pwra false

vwra true

Abstract hierarchies describing cst image<T, G>

vw accessible image

mutable image

case of T is int: data image and scalar image

case of G is Grid2d: image 2d rec

Image taxonomy in Olena 24 / 30

Introduction Image Taxonomy Conclusion Bibliography Types Hierarchies Pluging rules Morphers

What is a morpher?

Morphers are:

Special types of images.

Modify sometimes properties of an image.

Example

A morpher m applied on an image ima gives a new imageima′ = m(ima).A morpher can be seen as a function, but creates a new imageusing the properties of ima.Exemple: a particular morpher<I, aux params>.

Image taxonomy in Olena 25 / 30

Introduction Image Taxonomy Conclusion Bibliography Types Hierarchies Pluging rules Morphers

Morpher example: image seen through a function

Definition: image through fun<I, fun>

ima′ = {ima; f : value → value}.

Properties

output output(F, I::value)

input I

mutable false

pwra I::pwra

... and so on...

Example

image<int> ima = [ 2, 4, 6, 1, 0, 8]function f (i) = i + 3ima’ = [ 5, 7, 9, 4, 3, 11]

Image taxonomy in Olena 26 / 30

Introduction Image Taxonomy Conclusion Bibliography Types Hierarchies Pluging rules Morphers

Morpher example: image seen through a bijective function

Definition: image through bijective fun<I, bijfun>

ima′ = {ima; f = {fdir : value → value, finv : value → value}}.

Properties

output output(F, I::value)

input I

mutable true

pwra I::pwra

... and so on...

Example

ima=[ 2, 4, 6, 1], fdir (i) = i + 3, finv (i) = i − 3consequently ima’ = [ 5, 7, 9, 4]then ima′[p] = 5/p = 2ndelement ⇒ ima′ = [5, 5, 9, 4] butima = [2, 2, 6, 1].

Image taxonomy in Olena 27 / 30

Introduction Image Taxonomy Conclusion Bibliography

Noticed ambiguities

This taxonomy led to:

ambiguities detection

more relevant and precise definitions

Example:

Grids: regular v.s. graph.

Image contents v.s. storage type.

Image contents v.s. Contents nature.

Point wise, value wise and constness.

Image taxonomy in Olena 28 / 30

Introduction Image Taxonomy Conclusion Bibliography

Work relevance

To conclude:

Innovative and proheminent work.

Starting point for the next release.

Documentation.

Study of the best hierarchy before starting the code.

Assurance of global coherance.

Image taxonomy in Olena 29 / 30

Introduction Image Taxonomy Conclusion Bibliography

Bibliography I

Nicolas Burrus, Alexandre Duret-Lutz, Thierry Geraud, DavidLesage, and Raphael Poss.A static C++ object-oriented programming (SCOOP)paradigm mixing benefits of traditional OOP and genericprogramming.In Proceedings of the Workshop on Multiple Paradigm withOO Languages (MPOOL), Anaheim, CA, USA, October 2003.

Thierry Geraud.Advanced static object-oriented programming features: Asequel to scoop, 2006.

Olena library http://olena.lrde.epita.fr.

Image taxonomy in Olena 30 / 30