AUGMENTED REALITY

Antonino Furnari

furnari@dmi.unict.it

http://dmi.unict.it/~furnari

IPLab - Image Processing Laboratory

Dipartimento di Matematica e Informatica

Università degli Studi di Catania

http://iplab.dmi.unict.it

Computer Vision A.Y. 2014-2015

AUGMENTED REALITY

“a live copy, view of a physical, real-world environment whose elements are augmented (or supplemented) by computer-generated sensory input such as sound, video,

graphics or GPS data”

“it is related to a more general concept called mediated reality, in which a view of reality is modified (possibly

even diminished rather than augmented) by a computer”

“as a result, the technology functions by enhancing one’s current perception of reality.”

wikipedia

APPLICATIONS

VISION BASED AUGMENTED REALITY

Computer Vision allows to create augmented realityapplications by superimposing 2D or 3D contents on thescene;

In order to do so we need to:

1) detect and track the area where to show the content;

2) estimate its 3D position in the real world;

3) render the 2D/3D content according to the estimatedposition and the inferred geometry of the scene;

Two main technologies:

fiduciary markers;

markerless (i.e., object detection).

SOME HISTORY

The term “augmented reality” appears since the1940s;

The first augmented head mounted display is inventedby Ivan Sutherland in 1968;

First systems using mobile devices, internet andgeolocalization appear in the 90s;

Advances in the 2000s;

Augmented Reality diffusion in the 2010s.

HARDWARE

Some technologies which make AR interesting:

Handheld:

Mobile phones;

Tablets;

Wearable devices:

Google glass;

Microsoft Holo Lens;

Orcam (video - http://www.orcam.com/);

Epson Moverio.

FIDUCIARY MARKERS AUGMENTED REALITY: ARTOOLKIT

ARToolkit is an Open Source toolkit for marker-based augmented reality;

It is quite old (last update in 2007) but still a goodstarting point for understanding the AR concepts (open& well documented);

It offers functions for detecting and tracking single ormultiple markers while relaying on OpenGL/glut for2D/3D rendering;

http://www.hitl.washington.edu/artoolkit.

ARTOOLKIT DEMO

ARTOOLKIT

For additional informations about the nexttopics, the reader is referred to the very wellwritten ARToolkit documentation and tutorials: http://www.hitl.washington.edu/artoolkit/documentation/

Some other useful information can be found inthe examples provided with the toolkit;

BASIC PRINCIPLES

FIDUCIARY MARKERS

The marker plays the role of an object whichgeometry is known;

In particular:

we chose markers which are easilydetectable (thick black borders);

we know the real world size of the marker;

we chose the inner symbol which is neitherhorizontally nor vertically symmetric in orderto estimate its rotation.

MARKER DETECTION

original image thresholded image connected components

contours edges and corners fitted square

MARKER DETECTION

A simple detection algorithm is used to find just candidates: anypattern with thick black borders;

The actual marker is found normalizing the candidates and comparing them with the searched pattern using template matching;

The candidate giving the highest confidence is selected.

MARKER MATCHING

...

...

...

searched pattern

normalized candidates

found candidates

ESTIMATION OF THE 3D POSITION AND ORIENTATION

Now that we have an objectwhich geometry, size, positionand orientation are known, wecan estimate its 3D position withrespect to the camera;

It can be done computing theextrinsic parameters as seen forcamera calibration;

Intrinsic parameters which aregood for most cameras arepart of the toolkit. Specificparameters can be obtainedcalibrating the camera.

ARTOOLKIT COORDINATE SYSTEMS

EXVIEW DEMO

ARTOOLKIT – CAMERA CALIBRATION

Intrinsic parameters which are enough general to work with most of the cameras are available in the toolkit;

However, in order to improve the detection and tracking performances, a utility for camera calibration is provided in order to calibrate your own camera.

MARKERLESS AUGMENTED REALITY?

Tracking a number of feature points (e.g., SIFT) inorder to detect a marker object (e.g., a photo)and to estimate its position and orientation.

AUGMENTED REALITY TOOLKITS

DEMO TIME

QUESTION TIME

CONTACTS

For any doubts feel free to contact me:

furnari@dmi.unict.it

Room 30;

Slides availabe at:

My personal page:

http://www.dmi.unict.it/~furnari

Studium course page