ARDance

andEngineeringin

Augmented Reality

Presented by: Michael Korostelev

the Computer Fusion Laboratory

Director: Dr. Li Bai

Exchange scholarDr. Guohua Huang

Students:James RenMichael KorostelevMichael HuangFengcheng WuDavid Osinski

CFL Lab

Project Showcase

``

`````

``````````

Septa Electronic Payment Project2004 – now ($150K + $1.0M)

Dance and Engineering in Augmented Reality 2010 – now

($50K)

Wireless Distributed Operating System 2007 – now ($400K)

Wind-Solar Hybrid LED Lighting System 2010 - now

Sensing Technology for Import Food Inspection 2009 - now

Wireless Industrial Process Monitoring 2008 – now ($200K)

Project Showcase

Art and Engineering

Purely creative

Art and Engineering

Entertainment

Art and Engineering

Practical and profitable

Art and Engineering

Educational

CFL Lab

Artist Visions Film FestivalVisual Treasure Hunt• Visitors given clues about the

town of Lambertville NJ• They explore the town to find

QR codes• Scan QR codes and attach

their own message • Visitors’ input becomes

collaborative story

Art and Engineering

These projects bring engineers close to artists and act as a bridge between the disciplines

Each having a goal to relieve the apprehension laymen have to technology

Develop tools for creative people, rather than develop creative works

Motivation

Create a medium for rich creative works

Inspire ideas from artists and engineers alike

Enhance education and increase outreach

Establish framework for fundamental research

Project Goal

Enhance visual experience of a live performance with AR

• Visually enhance

storytelling

• Engage and involve

audience

• Convey complex ideas

• Reduce production costs

• Unbind creative

limitations

What Is AR ?

Display Techniques

Head Mounted Displays

Spatial Displays

Handheld Display

Dance and Engineering in Augmented Reality

Camera/Projector Installation

Augmented Stage• High frame rate digital capture• High lumen projector with wide

focal range• PC with accelerated graphics

• Marker less stationary objects• IR led marked moving objects• Volumetric objects• Dynamic lighting• Floor projection

2D/3D Projector-Camera Calibration

2D/3D Projector-Camera Calibration

Homography (perspective transformation) Stereo Calibration

Kinect “Hacks”

RGBD Sensor Based Calibration

• Calibrate RGBD sensor

Uncalibrated overlayDepth and Image camera separated physically so images do not match

Two different approaches to create image overlay

- Homography- Stereo Calibration

Projector – Sensor Extrinsics

Real Scene

Sensor Capture

Projection

Determining Projection Matrix

• Determining the projection matrix

Determining Projection Matrix

Displaying the scene

Displaying the Scene