L 3 AR D l T lLecture 3: AR Developer Tools
Mark Billinghurst
July 2011
COSC 426: Augmented RealityCOSC 426: Augmented Reality
Building Compelling AR ExperiencesBuilding Compelling AR Experiences
experiences
applications
toolstools
components Display, Tracking
Sony CSL © 2004
Low Level AR Libraries
ARToolKit Marker based trackingg
FLARToolKitFl h f ART lKFlash version of ARToolKit
SSTTSimple Spatial Template Tracking
O iraOpiraRobust Natural Feature Tracking
Wh ART lK ?What is ARToolKit?
Marker Tracking Library for AR applicationsOpen Source Multi platform (Linux Windows MacOS)Open Source, Multi-platform (Linux, Windows, MacOS)
Overlays 3D virtual objects on real markersUses single tracking marker Determines camera pose information (6 DOF)
ARToolKit Websitehttp://www.hitl.washington.edu/artoolkit/http://artoolkit.sourceforge.net/
ART lK S fARToolKit SoftwareARToolKit version: 2 65 or laterARToolKit version: 2.65 or laterCurrently two license models
Open Source (GPL): ARToolKit 2 72Open Source (GPL): ARToolKit 2.72Commercial (ARToolWorks): ARToolKit 4.0
OS Li Wi d M OS X iPh /A d idOS: Linux, Windows, MacOS X, iPhone/AndroidProgramming language: CR l d fRelated software
ARToolKit Professional: Commercial versionARToolKitPlus: Advanced versionARToolKitPlus: Advanced versionNyARToolkit: Java and C# versionFLARToolKit: Flash version
ART lKit F ilARToolKit Family
ARToolKitARToolKit NFT
ARToolKit Plus
JARToolKit (Java)
ARToolKit (Symbian)
FLARToolKit (Flash)NyToolKit- Java, C#,- Android WM
FLARToolKit (Flash)
( )Android, WM FLARManager (Flash)
ART lK ARToolKit contents
LibrarieslibAR – trackingb t ac glibARvideo – video capturinglibARgsub – image/graphics drawinglibARgsub image/graphics drawinglibARmulti – multi-marker tracking
UtilitiesUtilitiesCamera calibrationMarker training
ARToolKit Structure
ARvideo.lib
DirectShow
Three key libraries:AR32 lib ARToolKit image processing functionsAR32.lib – ARToolKit image processing functionsARgsub32.lib – ARToolKit graphics functionsARvideo lib – DirectShow video capture classARvideo.lib – DirectShow video capture class
Additional Software
To build an AR application you may need additional softwareadditional softwareHigh level rendering library
Open VRML, Open Inventor, osgART, etc
Audio LibraryAudio LibraryFmod, etc
Peripheral support
What does ARToolKit Calculate?Position of makers in the camera coordinatesPose of markers in the camera coordinatesPose of markers in the camera coordinatesOutput format
3x4 matrix format to represent the transformation matrix from the marker coordinates to the camera coordinates
C d SCoordinate Systems
Tracking Range with Pattern Size
Rule of thumb – range = 10 x pattern widthRule of thumb range 10 x pattern width
Tracking Error with Range
Tracking Error with Angle
AR Application Development
An ARToolKit ApplicationI iti li tiInitialization
Load camera and pattern parametersM i LMain Loop
Step1. Image capture and displayStep2. Marker detectionStep3. Marker identificationStep4. Getting pose informationStep5. Object Interactions/SimulationStep6. Display virtual objects
End ApplicationCamera shut down
Image capture: libARvideo
Return the pointer for captured imageARUint8 *arVideoGetImage( void );
Pixel format and byte size are defined in config hPixel format and byte size are defined in config.h#define AR_PIX_FORMAT_BGR
#define AR_PIX_SIZE 3
Graphics handling: libARgsub
Set up and clean up the graphics windowvoid argInit( ARParam *cparam double zoomvoid argInit( ARParam *cparam, double zoom,
int fullFlag, int xwin, int ywin,int hmd_flag );
void argCleanup( void );
cparam: camera parameter
zoom: zoom ratiofullFlag: 0: normal, 1: full screen modeXwin, ywin: create small window for debughmd_flag: 0: normal, 1: optical see-through mode
Graphics handling: libARgsubGo into the iterative cyclevoid argMainLoop(
void (*mouseFunc)(int btn int state int x int y)void (*mouseFunc)(int btn,int state,int x,int y),void (*keyFunc)(unsigned char key, int x, int y),void (*mainFunc)(void)
));
Swap bufferspvoid argSwapBuffers( void );
Graphics handling: libARgsubSet the window for 2D drawing
id D M d 2D( id )void argDrawMode2D( void );
Set the window for 3D drawinggvoid argDrawMode3D( void );
void argDraw3dCamera( int xwin, int ywin );
Display imageid Di I ( ARUi t8 *ivoid argDispImage( ARUint8 *image,
int xwin, int ywin );
Sample ARToolKit Applications
Ex. 1: Simple video displayEx. 2: Detecting a markerEx. 3: Using patternE 4 G tti 3D i f tiEx. 4: Getting a 3D informationEx 5: Virtual object overlayEx. 5: Virtual object overlay
Ex 1: Simple Video Display
Program : sample1.cKey points
Loop structureLoop structureVideo image handlingCamera parameter handlingWindow setupWindow setupMouse and keyboard handling
Sample1.c Main Functionmain(){{
init();argMainLoop( mouseEvent, keyEvent,
mainLoop);a oop);}
Sample1.c - mainLoop Functionif( dataPtr = (ARUint8 *)arVideoGetImage()) == NULL ) {arVideoGetImage()) NULL ) {
arUtilSleep(2); return;return;
}D M d 2D()argDrawMode2D();
argDispImage(dataPtr, 0, 0 ); arVideoCapNext(); argSwapBuffers();
Sample1.c – video initializationC fi th id i tConfigure the video inputvconf = <video configuration string>
Start video capturearVideoCapStart();
In init(), open the videoarVideoOpen( vconf );arVideoInqSize(&xsize, &ysize);
When finished, close the video pathWhen finished, close the video patharVideoCapStop();arVideoClose();
Changing Image Size
For input capturevconf = “videoWidth=320,videoHeight=240";
Note – the camera must support this image size
For displayargInit( &cparam 1 5 0 0 0 0 );argInit( &cparam, 1.5, 0, 0, 0, 0 );
Th d f d l The second parameter means zoom ratio for display image size related to input image.
Ex. 2: Detecting a MarkerP l 2Program : sample2.cKey pointsy p
Threshold valueImportant external variablesImportant external variablesarDebug – keep thresholded imagearImage – pointer for thresholded imagearImage – pointer for thresholded imagearImageProcMode – use 50% image for image processingprocessing
- AR_IMAGE_PROC_IN_FULL - AR IMAGE PROC IN HALFAR_IMAGE_PROC_IN_HALF
Sample2.c – marker detection
/* detect the markers in the video frame */ if( D t tM k (d t Pt th hif(arDetectMarker(dataPtr, thresh, &marker_info, &marker_num) < 0 ) {l ()cleanup();
exit(0);}for( i = 0; i < marker_num; i++ ) {argDrawSquare(marker_info[i].vertex,0,0);
}
Sample2.c – marker_info structuretypedef struct {
int area; int id; int dir; double cf; double pos[2]; double line[4][3]; double vertex[4][2];
} ARMarkerInfo;
Ex. 3: Using a Pattern
Program : sample3.cKey pointsKey points
Pattern files loadingStructure of marker information
- Region features- Pattern Id, direction- Certainty factor
Marker identificationMarker identification
Making a pattern templateU f ili Use of utility program:
mk_patt.exeShow the patternShow the patternPut the corner of red line segments on the left-top segments on the left top vertex of the markerPattern stored as a template in a file1:2:1 ratio determines the
tt i dpattern region used
Sample3.c – Pattern File Loading
int patt_id;char *patt name = “Data/kanjiPatt”char patt_name Data/kanjiPatt
/* load pattern file *// load pattern file /if(patt_id=arLoadPatt (patt_name) < 0){{
printf ("Pattern file load error !! \n"); exit(0);exit(0);
}
Checking for known patterns/* check for known patterns */k = -1;for( i = 0; i < marker num; i++ ) {for( i = 0; i < marker_num; i++ ) {
if( marker_info[i].id == patt_id) {/* you've found a pattern */printf("Found pattern: %d \n",patt_id);if( k == -1 ) k = i;elseelse /* make sure you have the best pattern (highest confidence factor) */if( k i f [k] f k i f [i] f )if( marker_info[k].cf < marker_info[i].cf )
k = i;}
}
Ex. 4 – Getting 3D information
Program : sample4.cKey points
Definition of a real markerTransformation matrixTransformation matrix- Rotation component
T l ti t- Translation component
Sample4.c – Transformation matrix
double marker_center[2] = {0.0, 0.0};double marker_width = 80.0;double marker trans[3][4];_
arGetTransMat(&marker info[i]arGetTransMat(&marker_info[i], marker_center, marker_width, marker trans);marker_trans);
Finding the Camera PositionThis function sets transformation matrix from marker
to camera into marker_trans[3][4]._ [ ][ ]arGetTransMat(&marker_info[k], marker_center,
marker_width, marker_trans);
You can see the position information in the values of marker trans[3][4]marker_trans[3][4].
Xpos = marker_trans[0][3];Ypos = marker_trans[1][3];Zpos = marker_trans[2][3];
ARToolKit Coordinate Frame
Ex. 5- Virtual Object Display
Program : sample5.cKey points
OpenGL parameter settingOpenGL parameter settingSetup of projection matrixSetup of modelview matrix
Appending your own OpenGL code
Set the camera parameters to OpenGL Projection matrix.argDrawMode3D();gargDraw3dCamera( 0, 0 );
Set the transformation matrix from the marker to the camera to Set the transformation matrix from the marker to the camera to the OpenGL ModelView matrix.argConvGlpara(marker_trans, gl_para);glMatrixMode(GL_MODELVIEW);glLoadMatrixd( gl_para );
After calling these functions, your OpenGL objects are drawn in the real marker coordinates.
3D CG Model RenderingARToolKit does not have a function to handle 3D CG models3D CG models.3rd party CG rendering software should be employed.
OpenVRMLpOpenSceneGraphetcetc
Loading Multiple Patterns
Sample File: LoadMulti.cUses object c to load Uses object.c to load
Object Structuret d f t t {typedef struct { char name[256]; int id; int visible;double marker_coord[4][2];double trans[3][4];double trans[3][4];double marker_width;double marker_center[2];
} ObjectData_T;
Finding Multiple Transforms
Create object listObjectData T *object;ObjectData_T object;
Read in objects - in init( )read_ObjData( char *name, int *objectnum );
Find Transform in mainLoop( )Find Transform – in mainLoop( )for( i = 0; i < objectnum; i++ ) {
Ch k tt..Check patterns..Find transforms for each marker}}
Drawing Multiple Objects
Send the object list to the draw functiondraw( object, objectnum );
Draw each object individuallyj yfor( i = 0; i < objectnum; i++ ) {
if( object[i].visible == 0 ) continue;( j [ ] ) ;argConvGlpara(object[i].trans, gl_para);draw object( object[i].id, gl para);draw_object( object[i].id, gl_para);
}
Limitations of ARToolKitPartial occlusions cause tracking failureAffected by lighting and shadowsAffected by lighting and shadowsTracking range depends on marker sizeP f d d b f kPerformance depends on number of markers
cf artTag, ARToolKitPlusPose accuracy depends on distance to markerPose accuracy depends on angle to markery p g
ARToolKit in the World
H d d f jHundreds of projectsLarge research communityg y
FLARToolKitFlash AS3 Version of the ARToolKit (was ported from NyARToolkit the Java Version of the ARToolkit)
enables augmented reality on the Browseruses Papervision3D for as 3D Engineavailable at http://saqoosha.net/dual license, GPL and commercial license
AR Application Components
Adobe Flash
Papervision 3D
FLARToolkit
Source PackagesO i i l“ FLART lki„Original“ FLARToolkit (Libspark, Saqoosha)
(http://www.libspark.org/svn/as3/FLARToolKit/trunk/ )
Start-up-guidesSaqoosha (http://saqoosha.net/en/flartoolkit/start-up-guide/ )
Miko Haapoja (http://www.mikkoh.com/blog/?p=182 )
Frameworks“„FrameworksSquidder MultipleMarker – Example (http://www.squidder.com/2009/03/06/flar-how-to-multiple-instances-of-multiple-
k /markers/ )
FLARManager (http://words.transmote.com/wp/flarmanager/ )
Papervision 3Dhttp://www.papervision3d.org/Flash-based 3D-EngineFlash based 3D EngineSupports
import of 3D Modelstexturingganimationscene graphscene graph
alternatives: Away3d, Sandy,…
Papervision ExamplesRed Bull|Flugtage Flight Labhttp://www.redbull.com/flightlab/#/Game/TEMP_0
Barcinski JeanJean | 3D PortfolioBarcinski JeanJean | 3D Portfoliohttp://www.barcinski-jeanjean.com/
more cool papervision websiteshttp://blog.papervision3d.org/2009/04/24/2009-webby-awards-nominees/
AR Tools
B ildi C lli AR E iBuilding Compelling AR Experiences
experiences
applications
tools Authoringtools Authoring
components Tracking, Display
Sony CSL © 2004
AR AuthoringS ft Lib iSoftware Libraries
osgART, Studierstube, MXRToolKitgPlug-ins to existing software
DART (M di Di ) ARDART (Macromedia Director), mARxStand Alone
AMIRE, ComposAR, etcNext Generation
iaTAR (Tangible AR)iaTAR (Tangible AR)
mARx Plug-in
3D Studio Max Plug-inCan model and view AR content at the same time
BuildAR
http://www.buildar.co.nz/Stand alone applicationppVisual interface for AR model viewing application Enables non-programmers to build AR scenesEnables non-programmers to build AR scenes
ImageTclARAdds AR components to ImageTcl
http://metlab.cse.msu.edu/imagetclar/Modular Library (Scripting, Tcl)Supports several tracking systems (vision, magnetic, inertial)Easy to learn but little support, small community
DARTDesigners AR Toolkit
http://www cc gatech edu/dart/http://www.cc.gatech.edu/dart/AR plug-in for Macromedia DirectorD l d f d iDeveloped for designersVisual programmingScripting interface
StudierstubeComplete authoring toolComplete authoring tool
http://studierstube.icg.tu-graz.ac.at/Framework (Low Level Programming, C++)Modularity, Extensibility, Scalability, HeterogeneityModularity, Extensibility, Scalability, HeterogeneitySupport for wide range of trackers, displays, input
Metaio UnifEye SDKComplete commercial authoring platform
http://www.metaio.com/products/p pOffers viewer and editor toolsVisual interface and low level SDKVisual interface and low level SDKDelivery on desktop or mobile platforms
OSGART Programming LibraryIntegration of ARToolKit with a High-Level Rendering Engine (OpenSceneGraph)g g ( p p )OSGART= OpenSceneGraph + ARToolKit
Supporting Geometric + Photometric Registrationpp g g
osgART:Features
C++ (but also Python Lua etc)C++ (but also Python, Lua, etc).Multiple Video Input supports:
Direct (Firewire/USB Camera), Files, Network by ARvideo, PtGrey, CVCam, VideoWrapper, etc.
Benefits of Open Scene Graph Rendering Engine Plug-ins etcRendering Engine, Plug ins, etc
More Libraries
JARToolKitMRToolKit MXRToolKit ARLib OpenVIDIAMRToolKit, MXRToolKit, ARLib, OpenVIDIADWARF, Goblin XNAAMIRED’F iD’Fusion
Advanced Authoring: iaTAR (Lee 2004)
I i AR A th iImmersive AR AuthoringUsing real objects to create AR applications
osgART
Developing Augmented Reality Developing Augmented Reality Applications with osgART
What is a Scene Graph?Tree-like structure for organising a virtual worldTree-like structure for organising a virtual world
e.g. VRML
Hi h f d th t d fiHierarchy of nodes that define:Groups (and Switches, Sequences etc…)TransformationsProjectionsGeometry…
And states and attributes that define:Materials and texturesLighting and blending…
Scene Graph Example
Benefits of a Scene GraphP fPerformance
Structuring data facilitates optimizationoptimization
- Culling, state management, etc…
AbstractionAbstractionUnderlying graphics pipeline is hiddenLow-level programming (“how do I display this?”) replaced with high-l l t (“ h t d I t t
Image: sgi
level concepts (“what do I want to display?”)
About Open Scene Graphhttp://www.openscenegraph.org/Open-source scene graph implementationBased on OpenGLObject-oriented C++ following design pattern principlesUsed for simulation, games, research, and industrial projectsActive development community
Maintained by Robert Osfield~2000 mailing list subscribersDocumentation project: www osgbooks comDocumentation project: www.osgbooks.com
Uses the OSG Public License (similar to LGPL)
About Open Scene Graph (2)
Pirates of the XXI Century Flightgear SCANeR
3DVRII Research Institute EOR VRlab Umeå University3DVRII Research Institute EOR VRlab Umeå University
Open Scene Graph FeaturesPlugins for loading and saving
3D: 3D Studio (.3ds), OpenFlight (.flt), Wavefront (.obj)…2D: .png, .jpg, .bmp, QuickTime movies
NodeKits to extend functionalityye.g. osgShadow
Cross-platform support for:Cross platform support for:Window management (osgViewer)Threading (OpenThreads)Threading (OpenThreads)
Open Scene Graph Architecture
Inter-operability with other environments,
e.g. Python
Plugins read and NodeKits extend Plugins read and write 2D image and 3D model files
core functionality, exposing higher-level node types
Scene graph and rendering
functionalityfunctionality
Some Open Scene Graph Demos
osgviewer osgmotionblur osgparticle
You may want to get the OSG data package:Via SVN: http://www.openscenegraph.org/svn/osg/OpenSceneGraph-Data/trunk
osgreflect osgdistortion osgfxbrowser
Learning OSGCheck out the Quick Start Guide
Free PDF download at http://osgbooks.com/, Physical copy $13US
J i h ili li Join the mailing list: http://www.openscenegraph.org/projects/osg/wiki/MailingLists
Browse the website: http://www.openscenegraph.org/projects/osg
Use the forum: http://forum.openscenegraph.orgp p g p g
Study the examplesRead the source?
ARTosgART
What is osgART?
osgART adds AR to Open Scene GraphFurther developed and enhanced by:Further developed and enhanced by:
Julian LooserH S hHartmut SeichterRaphael Grasset
Current version 2.0, Open Sourcehttp://www osgart orghttp://www.osgart.org
osgART Approach: Basic Scene Graph
Root
T f0.988 -0.031 -0.145 0 -0.048 0.857 -0.512 0 [ ]Transform 0.141 0.513 0.846 0 10.939 29.859 -226.733 1 [ ]
3D ObjectTo add Video see-through AR:
Integrate live videogApply correct projection matrixUpdate tracked transformations in realtime
osgART Approach: AR Scene Graph
Root
Transform
3D Object
osgART Approach: AR Scene GraphRoot
Transform
VirtualCamera
VideoLayer
Video3D Obj
Layer
Geode 3D Object
osgART Approach: AR Scene GraphRoot
Projection matrix fromtracker calibration
Orthographic
Transform
VirtualCamera
Transformation matrix updated from marker tracking in realtimeVideo
Layer
Orthographic projection
Video3D Obj
LayerFull-screen quad with live texture updated from Video source
Geode 3D ObjectVideo source
osgART Approach: AR Scene GraphRoot
Projection matrix fromtracker calibration
Orthographic
Transform
VirtualCamera
Transformation matrix updated from marker tracking in realtimeVideo
Layer
Orthographic projection
Video3D Obj
LayerFull-screen quad with live texture updated from Video source
Geode 3D ObjectVideo source
osgART ArchitectureLike any video see-through AR library, osgART requires video input and tracking capabilities.
AVideo Sourcee g DirectShow A
R Lib
Applica
e.g. DirectShow
rary
ationTracking Module(libAR.lib)
osgART ArchitectureosgART uses a plugin architecture so that video sources and tracking technologies can be plugged in as necessary
OpenCVVideo
V
OpenCVVideo -VidCapture -CMU1394 -
PointGrey SDK -VidereDesign -
V d W
os
App
Video Plugin
VideoWrapper -VideoInput -
VideoSource -DSVL -
Intranel RTSP - sgART
plication
TrackeARToolKit4 -
ARToolkitPlus - ner Plugin
MXRToolKit -ARLib -
bazAR (work in progress) -ARTag (work in progress) -
Basic osgART TutorialD l ki ART li ti f t hDevelop a working osgART application from scratch.Use ARToolKit 2.72 library for tracking and video capturecapture
osgART Tutorial 1: Basic OSG ViewerStart with the standard Open Scene Graph ViewerWe will modify this to do AR!y
osgART Tutorial 1: Basic OSG Viewer
The basic osgViewer…#include <osgViewer/Viewer>#include <osgViewer/ViewerEventHandlers>#include <osgViewer/ViewerEventHandlers>
int main(int argc, char* argv[]) {
// Create a viewerosgViewer::Viewer viewer;
// Create a root nodeosg::ref_ptr<osg::Group> root = new osg::Group;
// Attach root node to the viewerviewer.setSceneData(root.get());g
// Add relevant event handlers to the viewerviewer.addEventHandler(new osgViewer::StatsHandler);viewer.addEventHandler(new osgViewer::WindowSizeHandler);viewer.addEventHandler(new osgViewer::ThreadingHandler);viewer addEventHandler(new osgViewer::HelpHandler);viewer.addEventHandler(new osgViewer::HelpHandler);
// Run the viewer and exit the program when the viewer is closedreturn viewer.run();
}
osgART Tutorial 2: Adding VideoAdd a video plugin
Load, configure, start video capture…
Add a video backgroundCreate, link to video, add to scene-graphg p
osgART Tutorial 2: Adding VideoNew code to load and configure a Video Plugin:
// Preload the video and tracker// Preload the video and trackerint _video_id = osgART::PluginManager::getInstance()->load("osgart_video_artoolkit2");
// Load a video plugin.osg::ref_ptr<osgART::Video> video =
dynamic_cast<osgART::Video*>(osgART::PluginManager::getInstance()->get(_video_id));
// Check if an instance of the video stream could be createdif (!video.valid()) {
// Without video an AR application can not work. Quit if none found.osg::notify(osg::FATAL) << "Could not initialize video plugin!" << std::endl;exit(-1);
}
// Open the video. This will not yet start the video stream but will// get information about the format of the video which is essential// for the connected tracker.video->open();video->open();
osgART Tutorial 2: Adding VideoNew code to add a live video background
osg::Group* createImageBackground(osg::Image* video) {osgART::VideoLayer* _layer = new osgART::VideoLayer();_layer->setSize(*video);osgART::VideoGeode* _geode = new osgART::VideoGeode(osgART::VideoGeode::USE_TEXTURE_2D, video);addTexturedQuad(*_geode, video->s(), video->t());_layer->addChild(_geode);return _layer;
}
osg::ref_ptr<osg::Group> videoBackground = createImageBackground(video.get());
In the main function…
videoBackground->getOrCreateStateSet()->setRenderBinDetails(0, "RenderBin");
root->addChild(videoBackground.get());video->start();
osgART Tutorial 3: TrackingAdd T k l Add a Tracker plugin
Load, configure, link to videoAdd a Marker to track
Load activateLoad, activateTracked node
C li k i h k i ki llb kCreate, link with marker via tracking callbacksPrint out the tracking data
osgART Tutorial 3: TrackingLoad a tracking plugin and associate it with the video plugin
int _tracker_id = osgART::PluginManager::getInstance()->load("osgart_tracker_artoolkit2");
osg::ref_ptr<osgART::Tracker> tracker = dynamic_cast<osgART::Tracker*>(osgART::PluginManager::getInstance()->get(_tracker_id));
oa a t ac g p ug a assoc ate t w t t e v eo p ug
if (!tracker.valid()) {// Without tracker an AR application can not work. Quit if none found.osg::notify(osg::FATAL) << "Could not initialize tracker plugin!" << std::endl;exit(-1);
}
// get the tracker calibration objectosg::ref_ptr<osgART::Calibration> calibration = tracker->getOrCreateCalibration();
// load a calibration fileif (!calibration->load("data/camera para dat"))if (!calibration->load( data/camera_para.dat )) {
// the calibration file was non-existing or couldnt be loadedosg::notify(osg::FATAL) << "Non existing or incompatible calibration file" << std::endl;exit(-1);
}
// set the image source for the trackertracker->setImage(video.get());
osgART::TrackerCallback::addOrSet(root.get(), tracker.get());
// create the virtual camera and add it to the sceneosg::ref_ptr<osg::Camera> cam = calibration->createCamera();root->addChild(cam.get());
osgART Tutorial 3: TrackingLoad a marker and activate itLoad a marker and activate itAssociate it with a transformation node (via event callbacks)Add the transformation node to the virtual camera node
osg::ref_ptr<osgART::Marker> marker = tracker->addMarker("single;data/patt.hiro;80;0;0");if (!marker.valid()) {
// Without marker an AR application can not work. Quit if none found.osg::notify(osg::FATAL) << "Could not add marker!" << std::endl;osg::notify(osg::FATAL) << "Could not add marker!" << std::endl;exit(-1);
}
marker->setActive(true);
osg::ref_ptr<osg::MatrixTransform> arTransform = new osg::MatrixTransform();osgART::attachDefaultEventCallbacks(arTransform.get(), marker.get());cam->addChild(arTransform.get());
Add a debug callback to print out information about the tracked marker
osgART::addEventCallback(arTransform.get(), new osgART::MarkerDebugCallback(marker.get()));
Add a debug callback to print out information about the tracked marker
osgART Tutorial 3: Tracking
Tracking information is goutput to console
osgART Tutorial 4: Adding ContentN t the trackin data t se!Now put the tracking data to use!Add content to the tracked transformBasic cube code
arTransform->addChild(osgART::testCube());arTransform->getOrCreateStateSet()->setRenderBinDetails(100, "RenderBin");
osgART Tutorial 5: Adding 3D ModelO S G h l d 3D f di lOpen Scene Graph can load some 3D formats directly:
e.g. Wavefront (.obj), OpenFlight (.flt), 3D Studio (.3ds), COLLADAOthers need to be convertedOthers need to be converted
Support for some formats is much better than otherse.g. OpenFlight good, 3ds hit and miss.
Recommend native .osg and .ive formats.osg – ASCII representation of scene graphive Binary OSG file Can contain hold textures.ive – Binary OSG file. Can contain hold textures.
osgExp : Exporter for 3DS Max is a good choicehttp://sourceforge.net/projects/osgmaxexp
Otherwise .3ds files from TurboSquid can work
osgART Tutorial 5: Adding 3D Model Replace the simple cube with a 3D model
std::string filename = "media/hollow_cube.osg";arTransform->addChild(osgDB::readNodeFile(filename));
Replace the simple cube with a 3D modelModels are loaded using the osgDB::readNodeFile() function
arTransform >addChild(osgDB::readNodeFile(filename));
3D St di M
Export to .osg
Note: Scale is important Units are in mm
3D Studio Max
osgARTNote: Scale is important. Units are in mm.
osgART Tutorial 6: Multiple Markers
Repeat the process so far to track more than one marker simultaneouslyy
osgART Tutorial 6: Multiple MarkersR h f k h k
Load and activate two markers
Repeat the process so far to track more than one marker
Load and activate two markersosg::ref_ptr<osgART::Marker> markerA = tracker->addMarker("single;data/patt.hiro;80;0;0");markerA->setActive(true);
osg::ref_ptr<osgART::Marker> markerB = tracker->addMarker("single;data/patt.kanji;80;0;0");_markerB->setActive(true);
Create two transformations, attach callbacks, and add models
osg::ref_ptr<osg::MatrixTransform> arTransformA = new osg::MatrixTransform();osgART::attachDefaultEventCallbacks(arTransformA.get(), markerA.get());arTransformA->addChild(osgDB::readNodeFile("media/hitl_logo.osg"));arTransformA->getOrCreateStateSet()->setRenderBinDetails(100, "RenderBin");cam->addChild(arTransformA.get());
osg::ref_ptr<osg::MatrixTransform> arTransformB = new osg::MatrixTransform();osgART::attachDefaultEventCallbacks(arTransformB.get(), markerB.get());arTransformB->addChild(osgDB::readNodeFile("media/gist_logo.osg"));arTransformB->getOrCreateStateSet()->setRenderBinDetails(100, "RenderBin");cam->addChild(arTransformB.get());
osgART Tutorial 6: Multiple Markers
Basic osgART Tutorial: Summary
Standard OSG Viewer Addition of Video Addition of Tracking
Addition of basic 3D graphics
Addition of 3D Model Multiple Markersg p
FLARMFLARManagerhttp://transmote com/flarhttp://transmote.com/flar
FLARManager:
Makes building FLARToolkit apps easierIs open-source, with a free and commercial licenseIs open source, with a free and commercial licenseIs designed to allow exploration of both augmented reality and alternative controllersreality and alternative controllersWas initiated by Eric Socolofsky, developed with
ib i f FLART lki icontributions from FLARToolkit communityDecouples FLARToolkit from Papervision3DConfiguration without recompilation, via xml config
FLARManager: features
Gives more control over application environmentProvides multiple input optionsRobust multiple marker managementRobust multiple marker managementSupports multiple 3D frameworksOff f f i i iOffers features for optimizationAllows for customization
Resources
Websites
Software Downloadhttp://artoolkit sourceforge net/http://artoolkit.sourceforge.net/
ARToolKit Documentationhttp://www.hitl.washington.edu/artoolkit/
ARToolKit ForumARToolKit Forumhttp://www.hitlabnz.org/wiki/Forum
ARToolworks Inchttp://www.artoolworks.com/p
ARToolKit PlusARToolKit Plushttp://studierstube.icg.tu-
t/h dh ld / t lkit l hgraz.ac.at/handheld_ar/artoolkitplus.php
osgARThttp://www.osgart.org/
FLARToolKitFLARToolKithttp://www.libspark.org/wiki/saqoosha/FLARToolKit/
FLARManagerhttp://words transmote com/wp/flarmanager/http://words.transmote.com/wp/flarmanager/
BooksI E h O S Interactive Environments with Open-Source Software: 3D Walkthroughs and Augmented R li f A hi i h Bl d 2 43 DART Reality for Architects with Blender 2.43, DART 3.0 and ARToolKit 2.72 by Wolfgang Höhl
A Hitchhikers Guide to Virtual Reality by Karen y yMcMenemy and Stuart Ferguson
More Information
M k Billi h t• Mark Billinghurst– [email protected]
• Websiteshi l b– www.hitlabnz.org
Top Related