GRAPHICS, AUDIO-VISUALS AND INTERACTION (GAI) BASED AUGMENTED REALITY (AR) SYSTEM FOR SYMBIAN AND

ANDROID SMARTPHONE

Md. Abdullah - Al - Jubair

Master of Science 2012

Nusat Khidmat Maklumat Akademik MALAYSIA SARAWAK

Graphics, Audio-visuals and Interaction (GAI) based Augmented Reality (AR) System for Symbian and Android Smartphone

MD. ABDULLAH - AL - JUBAIR

I a

A thesis submitted in fulfillment of the requirement for the degree of

Master of Science

(Cognitive Science)

Department of Cognitive Science Faculty of Cognitive Sciences and Human Development

Universiti Malaysia Sarawak 2012

AUTHOR DECLARATION

I hereby declare that this dissertation entitled "Graphics, Audio-visuals and Interaction (GAI)

based Augmented Reality (AR) System for Symbian and Android Smartphone" is the result of

my own research excepted as cited in the references. The thesis is not concurrently submitted in

candidature of any other degree.

ýr 2dýv ö'1( a, ý

Md. Abdullah - Al - Jubair

Department of Cognitive Sciences

Faculty of Cognitive Sciences and Human Development

Universiti Malaysia Sarawak

11

ACKNOWLEDGEMENTS

All praise goes to ALLAH, the Almighty, on whom ultimately we depend on for sustenance and

guidance. Thankful to Allah that gave me strength to successfully completed the thesis and

research work. I would like to thank my parents, my maternal uncle, my younger sister and the

rest of the family for their continuous encouragement to finish my thesis in a high note.

I would like to record my gratitude to Dr. Edmund Ng Giap Weng for his supervision, advice

and guidance from the beginning to the end of this research work. He supported me with his vast

experience and knowledge. He provided me research ideas when it was necessary.

I also would like to record my gratitude to two respected examiners for review my thesis and

their valuable comments. Their valuable comments assisted me to make my thesis more enrich.

I would also like to express my special thanks to my friends, my colleagues for their love,

prayers and encouragement. I would like to thank all the members of the research group

COESTAR lab at Faculty of Cognitive Science and Human Development for their endless

support and cooperation.

iii

ABSTRACT

(Augmented Reality (AR) is a live, direct or indirect, view of a physical, real-world environment

whose elements are augmented by computer-generated sensory input such as sound, video and

graphics data. This research studied an open source `ARToolKitPlus' tracking library and

developed Graphics, Audio-visual and Interactions (GAI) based Augmented Reality system for

Symbian and Android Smartphone, which are integrated with mobile graphics module, audio

video module and 3D object interaction module The GAI based mobile AR system allows the

users to build the AR game applications. The development of the GAI based mobile AR system

involved video capturing, image processing, mobile graphics module, audio-visual module and

interactions module. Video capturing component for Symbian AR was implemented by using

Symbian and video capturing component for Android AR was implemented by using Android

camera API. An image library was built to perform image processing tasks. OpenGLES and

PowerVR graphics API were used to build the graphics module for this system. Audio video API

of Symbian and Android were used to build the GAI based audio video module. Color picking

algorithm was used to build the interactions module. In addition, a mobile game application was

developed to evaluate all the modules functionalities of 2D, 3D, text, animations, audio, video

and 3D object interactions. This research has performed the evaluation to prove the effectiveness

and efficiency of the GAI modules in the mobile game applications. The respondents generally

found that the GAI modules performance in the game applications is acceptable. The result of the

respondents' satisfaction in these mobile game applications indicates that the Graphics Audio-

visual and Interaction (GAI) based modules in mobile AR system were successfully

implemented in this research.

iv

ABSTRAK

Augmentasi Realiti (AR) merupakan suatu teknologi yang memberikan maklumat maya

kepada pengguna secara langsung atau tidak langsung di mana persepsi dan interaksi pengguna

dapat ditingkatkan melalui input deria yang dyanakan oleh komputer seperti bunyi, video, dan data

graft. Penyelidikan ini mengkaji integrasi ARToolkitPlus, iaitu sistem AR mudah alih untuk telefon

pintar Symbian dan Android yang berasaskan penjejakan dan sumber terbuka yang disepadukan

dengan modul-modul graft mudah alih, modul video, modul audio dan modul interaksi objek 3D.

Sistem AR mudah alih berasaskan multimedia ini juga membolehkan pengguna untuk membina

aplikasi-aplikasi AR mudah alih yang lain. Proses pembinaan sistem ini melibatkan rakaman video,

pemprosesan imej, modul graft mudah alih, modul multimedia dan modul interaksi. API telah

digunakan dalam penangkapan video menggunakan kamera Symbian dan Android. Suatu koleksi imej

telah dibina untuk melakanakan pemprosesan imej, transformasi 'greyscale' dan pengambangan.

Selain itu, OpenGLES dan Power VR graft API juga digunakan untukpembinaan modul graft dalam

sistem ini. Video audio API Symbian dan Android telah digunakan dalam pembinaan modul video

dan audio berasaskan multimedia. Algoritma pemilihan warnajuga telah digunakan untuk membina

modul interaksi. Di samping itu, satu aplikasi permainan mudah alih telah dibina untuk menilai

fungsi modul 2D, 3D, teks, animasi, audio, video dan interaksi objek 3D. Dalam penyelidikan ini,

penilaian telah dilakukan untuk membuktikan keberkesanan dan kecekapan modul multimedia dalam

aplikasi permainan mudah alih. Secara umumnya, responden yang mengunakan aplikasi permainan

mudah alih ini telah memberikan komen yang baik di mana modul multimedia dalam aplikasi

permainan dapat berfungsi dengan cekap dan berkesan. Keputusan ini menunjukkan bahawa modul

multimedia dalam sistem AR mudah alih telah berjaya diimplimentasi dalam penyelidikan ini

V

rusat miudmat MaklumatAkadera,.. UIYIVERSITI MALAYSIA SARAWAK

TABLE OF CONTENS

AUTHOR DECLARATION ..................................................:............................... ii

ACKNOWLEDGEMENTS .................................................................................. iii

ABSTRACT ................................................................................................... iv

ABSTRAK ...................................................................................................... v

TABLE OF CONTENTS ...................................................................................... vi

LIST OF APPENDICES ..................................................................................... xii

LIST OF FIGURES ......................................................................................... xiii

LIST OF TABLES .......................................................................................... xvii

CHAPTER ONE: INTRODUCTION

1.0 Overview ................................................................................................... 1

1.1 Background ................................................................................................... 4

1.2 Problem Statements ....................................................................................... 7

1.3 Research Objective ....................................................................................... 8

1.4 Scope ........................................................................................................ 9

1.5 Limitations ................................................................................................. 9

1.6 Definitions of Terms .................................................................................... 10

1.7 Summary .................................................................................................. 11

V1

CHAPTER TWO: LITERATURE REVIEW

2.0 Overview .................................................................................................. 13

2.1 Augmented Reality ..................................................................................... 13

2.2 Augmented Reality in Handheld and Embedded Devices .......................................... 15

2.3 ARToolKitPlus as a Registration and marker Tracking Component ............................... 19

2.4 Performance Measurement (Benchmark) of ARTooIKitPlus ......................................... 22

2.5 Image Processing ........................................................................................ 24

2.5.1 Camera Callibration .......................................................................... 24

2.5.2 Greyscale Transformation .................................................................... 25

2.5.3 Thresholding .................................................................................. 26

2.5.4 Labeling ........................................................................................ 28

2.5.5 Edge Detection ................................................................................ 28

2.5.6 Template Matching ........................................................................... 3 0

2.6 3D Mobile Graphics Generation ....................................................................... 31

2.6.1 2D and 3D Virtual Object .................................................................... 31

2.6.2 Text Rendering ................................................................................ 34

2.6.3 Animation Rendering ......................................................................... 35

2.6.4 Graphic API ................................................................................... 36

2.6.5 Performance measurement of graphics in mobile AR ................................... 38

2.7 Audio Video Visuals ..................................................................................... 39

2.8 Interactions ................................................................................................ 41

vii

2.9 Summary .................................................................................................. 44

CHAPTER THREE: METHODOLOGY

3.0 Overview ................................................... 46 ...............................................

3.1 Research Methodology ................................................................................. 46

3.1.1 Development Methodology ................................................................. 46

i) Analysis of acquire Software Requirements Specifications ....................... 49

ii) Collection of the Resources ............................................................ 50

iii) Construction and Modification of the Resources .................................. 52

iv) System Constructions .................................................................. 54

v) Integration the module and Testing ................................................... 58

3.1.2 Testing Methodology ........................................................................ 58

I) User Evaluation ........................................................................... 58

i) Development Questionnaires .................................................. 60

a) Research Instruments .......................................................... 60

b) Data Collection Procedure ............................................. 64

c) Data Analysis ............................................................ 65

II) Technical Evaluation ................................................................... 67

3.2 Summary ..................................................................................................... 67

viii

CHAPTER FOUR: DEVELOPMENT

4.0 Overview .................................................................................................. 69

4.1 Video Capturing .......................................................................................... 69

4.2 Sample Image Grabbing ................................................................................ 72

4.3 Image Processing ........................................................................................ 73

4.4 AR Registration and Detect Pattern Marker .......................................................... 75

4.5 ARToolKitPlus in Symbian^3 ......................................................................... 81

4.6 3D Graphics Generation ................................................................................ 83

4.6.1 2D and 3D Virtual Object ................................................................... 84

4.6.2 Implementation of 3D Studio Max Model ................................................. 91

4.6.3 Text Rendering ................................................................................ 92

4.6.4 Animations .................................................................................... 94

4.7 Audio and Video visuals ................................................................................ 99

4.8 Interactions ............................................................................................... 103

4.8.1 Mobile Keypad button based interaction ................................................. 107

4.8.2 Finger touch based interaction ............................................................. 108

4.9 Sample AR Game Applications ...................................................................... 110

4.10 Summary ............................................................................................... 112

ix

CHAPTER FIVE: FINDINGS AND DISCUSSIONS

5.0 Overview ................................................................................................. 114

5.1 Video Capturing ........................................................................................ 115

5.2 Geyscale Transformation and Thresholding ......................................................... 116

5.3 AR Registration ......................................................................................... 117

5.4 3D Virtual Object Graphics Generation .............................................................

118

5.4.1 2D and 3D virtual Object ..................................................................... 118

5.4.2 3D Studio Max Modeling .................................................................. 120

5.4.3 Text Rendering ............................................................................... 121

5.4.4 Animation .................................................................................... 122

5.4.5 2D Graphical User Interface ............................................................... 123

5.5 Audio and Video visual ............................................................................... 124

5.6 Interaction ............................................................................................... 126

5.6.1 Touch Screen based Interaction ........................................................... 127

5.6.2 Mobile Keypad based Interaction .......................................................... 127

5.7 Performance Measurement of the GAI based Mobile AR System ................................ 128

5.8 Evaluation ............................................................................................... 135

5.8.1 User Background Information ............................................................ 136

5.8.2 Subjects Ability in using Their Finger to Control the AR Game Applications..... 136

5.8.3 The Effectiveness and Efficiency of the GAI Modules in the

AR game Applications based on Users' Feedback ...................................... 142

X

5.8.4 User Satisfaction about GAI Modules Functionalities in the AR Game

Application ......................................................:........................... 151

5.8.5 Additional Comment from the Users for the Overall System Performance.......... 152

5.9 Summary ................................................................................................ 155

CHAPTER SIX: CONCLUSION

6.0 Overview ................................................................................................. 157

6.1 Contribution and Significance of the Study .......................................................... 158

6.2 Recommendations for Future Research and Studies ............................................... 160

6.3 Summary of the Research ............................................................................. 160

REFERENCES ............................................................................................. 162

X1

LIST OF APPENDICES

1 User Consent Form ................................................................................ 176

2 Questionnaire to collect User Background Information ...................................... 177

3 Functionality User Tasks ......................................................................... 178

4 User Feedback Questionnaires ................................................................... 180

5 List of Paper and Award .......................................................................... 182

xii

LIST OF FIGURES

FIGURE 2.1 SignPost Project .................................................................... 16

FIGURE 2.2 2D Square Marker .................................................................. 21

FIGURE 2.3 ARToolKitPlus Camera Calibration ............................................ 25

FIGURE 2.4 Formula of Grayscale Transformation ............................................ 26

FIGURE 2.5 Thresholded Image ................................................................ 27

FIGURE 2.6 Edge Detection ..................................................................... 29

FIGURE 2.7 Example of 3D Virtual Objects in Smartphone in AR ......................... 34

FIGURE 2.8 Absolute performance results. Left: Linear scale; Right:

Logarithmic scale .................................................................. 39

FIGURE 2.9 Interaction with Virtual Object using mobile keypads in

Smartphone ......................................................................... 44

FIGURE 3.1 System Development Life Cycle (SDLC, 2011) .............................. 48

FIGURE 3.2 Architecture of the System ........................................................ 55

FIGURE 3.3 Flowchart of the System .......................................................... 57

FIGURE 4.1 Snippet of implementation of video capture for Symbian device.......... 70

FIGURE 4.2 Snippet of implementation of video capture for Android device........... 71

FIGURE 4.3 Snippet of Image Grabber ........................................................ 72

FIGURE 4.4 Greyscale Transformation ........................................................ 73

FIGURE 4.5 Thresholding ....................................................................... 74

FIGURE 4.6 Original functions in ARToolKitPlus to Create Tracker ..................... 76

X111

FIGURE 4.7

FIGURE 4.8

FIGURE 4.9

FIGURE 4.10

FIGURE 4.11

FIGURE 4.12

FIGURE 4.13

FIGURE 4.14

FIGURE 4.15

FIGURE 4.16

FIGURE 4.17

FIGURE 4.18

FIGURE 4.19

FIGURE 4.20

FIGURE 4.21

FIGURE 4.22

FIGURE 4.23

FIGURE 4.24

FIGURE 4.25

FIGURE 4.26

FIGURE 4.27

Modified function to detect Pattern Marker .................................... 76

Snippet of Marker Detection ..................................................... 78

Snippet of executeSingleMarkerPoseEstimator functions .................... 79

Converting AR matrix to OpenGLES matrix .................................. 80

Snippet of GOOM Handling for Symbian^3 ................................... 82

OpenGLES changes required in Symbian^3 .................................... 83

Lighting .............................................................................. 86

Example of Lighting .............................................................. 87

Example of Texture Mapping .................................................... 89

Snippet of Image loading for Texturing ......................................... 90

Snippet of loading 3D Studio Max model data ................................. 91

Snippet of draw 3D model ........................................................ 92

Snippet of sample Text drawing ................................................. 93

Transformation Animation ........................................................ 95

Formula of Interpolation .......................................................... 96

Interpolation for Animations ..................................................... 97

3D Animations ..................................................................... 98

Implementation of audio class using audio API in Symbian ............... 100

Snippet of calling audio function in Android using media API............ 101

Implementation of video in symbian .......................................... 102

Snippet of calling video function in Android using media

player API ......................................................................... 103

xiv

FIGURE 4.28

FIGURE 4.29

FIGURE 4.30

FIGURE 4.31

FIGURE 4.32

FIGURE 4.33

FIGURE 5.1

FIGURE 5.2

FIGURE 5.3

FIGURE 5.4

FIGURE 5.5

FIGURE 5.6

FIGURE 5.7

FIGURE 5.8

FIGURE 5.9

FIGURE 5.10

FIGURE 5.11

FIGURE 5.12

FIGURE 5.13

FIGURE 5.14

FIGURE 5.15

Updating cursor position ........................................................ 105

Coding for reading the Alpha value of the Object .............................

106

Keypad button based Interaction Implementation ............................ 108

Finger touch based Interaction Implementations .............................

109

Some screenshots of Solar System application ............................... 111

Some screenshots of Sky Fighter game ...................................... 112

Screenshot of video capturing .................................................. 115

Greyscale Transformation ......................................................

116

Thresholding ......................................................................

117

3D Virtual Object in real world ................................................. 118

Flat shading and Smooth shading ................................................. 119

Texturing Mapping ............................................................... 120

Petronas Twin tower 3D model on the marker ............................... 121

Text rendering on the marker in Android Smartphone ...................... 122

Series of 3D model Animations ................................................ 123

GUI in mobile AR game ......................................................... 124

Screenshot of video playback after detect the marker ...................... 125

Sky Fighter game has theme music and shooting sound ................... 126

Touch Interaction on object (Duck) in Symbian"3 Smartphone............ 127

Interact virtual object using mobile's keypad button ......................... 128

Test models rendered for result

a) 3D model with Animation; b) Touch Interaction on 3D model;

c) Video clips playback in AR app ............................................. 131

xv

FIGURE 5.16 Performance comparison between GAI, Qualcomm and

ARTooIKitPlus based Applications ................................................... 135

xvi

LIST OF TABLES

TABLE 2.1 Benchmark performed on ARToolKitPlus .............................................. 23

TABLE 2.2 Benchmarks of Studierstube ES (STh ES) .............................................. 38

TALBE 3.1 Functionality user tasks specification of Solar System application .................. 62

TALBE 3.2 Functionality user tasks specification Sky Fighter game .............................. 63

TALBE 3.3 Data analysis based on the Research Objective .......................................... 66

TALBE 3.4 Summary of the area of measurement in the questionnaire in determining

the efficiency, effectiveness and satisfaction of the subjects towards

GAI modules in the AR game applications .......................................................... 66

TABLE 5.1 Result of GAI based mobile AR system on different Symbian

and Android ............................................................................... 13 2

TABLE 5.2 Result of AR Simple cube and AR Texturing application in

Symbian Phone ............................................................................ 134

TABLE 5.3 Result of Image Target and NST application in

Android Phone ............................................................................. 134

TABLE 5.4 Respondents' successes or failures in performing the functionality tasks

with the Solar System application ....................................................... 138

TABLE 5.5 Respondents' successes or failures in performing the functionality tasks

with the Sky Fighter game .............................................................. 140

TABLE 5.6 Respondents' responses to the questionnaires with regards on the

overall performance ...................................................................... 143

xvii

TABLE 5.7 Respondents' responses to the questionnaires with regards on the

Solar System application ................................................................. 145

TABLE 5.8 Respondents' responses to the questionnaires with regards on the

Sky Fighter game ......................................................................... 148

TABLE 5.9 Respondents' satisfaction with the applications ...................................... 151

TABLE 5.10 Respondents' ranking for both applications ........................................... 153

TABLE 5.11 Respondents' comments of the performance of Solar System application........ 154

TABLE 5.12 Respondents' comments of the performance of Sky Fighter game ................. 155

XVlll

Chapter 1

Introduction

1.0 Overview

Augmented Reality (AR) is considered as one of the highly sophisticated technologies in

research field of virtual reality (Augmented Reality, 2011; Technology Review, 2007).

Augmented Reality research intends at creating new Human Computer Interfaces (Wagner,

2007). AR puts data right into the real world. The data can be any computer generated virtual

objects like 2D, 3D, text, animation with audio-visuals. AR indistinct the difference between the

real world and the user interface. After that AR merges them in a way which allows the

developing of simple and intuitive graphical user interfaces for complex applications and games.

The ideal of mobile Augmented Reality system intends to support the user directly at the place of

work instead of requiring the user to attend to the workstations. As such, Augmented Reality

(AR) is becoming a complement to mobile computing research. Smartphone are the smallest and

most ubiquitous device and they are today a fully integrated part of most people's daily life

(Wagner, 2007). This study therefore, targets Smartphone as a device platform.

Graphics, Audio-visual and Interaction (GAI) based mobile Augmented Reality system is

a multimedia content based mobile AR solution which is built with ARToolKitPlus.

ARTooIKitPlus (ARToolkitPlus, 2011) is a tracking library for mobile Augmented Reality.

Wagner (2007) ported the well-known ARToolKit to ARToolKitPlus and has released

1

ARToolKitPlus as open source for further research. The benchmark series (Wagner &

Schmalstieg, 2007) on ARToolKitPlus proved that ARToolKitPlus can meet the requirement to

do better and can provide almost accurate marker tracking and registration in mobile devices.

Hence, ARToolKitPlus is chosen as a tracking library in this research work. Graphics, Audio-

visuals and Interaction (GAI) based mobile Augmented Reality system is integrated with mobile

graphics module and audio video module. The system allows the user to render the 2D and 3D

virtual objects using audio and video visuals in mobile AR environment. An interaction module

is also build and then integrated into this mobile AR system. The interaction module is build

based on color picking algorithm which allow the users to use this interaction component in their

application and games. This is to enable the user to control the virtual objects and Graphical User

Interface (GUI) by using finger touch and mobile keypad buttons on their Smartphone. Thus, the

one of the objective is to develop these modules in this research work. This mobile AR system in

this research targeted university students as the users. The user can utilize these GAI modules

and mobile AR system to develop their own mobile AR applications and games.

Solar System application and Sky Fighter games are developed using this GAI based

mobile AR system to evaluate these modules. The Solar System application was designed for

Android Smartphone. The application draws 3D animated planets with audio voice speech about

planets and allows user to interact any planet to playback the video. The Sky Fighter game was

designed for Symbian Smartphone. The game generates 3D Studio Max based 3D (Planes)

model with animation, text, 2D GUI and 3D model interactions with audio visuals using the

modules of this mobile AR system. There are some similar types of application are developed

2

before such as Live Solar System (LSS) (Sin & Zaman, 2010) which shows the 3D animated

planets, text and interactions. But these applications based on windows platform computer based.

The research work will mainly focus on developing a scientific approach which could

create a structure for delivering objects and knowledge, derived from domain of Augmented

Reality. Therefore, this research work will be able to improve graphics, sound, video as well as

interaction based mobile Augmented Reality system. The study incorporates graphics module

and interaction modules with ARToolKitPlus library, making graphics, sound, video and

interaction based mobile Augmented Reality system a significantly more superior and preferred

mobile AR application system.

The user requires a Smartphone to use this mobile AR system. A Smartphone is a

particular type of camera mobile phone which enables the developer to develop the applications

and games for its operating system (OS) like Symbian OS, Android OS, IOS (Symbian OS,

Android OS, IOS, 2011). This study targeted Symbian and Android OS Smartphone platforms.

Symbian and Android are very well known Smartphone OS, used worldwide and supports well

known programming languages (C++ and java) which make it easier to integrate these modules

into tracking library for the system. Since the Augmented Reality system requires capturing real

time video, using a Smartphone camera is ideal and the video can be rendered in the Smartphone

display.

3

Besides developing the modules and integrating them for mobile AR, this study is also

studied the open source ARToolKitPlus library and several image processing components like

thresholding, and edge detection to deal with AR registration. Augmented Reality registration is

a method which allows the 3D virtual objects to draw and accurately align in the real

environment. As a result, the user can perceive the virtual objects in the accurate position and

orientation (Allen, 2009). According to its official website (ARToolkitPlus, 2011),

ARToolKitPlus library no longer provide updates since June 2006. Due to absence of continuing

development, ARToolKitPlus has a graphics memory handling compatible problem with latest

Symbian^3. This study enables multiple pattern based marker detection and solves the

ARToolKitPlus compatibility issue with latest Symbian^3 by integrating GOOM (Graphics Out

Of Memory) technique (GOOM, 2011) in this mobile AR system.

1.1 Background

The user can view, hear and feel the virtual object created by AR technology, which are

integrated in the real world (Bonsor, 2001). 2D and 3D computer graphics which are generated

by AR are accurately integrated into the real environment (Malik, 2002). Sutherland (1968;

1965) is created the first head-mounted display (HMD). The Touring Machine (Feiner,

Maclntyre, & Höllerer, 1997; Feiner, Maclntyre, Höllerer, & Wearing, 1999) and Mobile

Augmented Reality Systems (Höllerer, Feiner, Terauchi, Rashid, & Hallaway, 1999) were the

first mobile systems which used the new hardware platform.

4

Nusat Khidmat Maktumat Akadmu" UNIYERSTfi MALAYSIA SARAWAK

Registration is one of the biggest problems faced in building AR (Azuma, 1997; Zhou,

Wang, Yan, & Xu, 2000). Therefore, computer vision and image processing techniques are

applied in AR to solve the registration problem (Azuma, 1997; Klinker, 1999). Computer vision

is one of the areas of Artificial Intelligence (Al). Analyzing and interpreting the images is the

objective of computer vision (Center of Technology in Government, 1998). Using computer

vision, the Smartphone camera sensor can be treated as an input modality in applications by

analyzing the incoming live video (Damala, 2007).

Smartphone also provide great new possibilities for the making of 3D games and virtual

reality environments (Wolfgang, 2011). A modern Smartphone is able to display 3D graphics

that allow for high-end gaming and realistic virtual reality applications. A 3D model of a virtual

world can be projected perspectively onto the 2D screen in the direction of the viewer

(Wolfgang, 2011). Graphics Processing Units (GPUs) have boosted the graphics capabilities of

Smartphone which makes it easier for the development of interactive 3D applications which

were inconceivable just some years ago (Jose & Torres, 2012). Additionally, the modern GPU is

also based on low energy consumptions. Smartphone manufactures have quickly adopted this

new hardware because the market is demanding new applications featuring advanced 2D/3D

graphics and interactive virtual worlds. This scenario opens an extensive and interesting AR

research area in Smartphone development. 3D user interfaces with animations seem to be the

most natural Graphical User Interface method for AR applications and games (Heumer, Amor,

Weber, & Jung, 2007). Due to the lack of processing power and missing dedicated 3D graphics

hardware support, today's modern animation techniques, such as skeletal animations or vertex

5