TRILINGUAL 3D FRUIT FLASHCARDS USING

AR TECHNOLOGY

HASNUL AFIQAH BINTI HUSSIEN

Bachelor of Information Technology (Media Informatic) with Honours

Faculty of Informatics and Computing

Universiti Sultan Zainal Abidin, Terengganu, Malaysia

2018

I

DECLARATION

I hereby declare that this report is based on my original work except for quotations and

citations, which have been duly acknowledged. I also declare that it has not been

previously or concurrently submitted for any other degree at Universiti Sultan Zainal

Abidin or other institutions.

__________________________________

Name: HASNUL AFIQAH BINTI HUSSIEN

Date:

II

CONFIRMATION

This is to confirm that:

The research conducted and the writing of this report was under my supervision.

_____________________________

Name: DR. ISMAHAFEZI BIN ISMAIL

Date:

III

DEDICATION

First of all, I would like to give big thanks to my supervisor, Dr. Ismahafezi bin Ismail

for giving me a lot of advice and guidance along the development of research. His

invaluable help of constructive comments and suggestions throughout this project have

contributed to the success of this research. Lastly, my deepest gratitude goes to my

beloved family for their support and motivation throughout the process of this research.

Also, thank you for those who directly or indirectly involved in helping me finish this

project.

IV

ABSTRACT

Nowadays, the use of technology is increasing and expanding in human life.

The use of technology has expanded into many fields including research, doctoral,

educational, and many more. The technology used in education ecspecially in early

education for children is to add more knowledge and help educators to teach more

effectively. Unfortunately, there are still children who abuse the existing technology

and cause their time wasted with unfortunate benefits. Therefore, with the advent of

Trilingual 3D Fruit Flashcards using AR Technology these children can use technology

properly. Additionally, this interactive flashcard also provides an opportunity for these

children to play with the technique interaction used virtual button. The objective of this

project was to provide opportunities for these children to study and use AR (Augmented

Reality) technology more profoundly. In addition, this project can also benefit parents

and kindergarten teachers in teaching children more effectively. A variety of

multimedia elements are provided in the Trilingual 3D Fruit Flashcards using AR

Technology app with the aim of attracting children to learning. In conclusion, this

project can help children learn more effectively and become part of an IT-savvy child.

V

ABSTRAK

Dewasa ini, penggunaan teknologi semakin meningkat dan meluas dalam

kehidupan manusia. Penggunaan teknologi telah meluas ke dalam banyak bidang

antaranya bidang penyelidikan, kedoktoran, pendidikan dan banyak lagi. Teknologi

yang digunakan dalam pendidikan terutamanya pendidikan awal kanak-kanak adalah

untuk menambahkan lagi ilmu pengetahuan dan membantu para pendidik mengajar

dengan lebih efektif. Malangnya, masih terdapat lagi kanak-kanak yang

menyalahgunakan kemudahan teknologi yang sedia ada dan menyebabkan masa

mereka terbuang dengan akitivti yang tidak berfaedah. Oleh itu, dengan adanya

‘Trilingual 3D Fruit Flashcards using AR Technology’ kanak-kanak ini dapat

menggunakan teknologi dengan sebaiknya. Selain itu, kad imbas interaktif ini juga

memberi peluang kepada kanak-kanak ini bermain dengan interaksi teknik yang

digunakan iaitu ‘virtual button’. Objektif projek ini dibangunkan adalah untuk memberi

peluang kepada kanak-kanak ini untuk mengkaji dan menggunakan teknologi

AR(Augmented Reality) dengan lebih mendalam. Di samping itu, projek ini juga dapat

memberi faedah kepada ibu bapa dan guru tadika dalam mengajar kanak-kanak ini

dengan lebih efektif. Pelbagai elemen multimedia yang disediakan dalam aplikasi

‘Trilingual 3D Fruit Flashcards using AR Technology’ ini dengan tujuan untuk menarik

minat kanak-kanak belajar. Kesimpulannya, projek ini dapat membantu kanak-kanak

belajar dengan lebih berkesan dan menjadi sebahagian daripada kanak-kanak yang

celik IT.

VI

TABLE OF CONTENTS

CONFIRMATION ...................................................................................................... II

DEDICATION ........................................................................................................... III

ABSTRACT ................................................................................................................ IV

ABSTRAK ..................................................................................................................... V

LIST OF FIGURES ................................................................................................... IX

LIST OF TABLES ...................................................................................................... X

LIST OF APPENDICES ....................................................................................... ...XI

CHAPTER 1 ................................................................................................................ 1

INTRODUCTION ……………………….……………………….…………………..1

1.1 BACKGROUND ……………………………………………….……….…1

1.2. PROBLEM STATEMENT .………….….…………………………..….…4

1.3. OBJECTIVES ……………….…………………………….…………..….4

1.4. SCOPE ………………….……………………………………………...….5

1.5. LIMITATION OF WORK ………………………….……………….…….6

1.6. EXPECTED RESULT …………….………………………………..……..7

1.7. THESIS STRUCTURE ………………….……….………………………..7

VII

CHAPTER 2 …………………….……………………………………………………8

LITERATURE REVIEW ………………..…………………………………………..8

2.1. INTRODUCTION ………………………………………………..…………...8

2.2. RELATED PRODUCTS …………………………………..………………....9

2.3. RESEARCH COMPARISON …………………………..…………………...14

2.4. METHOD AND TECHNIQUE ……………………………………….……..17

2.4.1. MARKER BASED …………..…………………..……………………18

2.4.2. VIRTUAL BUTTON ……………………….…………………………19

2.5 SUMMARY ……………………..…………………………………………...20

CHAPTER 3 ………………………………...….…………………………………...21

METHODODLOGY ………………………………………………………………..21

3.1. INTRODUCTION ………………..…………………………………………21

3.2. METHODOLOGY MODEL …………………...…………………………...22

3.2.1. ANALYSIS PHASE ………………………...…………………………23

3.2.2. DESIGN PHASE …………………….………………………..………25

3.2.3. DEVELOP PHASE …………….…………..………………………….27

3.2.4. IMPLEMENT PHASE ………………………………...………………30

3.2.5. EVALUATE PHASE ………………………………………………….31

VIII

3.3. FRAMEWORK ………………………………….………………………….33

3.4. HARDWARE AND SOFTWARE REQUIREMENTS …………………….34

3.5. SUMMARY …………………………...…………………………………….36

REFERENCES ………………….…………………………………………………. 37

IX

LIST OF FIGURES

Figure 2.1: Example of Invjoy AR by Chanlong ….…………………………………9

Figure 2.2: Example of Pelangi Dicken AR flashcards ……………………………...10

Figure 2.3: Example of Wizarkids flashcards 4D+AR ……………………………...11

Figure 2.4: Example of AR 4D flashcards game by MySmartFlash ..………………12

Figure 2.5: Example of Arabic 3D flashcard by Arabic AR Studio …………………13

Figure 2.6: QR code marker ………………………………………………………….18

Figure 2.7: Image marker …………………………..………………………………...18

Figure 2.8: Process create virtual button using Unity ……………………..…………19

Figure 2.9: Interaction technique using virtual button …..…………………………...19

Figure 3.1: ADDIE Model ………………………………………..…………………..22

Figure 3.2: Analysis Phase ..................................…………………………………….24

Figure 3.3: Storyboard …..……………………………………………………...…….26

Figure 3.4: Framework .………………………………………………………..……..33

X

LIST OF TABLES

Table 2.1: Comparison table of existing products …………………………………..14

Table 2.2: Comparison table of existing products with 5 element multimedia …….16

Table 3.1: Hardware requirements…………………………………………………...34

Table 3.2: Software requirements ……………….…………………………………..35

XI

LIST OF APPENDICES

Appendix A : Gantt Chart …………………………………………………………… 38

Appendix B : Milestone ………………………………………………………………39

1

CHAPTER 1

INTRODUCTION

1.1 BACKGROUND

At present, technology is one of the most important and necessary in human life.

Without the existence of technology, humans cannot live perfectly and peacefully.

Additionally, technology upgrades are currently very high around the world, including

Malaysia. The use of technology has expanded into various fields as an example in

business, medicine, education, agriculture, and others. Furthermore, with the

technology in our lives, it can be done easily and effectively. In Malaysia, the use of

technology towards education has been growing. Teachers use technology both in and

out of class to educate their students in more depth and detail.

In addition, students use technology to share information with each other or to gain

their knowledge. Not surprisingly, children in this Z generation are often exposed to the

use of technology in their lives. Some parents take the initiative by teaching their

children to learn using modern technology. Moreover, these children can be seen

more interest in learning to use technology than using books and so on.

This new technology, Augmented Reality, also commonly referred to as AR is

defined as a real-time direct or indirect view of a physical real-world environment that

has been enhanced or augmented by adding virtual computer-generated information to

it. Augmented Reality is considered by many to not be restricted to a particular type of

2

display technology such as head-mounted display (HMD), nor is it considered to be

limited to the sense of sight. AR can potentially be applied to all senses, augmenting

smell, touch and hearing as well. (Carmigniani, December 2011) When compared to

other reality technologies, augmented reality lies in the middle of the mixed reality

spectrum between the real world and the virtual world. Furthermore, augmented reality

is changing the way we view the world or at least the way its users see the world.

Additionally, augmented reality adds graphics, sounds and haptic feedback to the

natural world as it exists. In contrast to the synthetic environment rendered onto a virtual

reality (VR) system, an AR system augments the physical environment (as seen through

the camera lens) with a synthetic representation. Thus, the environments co-existing

together can be viewed as a hybrid environment–one completing the other, and vice

versa. According to the study, children are very fond of learning that has multimedia

elements including audio, video and animation. By this element, the brain cells of the

children will be faster, more relevant and effective and learning will be faster.

Unfortunately, children in this generation are more focused on learning using Malay

language and English language only but they are not vulnerable to the use of Arabic

language which also be emphasized in their lives. Therefore, Trilingual 3D Fruit

Flashcards using AR Technology is generated.

Trilingual 3D Fruit Flashcards using AR Technology is an interactive flashcards

which is using new era technology to help the kids learn and know about fruits in

trilingual language which is in Malay, English and Arabic. This interactive flashcards

is invented specially for kids as early as age 5-6 years old with implant 5 elements of

multimedia. In addition, this interactive flashcards focus on several fruits that users have

learned before. The users need to scan the image of the fruits that created on the

flashcard using their mobile phone. Then, 3D fruit models will appear at the mobile

3

phone automatically. Besides that, the users can touch the virtual button that created on

the flashcard such as Malay button, English button and Arabic button. Meanwhile, on

the mobile apps they also can click physical button such as home, about, sound, video

and information button. This interactive flashcards has sound that gives instruction and

interaction with the users. Also, the 3D fruit models was created is suitable for the kids,

so it has many of the movement to make this interactive flashcards more attractive. Last

but not least, this interactive flashcards also has built with videos that explain about the

information of the fruits. So the users can gain their knowledge with an interactive way.

4

1.2 PROBLEM STATEMENT

As we can see, kids nowadays are currently too much open with the technology

so they may not interest too much for reading the book or dictionary beside they will

feel bored with it. Furthermore, the kids like to learn to sound which is contains

interaction and listening. Unfortunately, book or dictionary cannot be dealt with this

problem because this way of education doesn’t have a sound that can give attraction to

them. Besides that, there a lot of books or flash card using AR technology that provide

in Malay and English, but there is no attention on Arabic language for kids. At the end,

the kids will growth without knowing about of Arabic knowledge.

1.3 OBJECTIVES

The objectives of the Trilingual 3D Fruit Flashcard using AR technology are:

1. To study the AR technology that used to create 3D flashcard.

2. To design and develop 3D flashcard using AR technique with more interactive.

3. To test the functionality of the project.

5

1.4 SCOPE

The main scopes of the system:

1.4.1 ADMIN

The admin can use this interactive flashcard to make a maintenance on the

application and update a new version. Next, the admin also needs to look the

responses to the feedback which comes from the users.

1.4.2 KID

This interactive flashcard is invented especially for kids as early as age 5-6 years

old. The kids can use this Trilingual 3D Fruits Flashcard using AR Technology to

learn about fruits in trilingual language in an interesting way. The kids also can learn

with the AR interactions which, is implanted in this interactive flashcard. In

addition, they also can experience to see virtual object appear in real-world with the

animation.

1.4.3 PARENT

The parent will be able teach their kids about fruits using this Trilingual 3D Fruits

Flashcard using AR Technology. The parent also can spent time with their kids in

an interactive way.

6

1.4.4 KINDERGARTEN TEACHER

The kindergarten teacher have an opportunity to teach their students about fruits

using this Trilingual 3D Fruits Flashcard using AR Technology with more effective.

After that, the teacher can save the time to give the kid understanding about the

fruits in trilingual language.

1.5 LIMITATION OF WORK

There are several limitations and constraint that occurred throughout the development

of this interactive flashcard. These limitations in conducting this study are:

1. The user can use this interactive flashcard using mobile phone only.

2. This interactive flashcard is build only for android version.

3. This interactive flashcard is developed exclusively for learning local fruits only.

4. This interactive flashcard is build for 5-6 years old.

7

1.6 EXPECTED RESULT

At the end of this project, the expected result following the development of the proposed

project is as follows:

1. The user can use Augmented Reality technology.

2. The user will be able to improve and gain their knowledge about Augmented

Reality technology.

1.7 THESIS STRUCTURE

This thesis consists of three chapters. Chapter one is introduced that contains

project background, problem statement, objectives, scope, limitation of work, expected

result and thesis structure. Chapter two is literature review and discuss the advantages

and disadvantages that had been carried out by another existing products that related to

the on-going project. Chapter three is a methodology that discusses method or technique

that are used in the project also design framework, specifying in detail design of

development project. Chapter four is the implementation of the project and report of

testing that had been done to the development system. Chapter five is the conclusion of

the overall project followed by references.

8

CHAPTER 2

LITERATURE REVIEW

2.1 INTRODUCTION

Augmented reality is a combination of what's real and what's computer-

generated by enhancing what we see, hear, feel and smell. Its combines real and virtual

images both can be seen at the same time. When compared to other reality technologies,

augmented reality lies in the middle of the mixed reality spectrum between the real

world and the virtual world. Furthermore, augmented reality is changing the way we

view the world or at least the way its users see the world. Additionally, augmented

reality adds graphics, sounds and haptic feedback to the natural world as it exists.

9

2.2 RELATED PRODUCTS

There are several products related to the topic of the project that will be

develop. Most of them used the same methods but different features and multimedia

elements. There are some advantages and drawback that can analyze and then try to

think of the added value that can be applied in the project that will develop.

2.2.1. Invjoy AR by Chanlong

Figure 2.1: Example of Invjoy AR by Chanlong.

Invjoy AR is a one of the interactive education for kids. As well as, this

flashcards combines the real 2-D world into the 3-D virtual world with this

cutting edge augmented reality technology. The users can play with the image

on the cards as you bring them to life. They can see the 3D models come alive

from the cards. The users can get the apps free from the Play store and App

Store.

10

2.2.2. Pelangi Dickens AR Flashcards

Figure 2.2: Example of Pelangi Dickens AR Flashcards.

Pelangi Dickens AR Flashcard is developed printed both sides in all sorts

of topics such as fruits, animals, vegetables and more. It contains Q&A session

on back side for the kids to learning while gaining their thinking skill. This

interactive flashcard is designed specially in handy sized cards which is

convenient to be carried anywhere and anytime user needs. At the same time,

this interactive flashcard is great and fun learning tools as it provided a bite-

sized information that makes it easy for children to learn and remember.

11

2.2.3. WizarKids Flashcards 4D+ AR

Figure 2.3: Example of WizarKids Flashcards 4D+ AR.

4D+ AR Flashcards from WizarKids is a delightful and useful 30 pieces

of cards. The experience becomes more captivating when the kids see flashcard

images coming to life in 4D+ through the magical effects of Augmented Reality.

The kids also can personalize the 6 experiences which is reading, listening,

speaking, learning, interacting and gaming. In addition, this interactive flashcard

also build with sound interaction. So the kids will be more interesting with

learning through this. Furthermore, this interactive flashcard app is available for

android and IOS. So the user can download it from free at the App Store and Play

store.

12

2.2.4. AR 4D Flashcards game by MySmartFlash

Figure 2.4: Example of AR 4D flashcards game by MySmartFlash.

AR 4D Flashcard game creates a fantasy world for children and adults.

They can interact with the object, listen to the way it is pronounced by a native

speaker, watch it moving and know the name of the objects in different

languages. Also, users can take incredible photos to share with friends.

Furthermore, AR 4D Flashcards have 50 colorful high quality paper flash card

which is land animals, sea animals, transport, fruits and so on. This free

multilingual 4D Flashcards app based on the AR technology for smartphones/

tablets / PC / notebooks, compatible with IOS, Android, and OS Windows. It

also presently available in English (default), German, French, Spanish, Russian,

Turkish, Chinese, Tamil and Arabic languages

13

2.2.5. 3D Flashcard by Arabic AR Studio

Figure 2.5: Example of AR Arabic 3D Flashcard by Arabic AR Studio.

Arabic 4D Flashcard is an interactive flashcard with using Augmented

Reality. Arabic 4D Flashcard application displays 4D animated live models of

animals. It makes children’s learning more interesting and interactive. The user

only can play this application through android mobile phone only. Arabic 4D

Flashcard, children as young as 2 years old can learn the Arabic “Hijaiyah”

alphabets and the names of animals. The animal image on the flashcard also

generates noises and entertaining animation. The learning process of the

children with parent will be more enjoyable. Additionally, the app is currently

available in Arabic and Malay language only.

14

2.2 RESEARCH COMPARISON

There are two types of comparison table of existing products that have related

to the project that will be develop. There are platform, function, method and some of

the advantages and drawback of the products that have been analyze in table 2.1.

Meanwhile, in table 2.2, there are comparison table of existing products between 5

elements of multimedia. In this way, we can analyse and then try to added value that we

can apply on the project that we will develop.

Product Platform function Method Advantage Drawback

Invjoy

AR by

Chanlong

(2017)

Android

/IOS

Creative

learning

with 4D

Augmented

Reality.

Marker

based

method

1. Handy-sized

cards easy to

carry around.

2. Chinese and

English

language.

1. Design not

interactive.

2. No specific

topic.

Pelangi

Dickens

AR

Flashcard

by Dicken

Publishing

Ltd (2016)

Adroid/

IOS

Die-cut

flashcards

type 25pcs

fruit cards

printed both

sides. Q&A

session on

back side

Marker

based

Method

1. The pictures

are brightly

colour.

2. Versatile and

multi-

functional.

3. It contains

games and

quizzes.

1. English

language

only.

2. Lack of

element

multimedia.

WizarKids

Flashcards

4D+AR by

Wizarkids

(2018)

Android/

IOS

Creative

learning

with 4D

Augmented

Reality.

30 pieces of

Interactive

Marker

based

Method

1. Real 3D

object.

2. Its contains

game.

1. English

language

only.

2. Lack of

element

multimedia.

15

Table 2.1: Comparison table of existing products.

Fruits

Flashcards.

AR 4D

Flashcards

game by

MySmart

Flash

(2017)

IOS,

Android

phone, OS

window/

tablets /

Creative 4D

Augmented

Reality

Multilingual

language

50 colorful

of flashcard

Marker

based

Method

1. Take photos

and share.

2. Provide

multilingual.

languages

3. Interactive

design.

1. No specific

topic.

2. Lack of

element

multimedia.

Arabic 3D

Flashcard

by Arabic

AR Studio

(2016)

Android Creative 4D

Augmented

Reality

Arabic and

Malay

language

Learn about

animals

Marker

based

method

1. Interactive

animation

2. Sound

interaction

1. Not attractive

design.

2. No English

language

16

Table 2.2: Comparison table of existing products with 5 elements of multimedia.

Title text image video audio animation

Invjoy AR by

Chanlong (2017)

Pelangi

Dickens AR

Flashcard by

Dicken

Publishing Ltd

(2016)

WizarKids

Flashcards

4D+AR by

Wizarkids

(2018)

AR 4D

Flashcards game

by

MySmartFlash

(2017)

Arabic 3D

Flashcard by

Arabic AR

Studio (2016)

17

2.3 METHOD/TECHNIQUE

In recent years, much researchers has focused on developing AR applications,

which could be generally classified into two types based on the different devices used

which is optical see-through AR, and video see-through AR. Optical see-though AR

uses semi-transparent screens to project computer generated objects, by which user

could also see-through it to gain the integrated AR scene. Video see-through AR uses

cameras to capture the live scene as videos.

At each frame, video image is processed and computer generated object are

added. The mixed scene of video see-through AR could be displayed on different

devices. Markers are often used for tracking with computer vision technology in video

see-through AR. Among these recent AR applications, video-based AR has attracted

the most attention from researchers (Lu, 2007 ).

Method is a technique to reduce the work content by analysing each operation

of given piece of work very closely in order to eliminate unnecessary operations by

materials or equipments. Addition, methods or techniques that have been used in

development of this Trilingual 3D fruit flashcards using AR technology is a marker-

based and also virtual button as the added value in this project.

18

2.4.1 MARKER-BASED TRACKING

Augmented reality tracking methods depend greatly on the type of

environment and the type of AR system built. Knowing that the environment

might be indoor, outdoor or a combination of both will dictate the lightning

control capability of the system beforehand, which is an important factor in

computer vision. Likewise, whether the AR system built is meant to be static or

dynamic, or whether the environment it will be used in is known or unknown,

will also dictate the type of computer vision methods to use.

Marker is such a sign or image that a computer system can detect using

image processing, pattern recognition and computer vision techniques. Once

detected, it then defines both the correct scale and pose of the camera. This

approach is called marker-based tracking, and it is widely used in AR. The

marker can be a QR code or an image. In marker- based tracking, the system

needs to detect the marker, identify it and then display the pose. In short, if user

want to play tangible AR environment, they need to have the flashcards which

is called as markers. Then, if they scan the image on the flashcards the 3D

models will pop out automatically on the screen (Siltanen, 2012 ).

Figure 2.6: QR code marker. Figure 2.7: Image marker.

19

2.4.2. VIRTUAL BUTTON

Virtual buttons are defined rectangular regions on an image target which

can trigger an event when occluded in a camera view provided by Vuforia SDK

additional features. We assume that we have an image target to augment it with

virtual contents to allow virtual button-based interaction. Therefore, while some

mobile AR applications use touch screen of physical button on the device to

interact with the content is not feasible. (Hyejin Kim, 2015)

The suggested method is using virtual buttons which are placed on the

image target. Therefore, we can provide precise hand interaction on the image

target surface. The method uses an image target tracker to make occlusion-based

interaction button. Occlusion-based interaction using a set of visible markers in

tangible AR environments which can detect occlusions of tracked markers. This

method uses much lower computational power than bare-hand tracking while

providing similar pointing results.

Figure 2.8: Process create virtual Figure 2.9: Interaction technique

button using Unity. using virtual button.

20

2.4 SUMMARY

This chapter discusses an overview about the concept of the system. Literature

review is important to help the developer to know the problem from the previous

products that can improves or flow the on-going project. Besides, it helps the developer

in understanding the project and the chosen technique more.

21

CHAPTER 3

METHODOLOGY

3.1 INTRODUCTION

In this chapter, we will talk about methodology and the process of developing

Trilingual 3D Fruit Flashcards using AR Technology. By the way, the methodology that

has been used to develop this interactive flashcards is described. The methodology will

guide the system to solve the problem smoothly and complete the project within the

given time. There are many types of methodology that can be applied and used in the

development of AR technology. The approach must be chosen correctly for the project.

It is important to guide the researcher to manage the given task. Hence, the ADDIE

model is used for this project. Furthermore, every details about each phase that involve

in the development of this project will be explained later in this chapter. Besides that,

also rates this chapter describes the software and hardware requirement that will be used

for development process.

22

3.2 METHODOLOGY MODEL

It is important to know and understand the ADDIE model if a developer want to

develop a successful project. The ADDIE model have five phase acronyms is Analysis,

Design, Development, Implementation, and Evaluation. ADDIE is an extremely

effective tool in training development that addresses instruction. Most employees have

a significant amount of information to learn in order to become more proficient at their

jobs. Therefore, the ADDIE model could be quite helpful for many in planning a course

of action that would lead to the successful implementation of a project. When dealing

with projects that involve instruction or steps, the approach fits nicely. (Bates, 2015)

In order to ensure effective learning outcomes from augmented reality (AR)

education development, careful planning are required before the development process

begins. ADDIE Instructional Design Model is used in this mobile AR application

development methodology. The ADDIE model for instructional system design (ISD) is

a basic model that can be applied to any kind of learning solution. The ADDIE model

has five steps processes which is analysis, design, development, implementation, and

evaluation as illustrated in the Figure 3.1.

Figure 3.1 : ADDIE Model.

ANALYSIS

DESIGN

DEVELOPMENT

IMPLEMENTATION

EVALUATION

23

3.2.1 ANALYSIS PHASE

In this phase, analysis was done to identify the need to develop the

mobile AR application. The analysis phase involves are requirement analysis,

task analysis and instructional analysis. Then, the activities involve are

identifying the problem statement, the goals and objectives of the mobile AR

application. This is important because, to developed the users need, existing

knowledge and any other relevant characteristics. The mobile AR application is

developed focusing on fruits learning in trilingual language which is the content

aim for children age over 5 to 6 years old.

The analysis is also carried out by doing comparison between existing

products that implanted AR technology in their products especially in fruits

learning. Through the analysis, the weaknesses of the existing fruits learning

courseware or mobile application can be identified. All information gathered

were used to develop goals and objective of developing mobile AR application.

The flow of analysis phase is shown in Figure 3.2

24

.

Figure 3.2 : Analysis Phase

Analysis Problem

Statement

Objectives

Target

Audience

Content

1. Traditional way not

interactive.

2. Lack of element of

multimedia.

3. No attention on trilingual

language.

1. To study the AR technology.

2. To design and develop 3D

flashcard using AR technique.

3. To test the functionality.

1. Admin

2. Kids over 5-6 years old

3. Parents

4. Kindergarten teachers

This project included:

1. Trilingual Fruit flashcard

- Fruit flashcard that

contain the image of the

fruits with trilingual

language.

- Used as a marker for AR.

2. Trilingual 3D Fruit flashcards

using AR technology

- A mobile apps contain

3D model for the

Trilingual Fruit

flashcards.

- Virtual button.

Strategy

Use 3D model.

Augmented Reality

Technology.

Requireme

nts

1. Software

- Unity 3D

- Vuforia SDK

- Maya 3D

2. Hardware

- Laptop

- Mobile phone (Android)

25

3.2.2 DESIGN PHASE

The design phase endeavors to identify specific learning objectives,

topic content, presentation methods and media, learner exercises and assessment

criteria to be used. During the design phase, the information gathered during the

analysis phase is reviewed and compiled to build the project. This phase requires

logical and creative thinking. An individual must get inside the mind of the

intended audience. The designers of the program must determine how the

audience likes to learn, what the personality of the user is, what type of project

methods would best achieve the project objectives and any other obstacles that

need to be addressed. In this phase, storyboard are created to inform how the

product will be develop.

3.2.2.1 Storyboard

Storyboard is also a part of multimedia design process. The

storyboard shows the look and feel of the application that will be

developed. Some of the storyboard can be seen in interface prototype.

26

Figure 3.3 : Storyboard.

27

3.2.3 DEVELOP PHASE

Development is the third phase of the planning process. During the

development phase, the actual course materials are created. A successful

development phase uses the information collected in the analysis phase and the

decisions made in the design phase. In the development phase pipeline is created

which production and assembly of the materials that were developed in the

design phase. In this phase, all steps in development of the project are collected,

prepared and created.

In the development phase, developers create and assemble content assets

blueprinted in the design phase. The project is reviewed and revised according

to feedback.

3.2.3.1 Production

a) Create marker

Before starting develop the 3D fruit models, the fruit flashcard must

be created earlier using Adobe InDesign. This flashcard used as for

markers which is when we scan the image that created on the flashcard

its will pop out the 3D fruit models at the mobile phone screen.

b) Animating 3D characters

Next process in production is animating 3D fruit models using

Autodesk Maya. There are 5 steps that included during the process:

28

- Modelling :

In this process, the 10 of 3D fruit models is created.

Reference images are required throughout this process.

- Texturing :

In this process, the 3D fruit models is applying with color

&surface properties. The surface of model must look like it does

match real-world and concept art.

- Rigging :

In this process, rig the 3D fruit models to make the

movement in the next process which is in animation process.

Furthermore, joint controller, skinning, enveloping, muscle

system and floating GUI is involved.

- Animation :

In this process, the 3d fruit models movement is created.

This process was the hardest part in animating 3D fruit models.

Then, it will make the 3D fruit models animation more

interesting and realistic.

- Rendering :

This process is the last process in animating 3D fruit

models. Rendering is a process which is painting all component

in the production stage to make final output.

29

c) Create 2D animation video

This process is to make the 2D animation video which is place the

video at the music video page. The process will be involved 2D

animation and song about the fruits. The process will be create using

Adobe Flash and Adobe Premiere.

d) Building AR

In this process, the 3D fruit modesl of the fruits will be imported into

Unity 3D software. Then, the developer can started building AR

experiences using Unity 3D and Vuforia Platform.

e) Interaction Button

Adding virtual buttons provide by Vuforia to the scene, which

simply means that we add buttons to the AR world, which can be touched

in the real world.

f) Building mobile interfaces and features

In this process, the mobile interfaces and features are created using

Adobe InDesign and Adobe Illustrator. The objective is to make the

mobile application more attractive and users understand the

functionality.

30

3.2.3.2. Post-Production

a) Coding

During this process, the developer must implement language c# to

the desired outcome of the project based on the findings during the

research phase.

b) Final Output

After all of steps in production stages are completed, final output

have been done and ready to perform.

3.2.4 IMPLEMENT PHASE

The implementation phase refers to the actual delivery of the instruction.

The purpose of this phase is the effective and efficient delivery of instruction.

This phase must promote the users understanding of material, support the users

achieve of objectives, and ensure user transfer of knowledge from the

instructional setting to the job.

31

3.2.4.1 Test Run

a) Test Run Project

This is when the developer test runs the project by herself. This is to

ensure that the project can be used and is running accordingly.

b) User Test Run

Before presenting the final product, developer give to the user test

the project and give the feedback. Criticisms and constructive feedback

is taken to be implemented later on.

3.2.5 EVALUATE PHASE

This phase measures the effectiveness and efficiency of the instruction.

Evaluation should actually occur throughout the entire instructional design

process within phases, between phases, and after implementation. Evaluation

may be formative or summative.

32

3.2.5.1. Debug

a) Identify Errors

After the test run had been done, errors that were found by the

users must be identified. Glitches and mistakes must be brought to

light for the developer to improve the project while also

implementing the feedback given by the alpha and beta testers.

b) Find Solutions

Once errors have been identified, solutions to solve them must

be done.

3.2.5.2. Final Product

a) Present Product

Product is ready to be presented and launched for a large

audience to use it.

33

3.3 FRAMEWORK

Figure 3.4 : Framework.

Figure above shows the framework design of Trilingual 3D Fruit Flashcards using AR

Technology. Vuforia SDK is a platform to create Augmented Reality for mobile

devices. The databases which are image target are saved there. Unity used to create the

Augmented Reality adn overall project. Meanwhile, User able to choose any button in

the mobile application. User also able to scan any image on the flashcard for display 3D

animation models. Addition, User can play with virtual button that implanted at the

image target. For Admin, Admin can update this application if there is features that need

to be improve. Furthermore, Admin also able to view the response the feedback that

coming from the users.

34

3.4 HARDWARE AND SOFTWARE REQUIREMENTS

Software and hardware are one of the important things in the making this project

completely as it is needed for the application development. The section will show the

list of all software and hardware requirement that involved in the development

process.

3.4.1 Hardware

HARDWARE USE FIGURE

Laptop

To create the sketches for the

storyboard, flashcard design

and create some info related to

“Trilingual 3D Fruit Flashcard

using AR Technology”. Also

used for on the coding and 3D

modelling.

Mobile Phone

(Android)

Used to play with AR

experiences using mobile

application. Its camera also

used to scan the image at the

flashcard. Then, its speaker

used to give the sound.

Table 3.1 : Hardware requirements.

35

3.4.2 Software

SOFTWARE USE FIGURE

Unity Used to create the

Augmented Reality app

and overall project.

Vuforia SDK

It uses computer vision

technology to recognize

and track Image target

and simple 3D objects,

such as boxes, in real

time.

Maya Autodesk Used to create 3D

character models.

Adobe

Illustrator/

Adobe InDesign

Used to create design of

flashcard and mobile

app interface.

Adobe Flash

Player

Used to create 2D

animation video.

Adobe Audition To record sound if

required.

Adobe Premiere To make video of 2D

animation video.

Table 3.2 : Software requirements.

36

3.5 SUMMARY

Based on this chapter, the methodology used for the project was elaborated.

The project requirements that were used are also stated. Choosing methodology to be

used for a system development is a crucial part in ensuring that the development start

off at a great place. The next step is the implementation of the project. The project will

be developed by using the right method to help developer to make the project more

efficient and systematic.

37

REFERENCES

[1] Bates, A. W. (2015). Teaching in a Digital Age,The ADDIE model. Anthony

William (Tony) Bates.

[2] Carmigniani, J. (December 2011). AUGMENTED REALITY METHODS AND

ALGORITHMS FOR HEARIN. Boca Raton, Florida: Florida Atlantic

University .

[3] Hyejin Kim, E. A.-I. (2015). Dynamic hierarchical virtual button-based hand

interaction for wearable AR. Korea: IEEE.

[4] Lu, Y. (2007 ). Methods for Augmented Reality E-commerce. Ames, Iowa:

Iowa State University.

[5] Marco. (2014, May 4). Virtual buttons in Augmented Reality with Vuforia.

Retrieved from marcofolio.net: https://marcofolio.net/virtual-buttons-vuforia/

[6] Siltanen, S. (2012 ). Theory and applications. Finland : VTT.

38

APPENDIX A: GANTT CHART

No. Activities Week

1. Topic Confirmation &

Discussion with

Supervisor

2 Project Title Proposal

3 Submission of

proposal

4 Preparation of

Chapter1/Chapter 2

5 Proposal Progress

Presentation &

Evaluation

7 Discussion &

Correction Proposal

8 Proposed Solution

Methodology

(Chapter 3)

9 Develop prototype

10 Drafting Report of the

Proposal

11 Submit draft of

report to supervisor

12 Seminar Presentation

& Final presentation

13 Correction Report

14 Final Report

Submission

39

APPENDIX B: MILESTONE

No Activities Months (4 months)

First week

September

–

Second week

September

Third week

September

-

Fifth week

October

Sixth week

October

-

Ninth week

November

Tenth week

November

-

Fifteenth

week

December

1 1.1 Topic Discussion

and determination

project title proposal

2 2.1 Proposal writing

2.2 Literature review

2.3 Proposal

progress

2.4 Presentation

& Evaluation

3 3.1 Discussion

correction proposal

3.2 Proposed

solution

3.3 Methodology

3.4 Proof of concept

4 4.1 Drafting report

of the proposal

4.2 Submit draft of

report to supervisor

4.3 Seminar

Presentation

4.4 Correction

Report

4.5 Final Report

Submission