AR FLASHCARD: LET’S LEARN MUSICAL INSTRUMENTS · 2021. 2. 3. · flashcard for helping children...
Transcript of AR FLASHCARD: LET’S LEARN MUSICAL INSTRUMENTS · 2021. 2. 3. · flashcard for helping children...
AR FLASHCARD: LET’S LEARN MUSICAL
INSTRUMENTS
NUR HADHIRAH HUSNA BINTI HAMDAN
BACHELOR OF INFORMATION TECHNOLOGY
INFORMATICS MEDIA WITH HONOURS
UNIVERSITI SULTAN ZAINAL ABIDIN
2021
AR FLASHCARD: LET’S LEARN MUSICAL INSTRUMENTS
NUR HADHIRAH HUSNA BINTI HAMDAN
BACHELOR OF INFORMATION TECHNOLOGY
INFORMATICS MEDIA WITH HONOURS
Universiti Sultan Zainal Abidin
2021
i
DECLARATION
I hereby declare that the report is based on my original work except for quotations and
citations, which have been duly acknowledged. I also declare that it has not been
previously or concurrently submitted for any other degree at Universiti Sultan Zainal
Abidin or other institutions.
_______________________________
Name: Nur Hadhirah Husna
binti Hamdan
Date: 8 January 2021
ii
CONFIRMATION
This is to confirm that:
The research conducted and the writing of this report were under my supervision.
_______________________________
Name: Tolahah Bin Muda
Date: 8 January 2021
iii
DEDICATION
In the Name of Allah, the Most Gracious and the Most Merciful.
Firstly, I would like to express my sincere thanks and appreciation to my supervisor,
Encik Tolahah Muda, who guide me and providing me guidance throughout the
development of this project. Without this help, this project would not completed
smoothly.
Next, my gratitude goes to panels, fellow lecturers and friends who also had given me
some idea and guidance to develop the project.
Finally, I am most thankful to my parents for providing me with the best they could
throughout the course of my study, especially during the project development. Many
things wouldn’t be possible without their unconditional love, encouragement and
support.
iv
ABSTRACT
AR FLASHCARD: LET’S LEARN MUSICAL INSTRUMENT are developed for
primary school students who learn music subjects. The flashcard will be based on
several types of musical instruments that known as percussion and blowing
instruments such as Tamborin, Kastanet, Kerincing, Kompang, Marakas and Rekorder
for helping user to visualizes the musical instruments, how the instruments sound and
video tutorial about the musical instruments. This applications will combine
multimedia elements such as 3D model object, audio and video for making the
studying way about musical instrument will be interesting and gain more
understandable. Multimedia Development LifeCycle (MDLC) methodology has been
used and discuss about the design process and development of the application. Also,
this application was invented for primary school students, teachers and other kids as
early as the age 5 – 6 years old for gaining knowledge in learning music using
Marker-Based Augmented Reality (AR) technique. This interactive application hoping
will be a great potential for helping teachers, students and kids for a better teaching
and learning session. This application method also will be useful for other subject
learning session.
v
ABSTRAK
AR FLASHCARD: LET'S LEARN MUSICAL INSTRUMENT dibangunkan untuk
pelajar sekolah rendah yang belajar subjek muzik. Flashcard akan didasarkan pada
beberapa jenis alat muzik yang dikenali sebagai alat perkusi dan tiupan seperti
Tamborin, Kastanet, Kerincing, Kompang, Marakas dan Rekorder untuk membantu
pengguna memvisualisasikan alat muzik, bagaimana bunyi instrumen dan tutorial
video mengenai muzik instrumen. Aplikasi ini menggabungkan elemen multimedia
seperti objek model 3D, audio dan video untuk menjadikan kaedah belajar tentang
alat muzik menjadi menarik dan lebih difahami. Metodologi Multimedia Development
LifeCycle (MDLC) telah digunakan dan membincangkan mengenai proses reka
bentuk dan pengembangan aplikasi. Aplikasi ini dicipta untuk pelajar sekolah rendah,
guru SK dan kanak-kanak lain seawal usia 5 - 6 tahun untuk menimba pengetahuan
dalam mempelajari muzik menggunakan teknik Marker-Based Augmented Reality
(AR). Aplikasi interaktif ini berharap dapat menjadi potensi besar untuk membantu
guru, pelajar dan anak-anak untuk sesi pengajaran dan pembelajaran yang lebih
baik. Kaedah aplikasi ini juga akan berguna untuk sesi pembelajaran subjek lain.
vi
CONTENTS
PAGE
CONFIRMATION ii DEDICATION iii ABSTRACT iv ABSTRAK v
CONTENTS vi LIST OF TABLES vii LIST OF FIGURES viii LIST OF ABBREVIATIONS ix CHAPTER 1 INTRODUCTION 10
1.1 Project Background 10 1.2 Problem Statement 13 1.3 Objectives 14
1.4 Scope 14 1.5 Limitation of Work 16 1.6 Activities, Milestones (Gantt Chart) 17 1.7 Expected Result 18
1.8 Summary of the Chapter 18
CHAPTER 2 LITERATURE REVIEW 19 2.1 Introduction 19
2.2 Technique 19 Marker Based AR 20
Markerless AR 21
2.3 Related Products 22
Comparison of Related Products 25 2.4 Research Comparison 26
2.5 Chapter of Summary 28
CHAPTER 3 METHODOLOGY 29 3.1 Introduction 29
3.2 Methodology Model 29 Concept Phases 30
Design Phase 32 3.2.1.1 DESIGN AR FLASHCARD 32 3.2.1.2 DESIGN INTERFACE IN THE AR APPLICATION34
Material collecting 41 Pictures 42
Assembly 43 Testing 44
Distribution 45 Conclusion 46
REFERENCES 47
vii
LIST OF TABLES
Table No. Title Page
Table 1.1 Gantt Chart FYP I 17
Table 2.1 Summary Related Products 25
Table 2.2 Summary Research Comparison 26
Table 3.1 Description of Application Concept 30
Table 3.2 Software Requirement 43
viii
LIST OF FIGURES
Figure No. Title Page
Figure 2.1 Marker-Based AR 20
Figure 2.2 Markerless AR 21
Figure 3.1 Multimedia Development Life Cycle (MDLC) model 30
Figure 3.2 FRONT FLASHCARD 33
Figure 3.3 BACK 33
Figure 3.4 MAIN MENU 34
Figure 3.5 HOW TO PLAY 35
Figure 3.6 VIDEO GUIDANCE 36
Figure 3.7 AR VIEW BEFORE 37
Figure 3.8 AR VIEW AFTER 38
Figure 3.9 AR VIEW (VIDEO) 39
Figure 3.10 CREDIT 40
ix
LIST OF ABBREVIATIONS
AR Augmented Reality
10
CHAPTER 1
INTRODUCTION
This chapter present the problem statement found from research, background and
objective of the project and proposed approach to achieve the goal of project
1.1 Project Background
Malaysian Music Curriculum for primary school has been implemented since 1983. In
addition, the music lesson has been introduced in Malaysian Higher Institution more
than 30 years and above. From there, there’s lot of higher institution such as
universities out there do offer music courses for the students to pursue their interest in
music whether in education field or music industry. Several results have been showed
that students who take music as an elective subject had shown good performance in
the national exam conducted by Malaysian Ministry of Education on 2004. In
additions, there are a lot of benefits by learning music which enable a person to
achieve success in their life-time.
In general, music education in Malaysia for primary school cover on topic learning
music percussion that is hammered, tapped or shaken to produce sound such as
kompang, kerincing, castanet, maracas and tambourine. The primary school syllabus
also do cover on students to identify music from music collections which consists of
recognizing types of string musical instruments such as guitars and blowing
instrument such as recorders. Therefore, music education in Malaysia consists of
traditional Malay music and as well as classical western music. Unfortunately, lack of
learning experience to study the musical instruments become one of the challenging
11
scenarios in the Malaysian music education. The current trend music education
content can be experienced through a wide variety of media ranging from non-
interactive to interactive digital experience such as e-books with multimedia,
simulation that are more interactive and immersive among the students as most of
learning method nowadays are proven through digital experience such as AR
flashcard for helping children to learn the alphabets and developing their mind’s
recognize and pronounce words appropriately.
Augmented Reality (AR) has increased popularity in both industry and education field
since its introduction because AR is about the experience where it combines the real
world with virtual world and it can be presented through various channels such as
smartphones as well as personal computers and laptops. The display presentation
could contain video, 3D modelling, images, animations and sound elements where
they are suitable for the development of educational material. AR technology allows
users to visualize virtual objects in the real world.
The AR flashcard: let’s learn musical instruments is an interactive flashcard
containing musical instruments to help the students learn information and recognizes
the musical instruments with sounds. This interactive flashcard is suitable for primary
school students as the reference for learning. The flashcard also create for kids as
early as the age 6 years old to learn musical instruments in more engaging way as it
comes with multimedia elements. Plus, the flashcard also will be useful for primary
school teachers as a teaching and learning aid in the classroom especially for those
who are not in the field of music but have to teach the subject. The AR flashcard focus
on 6 types of musical instruments such as kompang, kerincing, castanet, tambourine,
12
marakas and recorder. The users need to scan the image of musical instruments using
their mobile phone and 3D models of the musical instruments will appear on the
mobile phone automatically.
For this application, I develop the AR application for learning musical instruments
that will available with musical instruments information, sounds and video tutorials of
the musical instruments learned. Perhaps, this project would receive user’s
satisfactions and receive good words to improve learning method quality.
13
1.2 Problem Statement
1) Most students and kids nowadays need interactive learning methods with
multimedia elements such as sounds to make it more understandable, easier to
remember and interesting.
2) There are some teachers who have no idea for music subject teaching materials
that make the learning environment different so that students are interested in
learning music and pay more attention during class time.
3) Most of education field do implements their lessons materials in Augmented
Reality (AR) technology, so why not to apply this method of learning to cover
musical instrument topic in music subject.
4) The learning methods using books and slide only are less interactive as it’s
provide students to read and observe the content only. Meanwhile, with AR
application students tends to have interaction while getting information during
the lessons.
14
1.3 Objectives
There are several goals in this project:
1) To study AR mobile application for helping students learn and recognizes
about musical instruments in music subject better with multimedia element.
2) To develop an interactive learning for assist teachers and students as reference
material for teaching materials.
3) To develop AR flashcard about musical instruments that helping teachers who
are not music options teach music subjects better.
4) To test as added value for learning music subjects by providing knowledge
about the types of musical instruments and information in digital form among
the children.
5) To test the new way learning for getting information with augmented reality
technology is functional in music education field among primary school
students.
1.4 Scope
The main scopes of the project:
1. Application Scope
The scope for this project focuses on 6 types of musical instruments that cover
lesson in music subjects for primary school students and teachers. It is intended for
them to visualize and recognizes the 3D model musical instruments. This applications
provides guidance on usage during the lessons. At the end of the project, the user
would be visualizes while observing with information about musical instruments,
sounds and video tutorial. Then, the students will be able to recognize musical
instruments they learn so, they will become familiar and easier to remember.
15
2. User Scope
Besides, this project also aimed to students and other kids to learn about musical
instruments interestingly. The project also aimed to primary school teachers as the
added material during the class. This augmented reality project comes with coloured
flashcard. They will learn by scan the AR flashcard and get information type of
musical instruments in this Augmented Reality application. The sound will be
available for every musical instruments to enhance user experience and the
applications will be in Android platform.
In order to develop this project, several properties such as hardware, software and
technology are required. The specs of laptop used to develop project available with
Intel core i5, 12 gb DDR4 for faster speed performance and SSD storage. The mobile
phone also be needed to run and testing the project and it is used for the android
version mobile applications. Meanwhile, software requirement intended into this
project are Adobe Photoshop CC to create graphic design on marker. Autodesk Maya
used to modelling the 3D environment, Unity to create virtual reality app, overall
project and modelling the 3D environment too. The technology applicable in this
project are in form augmented reality in android platform.
16
1.5 Limitation of Work
This flashcard only available for 6 types of musical instruments for percussion and
blowing instruments only such as tambourine, maracas, kompang, kerincing,
castanet and recorder. Perhaps it may covers as a guidance for them to learn basic
musical instruments. Also, this project is only build for android version mobile
application. But still, the user able to get the lessons through AR flashcard to learn
musical instrument. The application is built and only suitable for kids at the age 6-12
years old and primary school music teachers.
17
1.6 Activities, Milestones (Gantt Chart)
Table 1.1 Gantt Chart FYP I
Task
Week
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Discussion with supervisor
Research the project
Topic Discussion and
Determination
Project Title Proposal
Proposal Writing -
Introduction
Proposal Writing –
Literature Review
Proposal Progress
Presentation and Evaluation
Correction Proposal
Proposed Solution
Methodology
Proof of Concept
Final Year Project Format
Writing Workshop
Drafting Report of the
Proposal
Submit Draft of Report of
the Proposal
Preparation for Final
Presentation
Final Presentation and
Panel’s Evaluation
Final Report Submission
18
1.7 Expected Result
This project is expected as advanced existing methods of mobile based-learning, to
view popup 3D model musical instruments and informative from the flashcard. Thus,
this project will bring about an immersive experience for students, kids to become
familiar and have a better understand in learning. In addition, this project also support
with Augmented Reality technology, 3D modelling and flashcard so it will be
different experience for users to learn musical instruments using AR mobile
application. User also will also be able to test their knowledge after by answering
questions such as recognizing the musical instruments with name, sound etc. after
ended the lesson.
1.8 Summary of the Chapter
This chapter discussed about the project’s background, problem statement, project
objective, scope, limitations of works and expected results.
19
CHAPTER 2
LITERATURE REVIEW
2.1 Introduction
In this chapter, a survey of some existing works to answer the related problems in
chapter 1 are introduced. This chapter describe and explain literature review that have
been carried out on the application that will be used as references in developing this
application. Previous researches or existing application will be discussed in this
section and make a comparison between each research with the proposed project.
Besides that, review on research papers will also help to improve the proposed project
through reviewing the strengths, weaknesses and solution author had provided in the
paper
2.2 Technique
Related techniques refer techniques of Augmented Reality that already existing and
used until now. Two types of augmented reality are known which marker-based AR
and markerless.
20
Marker Based AR
The marker based AR is also called as location-based AR which uses cameras and
visual prompts. Marker-based AR typically uses the camera feature of mobile devices
or computer to analyze and capture the markers. Users can observe different view of
the real objects (3D model) by pointing the camera to the markers. For example, QR
codes are mostly used in the type of AR. This study will focus on Marker-Based AR
as the project will develop in Marker-Based method. Usually, 2D images are used to
be the visual markers. Users can get the result marker in 3D virtual model when point
the camera to the marker and need to move the markers during scan to view the 3D
virtual model in different angle.
Figure 2.1 Marker-Based AR
21
Markerless AR
Same goes functioning with Marker-Based AR but the Markerless AR are different
because this technology are not require a marker to retrieve the 3D content. The
markerless AR will places virtual 3D objects in the physical environment on location-
based. Markerless AR will detect objects when points the camera to a scene in
environment such as walls or any points until the 3D object appear on the user phone
screen. The markerless systems used a device’s location services, sensors and camera
systems to enable AR application to detect location and orientation virtual objects in
real world environments. Markerless AR helps the user to see wider AR view. The
Markerless AR has given a significant impact in gaming evolution such as Pokemon
Go that enables the user to interact with their surroundings and it feels comes alive
based on where they look. This game has reach more than 1 billion users worldwide.
Markerless AR are also possible in interior design industries. For example, IKEA has
enable their customers to simulate furnitures and virtually place household items to
design inside their homes.
Figure 2.2 Markerless AR
22
2.3 Related Products
Learning the Malay Traditional Musical Instruments by using Augmented
Reality Application
This application uses a book as a marker to help user visualizes about Malay
traditional musical instruments. The application also do helps the user learn the
history behind of every musical instrument and how the intruments sounds. The AR
application in this project is expecting to be a tool for promoting traditional musical
for the new young generation that is consist of musical instrument belong to 3 national
races in this country Malay, Chinese, Indians and other ethnics. For the application
content, 3D model will display with its history information and instruments sound.
The application do provide the book that consist of 4 types of Malay musical
instrument such as rebana, gong, gambus and seruling. In this AR application, there
will be 2 parts for the user to interact, the user may scan the book marker to view 3d
model, click at the ‘Rotate’ button and ‘Sound’ button in the applicaton during the
exploration. The book marker design will be content with name and history of the
musical instruments.
23
AR Malaysian Traditional Instrument Demo
This AR application are intended for the users to learn on Malaysian cultures by
promoting Malaysian Traditional Musical Instruments. Besides, the AR application
are created to encourage our younger generation to appreciate and know the origin of
the musical instruments. The instruments application are pairing with flashcard for
user to view the 3D appearance of beautiful color and texture of traditional musical
instruments. The AR view in the application also allows the user to listen every
instrument sounds and melody. The marker in this application contain information for
every instruments which are the height, creator and origin about the instruments. This
application are content with 15 types of traditional musical intruments which is good
to assist in encourage our generation to know about Malaysian culture.
24
X Drum – 3D & AR
This application is an app playing 3D Drumkit App and Piano. In this application, the
user are able to learn and play 3d Drum and Piano by tapping onto the instruments
drum pad and piano keyboard following the flying notes with color as a guidance. The
user also will play real 3D drum kit and a double keyboard based on Garage and Club
Stage scene. This application do has features such as play over hundred of drum loops
in different genres, 8 studio quality drum sets and electronic beat sounds, play dual
row piano keyboard with 8 sound effect and user also able to project real drum kit in
room with AR Technology. This application is actually for helping user playing and
practicing to play drum in impressive way by following the flying notes as guidance
for the starter to play instruments with the tempo.
Space 4D+
The Space 4D+ is a Marker Based-AR in form of flashcard for user to learn about
space. It is a very convenience app as it brings the enjoy learning surrounding with
flashcard. The Space 4D+ is a fast and lightweight app that allows the user to scan
colourful oriented space cards and then witness how the space alive. The Space 4D+
gives the user learn with interesting and informative facts content about Solar System,
Planet, Space objects, Satellites, Rovers, Space Mission and many more in
Augmented Reality. This mobile application do compatible with all devices.
25
Comparison of Related Products
Table 2.1 Summary Related Products
Learning the
Malay
Traditional
Musical
Instruments
by using
Augmented
reality
Application.
X Drum –
3D & AR
Space
4D+
AR
Malaysian
Traditional
Instrument
Demo.
AR
Flashcard :
Let’s
Learn
Musical
Instruments
Image Yes Yes Yes Yes Yes
Animation
/
Video
No Yes Yes No Yes
Audio Yes Yes Yes Yes Yes
Text No Yes Yes No Yes
AR-
Method
Book Marker
Based AR
Markerless
AR as user
able to
project
own AR.
Flashcard
Marker
Based
AR
Flashcard
Marker
Based AR
Flashcard
Marker
Based AR
26
2.4 Research Comparison
Table 2.2 Summary Research Comparison
Paper Method/
Model/
Technique
Descriptions Advantages Disadvantages
Teaching
Children
Musical
Perception
With MUSIC
AR
Music -
Augmented
Reality
technology for
teaching
sound
properties.
Music-AR
allows the
children to
manipulate
virtual objects
sounds and
focus on the
concept of
sound
intensity.
Children feel
motivated and
could
understand
music with
Augmented
Reality.
Lack of
activity games
in the
application.
Applying
Augmented
Reality
Technology to
Promote
Traditional
Thai Folk
Musical
Instruments on
Postcards
To promote
the traditional
folk musical
instruments.
The method
allows the
users view the
media or
music
instrument in
2D modelling
provide with
sound when
scan image in
a postcard.
The method
get
engagement in
the classrooms
so the students
are eager to
learn during
the class and
encourage
them to have
knowledge
1.Required to
use internet
connection.
2.Graphic is
only in 2D
modelling.
27
about their
own culture.
Motivation
Measurement
of an
Augmented
Reality
Traditional
Musical
Instruments
System
Presented an
Augmented
Reality to
introduce
traditional
musical
instruments
among the
children in
order to grow
back their
interest in
learning their
own culture.
The purpose
of the project
is to gain
interest and
motivation of
children in
learning
traditional
musical
instruments in
Bangka area.
The
application
was build in
android based
and content
with
traditional
Bangka music
instruments.
System built
on markers
using vuforia
and unity
tools about
Bangka
traditional
music
instruments.
1.AR could be
used to attract
the interest
and
motivation of
users to get to
know music
instruments.
2.Results from
respondents
shows that
motivation in
arousing the
interest of
children to
learn their
culture music
instruments in
Bangka region
are influence
them.
3.AR can be
used to
increase
motivation in
learning
something.
1.The elements
in application
are only in
form of 3D
model.
28
2.5 Chapter of Summary
In this chapter, discussed about the literature review as a reference for the project
development. The literature review helps developer to discover the problem of
previous research, applications that can improve the flow on-going-project. Besides, it
is also gain understanding about the application during the development process.
29
CHAPTER 3
METHODOLOGY
3.1 Introduction
This chapter will briefly explain the details of the methodologies and procedures to
develop the whole application. The project methodology should systematically solve
all the problems arise in the system analysis to make sure this project complete and
working well. There are many types of methodology that can be applied and used in
the development of AR technology. The approach chosen is important since it will
guide researcher to manage the given task. Hence, the system structure will plan and
develop according to the Multimedia Development Life Cycle (MDLC) method.
Every details abot each phase that involve in the development will be explain in this
chapter.
3.2 Methodology Model
Multimedia Development Life Cycle is used in this research, which is developed by
Luther-Sutopo. Multimedia Development was chosen based on steps of the method
where each stage will be explained in detail. According to Luther-Sutopo (2003),
argues that Multimedia Development Life Cycle consists 6 (six) steps process used, i.e
concept, design, material collecting, assembly, testing, distribution and
implementation.
30
Figure 3.1 Multimedia Development Life Cycle (MDLC) model
Concept Phases
Based on the introduction it can be identified that utilization of Augmented Reality
technology provides a positive impact to the understanding of learning materials for
user. This is underlying development application based on the Augmented Reality as
an alternative learning media.
Table 3.1 Description of Application Concept
Title AR Flashcard : Let’s Learn Musical Instruments
Types of
Applications
1.AR-based learning application
2. Use 3D model
Target Audience The user of the application is intended for primary
school students that learn music subject, kids as early as 6
years old and school teachers.
31
Content This project included:
1.Flashcard of 6 types Musical Instrument
- Contain images images picture with name.
- Used as a marker based for AR
2. Activities
- mobile applications contain 3D model
- sound of musical instrument
- video tutorial at the back of flashcard
- True or False Question / Multiple choice question
Limitation of
works
1. Application only built for kids aged 6 years and
above, primary school students and teachers.
2. Only available on android mobile.
Requirement 1. Software
- Autodesk MAYA, Unity 3D, Vuforia
2. Hardware
- Laptop & android mobile phone
32
Design Phase
Design phases will be explained on the design of the application that will be
developed. In this phases, the content for the application need to be designed and
visualized so it will be easy for user to reach the information. The developer will make
storyboard book design to create a visual map of application. Design process then
done by created a marker design, interface of the application and 3D model
description in application interface which will be used as the AR Flashcard: Let’s
Learn Musical Instrument.
3.2.1.1 DESIGN AR FLASHCARD
Marker Based AR
To design the marker will use several image of the musical instruments that are
consist 6 types musical instrument known as percussion instruments and blowing
instrument such as tamborin, marakas, kerincing, kastanet, kompang and recorder
that used in musical subjects for the skill of playing percussion instruments with
tempo and playing recorders. The marker is edited using Adobe Photoshop and will
print in form of a Flashcard. The marker design consist with instrument names and at
the back of marker will be a medium to scan for video tutorial (how to play the
musical instruments).
33
Figure 3.2 FRONT FLASHCARD
Figure 3.3 BACK
34
3.2.1.2 DESIGN INTERFACE IN THE AR APPLICATION
To develop the interface will start with design some functions that will be apply in the
AR mobile application. The interface that will be in the application consist of the
MAIN MENU with musical instruments element as the background to show that
application domain about learning musical instruments. In MAIN MENU, there are
several button provided to connect with other interfaces. Every buttons are consist of
function for AR CAMERA, user guidance that known as HOW TO PLAY, another
user guidance for playing video named ABOUT VIDEO at the button and lastly there
will be CREDIT button to connect with CREDIT interface.
MAIN MENU is the main home page in this application. User will explore in this
application through button they choose at the main menu. On main menu, user only
need to click AR CAMERA button to start play.
Figure 3.4 MAIN MENU
35
Interface HOW TO PLAY is as user guidance page for the use of this application.
Page will display 2 steps as a user guide. Step 1 instructs the user to scan the camera
on the flashcard and step 2 informs the output produced after the user scans the
flashcard.
Figure 3.5 HOW TO PLAY
36
Next, the ABOUT VIDEO page is a continuation of step guide to users. Step 3
informs the user to scan the back of the flashcard to view the interactive video of the
musical instrument.
Figure 3.6 VIDEO GUIDANCE
37
After the user clicks on the AR CAMERA button, the application will go to the AR
VIEW page. Page AR VIEW is for users to scan the flashcard of a musical instrument.
Figure 3.7 AR VIEW BEFORE
38
After the user scans the flashcard, then the output will be produced on their phone
screen in the form of a 3D model. The AR VIEW display, user can click on the voice
icon button to say the name of the musical instrument in english or the sound of a
musical instrument. On the AR VIEW page display there is also a question mark
button for users to click and go to the HOW TO PLAY page.
Figure 3.8 AR VIEW AFTER
39
This display is for users to scan the flashcard section to play videos. Users just need to
click on the AR CAMERA button in the main menu / home page to play a video
tutorial of musical instruments.
Figure 3.9 AR VIEW (VIDEO)
40
Credit page is to display the developers involved in the production of this application.
Figure 3.10 CREDIT
41
Material collecting
The materials needed in the making of this application are obtained from several
sources on the Internet with some modifications.
Various resources that have been used as a reference for the development of this
project. Among them is a reference for the types of musical instruments that will be
placed in this application. Reference sources for the content of musical instruments in
this application have been obtained from internet sources such as referring to the
standard curriculum slides of music subjects for primary school students. In addition,
several interview sessions were conducted online with a primary school teacher
teaching music subjects. According to the interview session, students at the primary
school level were exposed to musical instruments that produce percussion sounds such
as percussion instruments and blowing skills, namely recorders.
After the research is done. The content that has been set in this application is as
follows:
Content
6 types of musical instruments have been selected which consist of percussion
and wind instruments namely tambourine, marakas, kerincing, kastanet and
rekorder.
Information such as musical instruments in Malay and English.
Video tutorial for each musical instrument.
Finally, questions in the form of correct or incorrect form and answer choices
will be made as a measure of their memory after using this application.
42
Pictures
The collection of project materials such as pictures has been obtained from several
reference sources on the internet for free on freepik and pngwing with some
modification and appropriate to the project. These images have been taken for the use
of flashcard generation and application interface. For example, a 2D model musical
instruments will be obtain from freepik for designing the interface of the mobile
application. The 3D model development will be needed the real picture of the musical
instrument for modelling. For application interface, all the picture will be downloaded
and then processed for design in Adobe Photoshop CC 2017.
Youtube
Youtube tutorial is essential as the main reference for developing the 3D modelling of
each musical intruments. The audio and video material for video tutorial on how to
play the musical instrument will also refer to Youtube resources. The audio and video
obtain then will be saved in MP4 file format for developing the mobile application.
Design and Layout interface
The design and layout process for mobile application musical instrument also do
obtain from other mobile applications and dissertation. The reference of the interface
design also refer to pinterest so the interface application will looks appealing and suits
with the domain of the application. As this project will be about musical instruments,
so the element of musical instrument will be apply as the background of the interface
mobile application.
43
Assembly
Assembly phase is a process for development all the items discuss in the design
phases. The development will be follow as the storyboard designed or timeline of the
interface designed. During this phase, the actual course materials are created. At this
phases, all the information collected in the concept and storyboard design phases will
be follow to ensure the successful of the development AR mobile application. In this
phase, all the 3D modelling musical instrument and developing the storyboard will be
done here. There are several hardware and software will be used during the
development process. The main software will be used is unity 3D for development the
augmented reality view using C# programming language in Visual Studio. The
programming language C# used to give command for augmented reality to work when
scan. For 3D modelling musical instrument will be used Autodesk MAYA to create
3D model of tamborin, kastanet, kerincing, marakas, kompang and rekorder.
Table 3.2 Software Requirement
PROCESS
DEVELOPMENT
DESCRIPTION FIGURE
Create marker
The musical instrument flashcard
that is used as a marker is created
using Adobe Photoshop CC 2017.
Adobe Photoshop
3D Modelling
To create the 3D model of the 6
types musical instruments which is
Autodesk MAYA
44
tamborin,kastanet, kerincing,
makaraks, kompang
and rekorder.
Create marker in
Vuforia SDK.
The Vuforia use to recognize and
track marker and 3D objects so the
3D model will be appear on AR
view when scan flashcard.
Vuforia SDK
Building 3D
environment in
Unity 3D.
The Unity 3D use to develop the
augmented reality app and overall
interface project.
Unity 3D
Testing
Testing phases is done after the assembly phases completed. Testing process is done
by running the application to see if there are any errors or inconsistencies with the
concept phase. Testing are conducted for every interface and button functions. There
will be 2 test conduct during the testing phases.
a) Alpha Testing:
During Alpha Testing, the developers test runs the project by herself to ensure that the
project can be used and running accordingly.
45
b) Betha Testing:
On Betha Testing, the developer will give user to test the project and give feedback.
The target audience will leave feedback to be taken to improve to detect error and
improve the project. In this part of testing, several kids and primary school students
will be interview to test this application and receive their feedback through
questionnaire paper.
Distribution
This phases is the final phases in the Multimedia Development LifeCycle
methodology. At this phases, the implementation and evaluation of the final results of
the multimedia application is done. This phases is also called the evaluation phases for
application development. At the end of this project, the final product Augmented
Reality application material then will print on paper in form of Flashcard. Then, it is
ready to use as material for learning musical instruments for large audience. But
before that, all of the interface, storyboard, flashcard and prototype in this project will
be evaluate in detail. In order to measure the effectiveness and achieved goals of
mobile application, final testing in Alpha and Betha testing will be done throughout
the application to identify error, problem and receive feedback in order to improve the
project.
46
Conclusion
In this chapter, the methodology use in this project development discuss the
application development, hardware and software specification that are required for
developing the project. Every phases in the development will get along with the
phases in methodology mentioned in this chapter. This chapter chose Multimedia
Development LifeCycle (MDLC) as a guideline to ensure the complete final product
of the AR Flashcard: Let’s Learn Musical Instrument in Augmented Reality mobile
application.
47
REFERENCES
Zulhaida Masmuzidin, Masyarah, Syahela Hussein, Nur Amira Abd Rahman, Alia
Aqqil Hasman, Mohamad. (2020). Learning the Malay Traditional Musical Instruments
by using Augmented Reality Application.
Phunsa, Suwichai.(2014). Applying Augmented Reality Technology to Promote
Traditional Thai Folk Musical Instruments on Postcards. Proceedings of the
International Conference on Computer Graphics, Multimedia and Image
Processing, Kuala Lumpur, Malaysia, 2014
Farinazzo Martins, Valéria Gomez, Letícia Dionísio Corrêa, Ana Grasielle. (2015).
Teaching Children Musical Perception with MUSIC-AR. EAI Endorsed Transactions
on e-Learning
Lemos, Bruno Corrêa, Ana G. D. Nascimento, Marilena Lopes, Roseli D.( 2017).
Augmented Reality Musical App to Support Children's Musical Education. Computer
Science and Information Technology.
Chow, Jonathan Feng, Haoyang Amor, Robert Wünsche, Burkhard C.( 2013). Music
education using augmented reality with a head mounted display. Conferences in
Research and Practice in Information Technology Series
Pradibta, Hendra.( 2018). Augmented reality: Daily prayers for preschooler student.
International Journal of Interactive Mobile Technologies.