UNIVERSITI PUTRA MALAYSIA

EFFECT OF INTELLIGENCE TUTORING SYSTEM ON STUDENTS'

ACHIEVEMENT IN ALGEBRAIC EXPRESSION

TSAI CHEN CHIEN

FPP 2007 2

EFFECT OF INTELLIGENCE TUTORING SYSTEM ON STUDENTS' ACHIEVEMENT IN ALGEBRAIC EXPRESSION

By

TSAI CHEN CHIEN

Thesis Submitted to the School of Graduate Studies, Universiti Putra Malaysia, In Fulfilment of the Requirements for the Degree of Master of Science

June 2007

Abstract of thesis presented to the Senate of Universiti Putra Malaysia in fulfilment of the requirement for the degree of Master of Science

EFFECT OF INTELLIGENCE TUTORING SYSTEM ON STUDENTS' ACHIEVEMENT IN ALGEBRAIC EXPRESSION

By

TSAI CHEN CHIEN

June 2007

Chairman: Associate Professor Aida Suraya Md. Yunus , PhD

Faculty: Faculty of Educational Studies

In this quasI experiment study, an Intelligent Tutoring System (ITS), was

compared to the Computer Assisted Instruction (CAl) in tutoring students to learn

the topic on Algebraic Expression. The ITS system consisted of two main

structures in the architecture, the Student Model and Tutorial Model. The Student

Model stores student's responses throughout the tutorial process while the Tutorial

Model presents a suitable tutoring strategy to students according to their progress.

In order to identify the effect of ITS on student' s achievement, 62 students in one

of the private school in Perak were involved in the study. The Subjects consist of

two intact groups. The groups were randomly assigned as the control group and

the experimental group. One of the group studied algebraic expression in a CAl

11

learning environment, while the other were in a CAl and ITS (CAI+ITS)

environment.

Before the experiment began, subjects were gIven a pre-test on algebraic

expression, together with questionnaires of learning style and attitude towards

mathematics. Post-test was given by the end of the study. The study was

administered in eight slots with one hour per slot. Both groups of subjects studied

the algebraic expression in CAl environment at first stage of the study. After

completing the first stage, subjects from CAl group went through a tutoring

session with CAl, whereas subjects from the CAI+ ITS environment continue their

study in ITS tutorial.

ANCOVA test was used to analyse the main hypothesis of the study. A Pre-test

was set as a covariate and the gain score between the pre-test and post-test of study

was measured. The result of the study showed that there was a significant

difference in the students achievement in algebraic expression between students

who learn with CAI+ ITS and who learn with CAl only as delivery systems.

However, there were no significant differences in student's achievement in

algebraic expression among students with different learning styles or students with

different attitude towards mathematics.

111

The findings of the study indicated that CAI+ ITS was more effective in helping

students learn algebraic expression as compared to using CAl alone. However,

when a separate analysis was conducted for different delivery systems, neither

CAI+ITS environment nor CAl only environment showed significant effects of

any learning style or any category of attitude towards mathematics on student's

achievement. This study suggests that educators and software developers should

develop more ITS based learning tools or integrate ITS elements in courseware.

IV

Abstrak tesis yang dikemukakan kepada Senat Universiti Putra Malaysia sebagai memenuhi keperluan untuk ijazah master Sains

KESAN PENGGUNAN SISTEM TUTORAN CERDAS KE AT AS PENCAP AlAN PELAJAR DALAM UNGKAP AN ALGEBRA

Oleh

TSAI CHEN CHIEN

Jun 2007

Pengerusi: Profesor Madya Aida Suraya Md. Yunus, PhD

Fakulti: Pengajian Pendidikan

Dalam penyelidikan jenis eksperimen Quasi ini, penggunaan Sistem Tutoran

Cerdas (ITS) telah dibandingkan dengan Pembelajaran Berbantukan Komputer

yang tradisional (CAl) dalam membantu pelajar mempelajari topik ungkapan

algebra. Sistem Tutoran Cerdas ini mengandungi dua struktur asas dalam reka

bentuknya, iaitu Model Pelajar dan Model Tutoran. Model Pelajar menyimpan

maklum balas pelaj ar dalam proses tutoran manakala Model Tutotoran akan

memaklumkan strategi tutoran yang sesuai kepada pelajar berdasarkan

perkembangan pelaj ar berkaitan.

Bagi mengenalpasti kesan ITS keatas pencapaian pelaj ar, 62 orang pelajar dari

salah sebuah sekolah swasta di Perak telah terlibat dalam kajian ini. Subjek adalah

v

dalam dua kumpulan lengkap. Subjek dalam kaj ian ini telah dibahagikan kepada

kumpulan kawalan and kumpulan eksperimen secara rawak. Salah satu daripada

kumpulan berkaitan belajar ungkapan algebra dalam persekitaran CAl, manakala

satu lagi kumpulan diberikan persekitaran CAl dan ITS(CAI+ITS).

Sebelum eksperimen itu dijalankan, pelajar diberi ujian pra berkaitan dengan

ungkapan algebra, bersama dengan soal selidik tentang stail pembelajaran dan

sikap terhadap matematik. Uj ian post diberikan pada akhir sesi ekspeirmen.

Penyelidikan ini dijalankan sebanyak lapan sesi dengan j angka masa satu jam per

sesI. Pada tahap pertaman, kedua-dua kumpulan subjek belajar mengenai

ungkapan algebra melalui CAL Selepas melengkapi tahap pertama, subjek dalam

kumpulan CAl meneruskan pembelajarannya melalui tutoran CAL Manakala

subjek daripada persekitaran CAI+ITS akan melengkapi sessi tutoran dengan

tutoran ITS.

Ujian ANCOV A telah digunakan untuk menganalisis hasil dapatan penyelidikan.

Ujian Pra ditetapkan sebagai asas perbandingan skor pencapaian pelajar dan

perbezaan skor antara skor ujian pra dan ujian post diukur. Dapatan kaj ian

menunjukkan terdapat perbezaan yang signifikan pada pencapaian pelajar dalam

ungkapan algebra antara subjek yang belajar dengan CAI+ITS dan subjek yang

belajar dalam CAl sahaja. Walau bagaimanapun, tidak terdapat perbezaan yang

VI

signifikan dalam pencapaian pelajar berdasarkan stail pembelajaran atau sikap

terhadap matematik antara pelajar dalam kajian ini .

Dapatan kaj ian menunjukkan kombinasi CAl dan ITS adalah lebih berkesan dalam

membantu pelajar mempelajari ungkapan algebra berbanding dengan penggunaan

CAl sahaj a. Walau bagaimanapun, tidak terdapat sebarang kesan yang signifikan

oleh mana-mana stail pembelajaran pelajar atau sikap pelajar terhadap matematik

yang berlainan keatas pencapaian pelajar apabila menggunakan sistem

penyampaian yang berlainan. Kajian ini mencadangkan agar para pendidik dan

pembangun perisian menghasilkan lebih banyak bahan pembelajaran berdasarkan

ITS atau mengintegrasikan elemen ITS dalam penghasilan koswer.

Vll

ACKNOWLEDGEMENTS

This study would not have been completed successfully without the guidance and

help of others. Thus I would like to express my most sincere appreciation and

gratitude to the following individuals who have assisted me at various stages of

this research.

First and foremost I wish to extend my heartfelt thanks to Associate Professor Dr.

Aida Suraya Md. Yunus who is both my academic advisor and Chairperson of the

Supervisory Committee, for her constant advice and care. Dr. Aida has been the

main source of guidance throughout my research, culminating in the final write up

of the thesis. My thanks to her again, who has undertaken the task of guiding and

correcting my writings.

I would also like to express my deep sense of gratitude to Associate Professor Dr.

Wan Zah Wan Ali, who has supervised my thesis from the very beginning to the

end. Her encouragement, guidance and comments have given this research depth

and insight. I have learnt much from her.

Special thanks and appreciation are extended to another member of my

supervisory committee, Professor Dr. Ab Rahim Bakar. His contributions to this

Vlll

study are invaluable especially in the area of statistical analysis . I have benefited

greatly from his vast experience, expertise and knowledge.

I want to thank the Principal of Shen Jai High School, Ipoh, Perak for allowing me

to conduct this study in the School. I would also like to thank the staff of the

School for their cooperation which has helped make this study a success.

Special thanks to my father, Mr Tsai Chang Shen, my mother, Chua Lee Choo,

and my dear friend, Saw Hooi Chin, and Daniel Wee Tiong How who have given

me love, support, patience, encouragement and for always being there for me.

IX

I certify that an Examination Committee has met on 2 1 st June 2007 to conduct the final examination of Tsai Chen Chien on her Master of Science thesis entitled "Effect of Intelligent Tutoring System on Students' Achievement in Algebraic Expressions" in accordance with Universiti Pertanian Malaysia (Higher Degree) Act 1 980 and Universiti Pertanian Malaysia (Higher Degree) Regulations 1 98 1. The Committee recommends that the student be awarded the degree of Master of Science.

Members of the Examination Committee were as follows:

Muhammad Hasan A. Rahman, PhD Associate Professor Faculty of Educational Studies Universiti Putra Malaysia (Chairman)

Rohani Ahmad Tarmizi, PhD Associate Professor Faculty of Educational Studies Universiti Putra Malaysia (Internal Examiner)

Mokthar Nawawi, PhD Lecturer Faculty of Educational Studies Universiti Putra Malaysia (Internal Examiner)

Sharifah Noru) Akmar Syed Zamri, PhD Associate Professor Faculty of Education Universiti Malaya (External Examiner)

,.. .... ...,_�. HAZALI, PhD Professor e uty Dean School of Gradua e Studies Universiti Putra Malaysia

Date: 24 October 2007

x

This thesis submitted to Senate of Universiti Putra Malaysia and has been accepted as fulfilment of the requirement for the degree of Master of Science. The members of the Supervisory Committee were as follows:

Aida Suraya Md Yunus, PhD Associate Professor Faculty of Educational Studies Universiti Putra Malaysia (Chairman)

Wan Zah Wan Ali, PhD Associate Professor Faculty of Educational Studies Universiti Putra Malaysia (Member)

Ab. Rahim Bakar, PhD Professor Faculty of Educational Studies Universiti Putra Malaysia (Member)

Xl

AINI IDERIS, PhD Professor and Dean School of Graduate Studies Universiti Putra Malaysia

Date: 1 5th November 2007

DECLARA TION

I hereby declare that the thesis is based on my original work except for quotations and citations, which have been duly acknowledged. I also declare that it has not been previously of concurrently submitted for any other degree at UPM or other institutions.

Xll

\�cw���

TSAI CHEN CHIEN

Date: 1 9th September 2007

TABLE OF CONTENTS

ABSTRACT ABSTRAK ACKNOWLEDGEMENTS APPROVAL DECLARATION LIST OF TABLES LIST OF FIGURES

CHAPTER

1

2

INTRODUCTION 1 . 1 . Background of the study 1 .2 . Statement of the problem 1 .3 . Objectives of the study 1 .4. Research question 1 .5 . Hypothesis 1 .6. Significance of the study 1 .7. Operational Definition

1 .7. 1 . Computer assisted instruction (CAl) 1 .7.2. Intelligent Tutoring System (ITS) 1 .7 .3 . Achievement in algebraic expression 1 .7.4. Attitude towards mathematics 1 .7 .5 . Learning style in mathematics

1 .8 . Limitation of the study

LITERATURE REVIEW 2 . 1 Computer Assisted Instruction (CAl)

2.2

2.3 2.4 2 .5 2 .6

2.7

2 . 1 . 1 Effectiveness of CAl in Mathematics Learning

Artificial Intelligence In Education (AIED) 2 .2 . 1 Model of ITS 2.2.2 Impact of ITS on Students Achievement Application of Learning theory in CAl and ITS Learning difficulties in Mathematics Learning Attitude towards Mathematics Learning Learning style 2 .6 . 1 Learning Style and Mathematics 2 .6.2 Relationship between Students learning

style and Learning by Computer Theoretical Framework of the study

Xlll

Page 11 V V111 X xu xvi XIX

1 5 8 9 9 1 1 1 3 1 3 1 4 1 4 1 5 1 6 1 7

1 9

27 34 35 44 49 54 56 58 66

68 7 1

2 .8 Conceptual Framework of the study 73 2.9 Conclusion 75

3 METHODOLOGY 77 3 . 1 Research design 78

3 . 1 . 1 Independent, Dependent and Moderating variable 80

3 . 1 .2 Internal and extemal validity of the Quasi -Experimental Design 82

3 .2 Population, Sampling and Location of the Study 89 3 .3 Instrument 92

3 . 3 . 1 Mathematics test 92 3 .3 .2 Attitude towards mathematic

questionnaire 95 3 .3 .3 Learning style questionnaire 98

3 .4 Delivery system 99 3 . 5 Procedure of Study 1 02

3 .5 . 1 Pilot study 1 05 3 .5 .2 Data collection procedure 1 07

3 .6 Data analysis 1 1 0

4 RESULTS AND FINDINGS 1 1 5 4. 1 Demographic background 1 16 4.2 Mathematics Achievement in Algebraic

Expression Test 1 1 9 4.3 Learning Style of Respondents 1 20 4.4 Students Attitudes Towards Mathematics 1 2 1 4.5 Test of Hypothesis 1 3 0

4.5.1 Differences on Students Achievement in Algebraic Expression between Students who use CAI+ITS and Students who use CAl only as Delivery System. 1 3 2

4.5 .2 Difference on students achievement in algebraic expression for students with different learning styles in both delivery systems, CAI+ ITS and CAl only 1 34

4.5.3 Difference on students achievement in algebraic expression between students with different learning styles in CAl group 137

4.5.4 Difference on students achievement in algebraic expression between students with different learning style in CAI+ITS group. 1 3 9

XIV

5

4.5 .5 Difference on students achievement in algebraic expression for students with different attitude towards mathematic in both delivery systems, CAI+ITS and CAl only.

4 .5 .6 Difference on students achievement in algebraic expression between students with different attitudes towards mathematic in CAl only group

4 .5 .7 Difference on students achievement in algebraic expression between students with different attitudes towards mathematic in CAI+ITS only group

4.5.8 Summary of result 4.6 Conclusion

SUMMARY,DISCUSSION, CONCLUSION IMPLICATIONS, AND RECOMMENDATIONS 5 . 1 Summary 5 .2 Discussion 5 .3 Conclusion 5.4 Implication of the study 5 .5 Recommendation 5 .6 Recommendation for future study

BIBLIOGRAPHY APPENDICES BIODAT A OF THE AUTHOR

xv

1 4 1

1 44

1 46 1 48 1 49

1 5 1 1 55 1 62 1 64 1 68 1 69

1 72 1 86 236

LIST OF TABLES

Table Page

1 Reliability Analysis of the Instrument for Pilot Study 1 07

2 Summary of Test Used in the Study 1 1 2

3 Frequency and Percentage of Respondents Gender 1 1 6

4 Frequency and Percentages of Respondents by Age 1 1 7

5 Frequency and Percentages of Respondents by Computer Ownership 1 1 7

6 Frequency and Percentages of Respondents by English Grade in UPSR 1 1 8

7 Frequency and Percentage of Respondents by Mathematics Grade in UPS 1 1 8

8 Pre-test and Post-test for CAI+ITS and CAl only approach 1 20

9 Respondent's Learning Style 1 20

1 0 Distribution of Learning Style and Delivery System 1 2 1

1 1 Frequency and Percentage of Respondents in 'How I feel about mathematics' Questionnaire 1 23

1 2 Frequency and Percentage of Respondents in 'Experience With Mathematics' Questionnaire 1 25

13 Respondent's Attitudes Towards Mathematics 1 27

XVl

1 4 Itemised Analysis for Attitudes Towards Mathematics Questionnaire 1 28

1 5 Respondents Category of Attitudes Towards Mathematics 130

1 6 Tests of Normality 1 3 1

1 7 Homogeneity of Regression Test for Gain score in Different Delivery Systems 132

1 8 ANCOVA of Respondent's Score Gain in Learning Algebraic Expression for Different Delivery Systems 134

1 9 ANCOV A of Gain Score in Learning Algebraic Expression for Respondents with different Learning Style in both Delivery Systems 136

20 Gain Score for Respondents with Different Learning Style in Both Delivery Systems 137

2 1 ANCOVA of Gain Score in Learning Algebraic Expression for Respondents with different Learning Style in CAl only Group 138

22 Post Hoc Comparison Test for Gain Scores between Different Learning Style group in CAl only Delivery System 139

23 ANCOV A of Gain Score in Learning Algebraic Expression for Respondents with different Learning Style in CAI+ITS Group 140

24 Post Hoc Comparison Test for Gain Scores between Different Learning Style group in CAI+ITS Delivery System 1 4 1

25 ANCOV A of Gain Score in Learning Algebraic Expression for Respondents with different Attitude Towards Mathematics in both Delivery Systems 143

xvii

26 Gain Scores for Different Category of Attitudes Towards Mathematics 1 44

27 ANCOV A of Gain Score in Learning Algebraic Expression for Respondents with different Attitude Towards Mathematic in CAl only group 1 45

28 ANCOVA of Gain Score in Learning Algebraic Expression for Respondents with different Attitude Towards Mathematics in CAI+ITS group 1 47

29 Summary of Hypothesis testing for the Study 1 48

XVlll

LIST OF FIGURES

Figure Page

1 The architecture of an ITS system 36

2 The architecture of Ms Lindquist 39

3 S-R Learning Model from Skinner 50

4 An i llustration relating major literature and key area from the literature which contributed towards students achievement in mathematics 73

5 Conceptual framework of study 74

6 Experimental design in this study 79

7 Procedure of the study 1 04

8 Data collection procedure 1 08

XIX

CHAPTER 1

INTRODUCTION

This chapter provides an overview of the background of the study, the statement

of the problem, objective, research questions, hypotheses, significance of the

study, operational definitions, and limitation of the study.

1.1 Background of the Study

Students often find themselves in a situation where progress is hindered because

they lack fundamental algebraic skills. Many educators (Bowen, 1 987; Heffernan,

200 1 ) have expressed concern about student's lack of basic algebraic skills. They

noticed students often make mistakes while executing algebraic operations. In

addition, the importance of algebra has been addressed by the Chick, Stacey,

Vincent and Vincent (200 1) . However, it i s hard for educators to trace student's

mistakes in algebraic. The mathematics teacher, therefore, should experiment with

different methodologies in order to assure that he or she is providing the students

with appropriate alternatives for learning (Fey, 1 982).

As the world enters the new millennium, Information Communication Technology

(lCT) is widely used in field of business, education, management and other field.

Educators realized that computer is in great demand these days. To provide a

more effective learning environment, Computer Assisted Instruction (CAl) had

been developed. In recent years, CAl has expanded its influence into all subject

areas (Piccoli, Gabriele, Ahmad, Rami, Ives & Blake, 200 1 ). Numerous studies

conducted by Alexender ( 1 999), Jain and Getis (2003) and Leung (2003) to

compare the effectiveness of CAl with the traditional method. These researches

revealed statistical differences on student achievement and attitude between a CAl

group and traditional group.

While CAl may be somewhat effective in helping learners, they do not provide the

same kind of personalised attention that a student would receive from a human

tutor (Ester, 1 994). In order to achieve a more efficient learning environment, and

to deliver the best learning process, research on ICT in mathematics education is

focusing in the area of Artificial Intelligence (AI). Ester ( 1 994) believes that AI is

able to prepare a more human based interaction learning environment for students.

Human based interaction learning environment is important in learning because it

involves students in active learning.

2

Heffernan (200 1 ) stated that as the techniques of AI becomes widely known and

appreciated in the field of educational computing, AI with interests in education

has also undergone changes in direction. He also stressed that the overall aim of

developing AI is to enable the computer to be effective and act as a knowledgeable

agent in the teaching and learning process. A major stand of research has been the

design of the so-called Intelligent Tutoring Systems (ITS) which require

knowledge representations to provide models of the subject domain, the leaner

capabilities and the tutorial pedagogy (Heffernan, 200 1 ).

According to Cumming and Abbott ( 1 988), ITS such as PROLOG by Robert

Kowalski have been under development for at least nearly twenty years ago.

Other expert systems, like SNOOPY (Schauer & Staringer, 1 988) have already

been used in western schools since twenty years ago. SNOOPY is a simple system

to demonstrate some of the methods and techniques used in Artificial Intelligence.

It is essentially a dialogue program with simple natural language understanding

capabilities.

The objective of an ITS is to provide a teaching process that adapts to the students'

needs by exploring and understanding the student' s special needs and interests

(Kaplan & Rock, 1 995). Research in the field of ITS has always had a strong

focus on the development of comprehensive student models, based on the

3

assumption that within a problem solving context, leamer's thinking processes can

be modelled, traced, and corrected using computers (Julika, 1 999).

According to McArthur, Lewis, and Bishay ( 1 994), ITS attempts to capture a

method of teaching and learning exemplified by a one-to-one human tutoring

interaction. One-to-one tutoring allows learning to be highly individualized and

consistently yields better outcomes than other methods of teaching. Unlike

previous CAl systems, ITS represent some of the knowledge and reasoning of

good one-to-one human tutors, and consequently can coach in a much more

detailed way than CAl systems.

Thus, in order to find out which of the learning environment, the ITS environment,

or the CAl environment, is more suitable to help our students in their mastery

learning of algebraic expression, this study attempts to investigate and compared

the effectiveness of a CAl plus ITS (CAI+ITS) based approach and a CAl only

approach in the learning of mathematics among secondary school students.

4

1.2 Statement of The Problem

According to Kiamanesh ( 1 997), classroom instruction, attitude towards

mathematics and learning styles are factors that affect mathematics achievement.

In relation to this, previous studies have indicated that students always face

difficulty in learning algebraic expression in mathematics (Booth, 1 984, 1 988;

Greeno, 1 982; Kieran, 1 988, 1 992; Lins, 1 990). According to Chick et al . (200 1 ),

students who have difficulties in solving mathematics problem such as solving

algebraic related mathematics problem, usually have problem keeping up with

classroom instruction. In an effort to help students overcome these learning

problems, some studies have discovered the potential of self-paced CAl as a

delivery system.

Previous studies such as Bangertdrowns, Kulik, & Kulik ( 1 985), Hughes ( 1 974),

Leung (2003), Teh, George, and Fraser ( 1 995), Owens and Waxman ( 1 994),

Yalcinalp, Geban and Ozkan et. al . ( 1 995) found that CAl have led to

improvement in students achievement. Students in the CAl group were able to

perform significantly better than students in the control group. However, recent

studies such as Glickman and Dickson (2002), have indicated that Intelligent

Tutoring System (ITS) as a delivery system have a greater potential than CAl to

help students overcome difficulties in solving mathematics problems.

5