2011 3rd International Congress on Engineering Education (ICEED)

Identification of Student Achievement and Academic

Profile in The Linear Algebra Course: An Analysis

U sing The Rasch Model

Zulkifli Mohd Nopiah\ Nur Arzilah Ismail, Haliza Othman, Izamarlina Asshaari, Noorhelyna Razali, Mohd Haniff Osman and Mohd Helmi Jamalluddin

Unit Pengajian Asas Kejuruteraan, Fakulti Kejuruteraan dan Alam Bina Universiti Kebangsaan Malaysia

43600 UKM Bangi, Selangor, Malaysia 'zmn1993@gmail.com

Abstract- There has been great concern placed on the declining

level of mathematical comprehension among engineering

students. Earlier identification and classification of the students

proves to be vital for future actions. Students profiling is

important for any institution especially when the educator seeks

to identify the factors of strengths and weaknessess of their

students towards any subject knowledge. This paper presents an

analysis of 215 students' profiling and Linear Algebra results for

semester 1 Academic Session of 2010/2011 from the Faculty of

Engineering and Built Environment (FKAB), Universiti

Kebangsaan Malaysia (UKM) using the Rasch modeling

technique. Students profiling which consists of gender,

department, programme, pre-university qualifications and

results are used as variables in this study. Students' final results

of the Linear Algebra course are also considered as another

variable. The major finding was highlighted to be the variables of

pre-university qualification and sex, where students with

Malaysia Higher School Certificate (STPM) holder and male

students have been found to be performing well in this course, as

comparison is being drawn to other qualifications and gender.

Keywords-Students profiling; Linear Algebra; Rasch-model Approached; Bloom's Taxonomy

I. INTRODUCTION

There has been great concern in the Malaysian educational system lately, on the performance of university students from different backgrounds on pre-university programmes. Currently, student entries at the Malaysian universities are based on their academic results of Matriculation, Malaysian High School Certificate (STPM), Diploma and Fundamental Study In Science (ASASI Sains). A previous study by [1] indicates that STPM students perform better in Linear Algebra as compared to other students with different pre-university backgrounds. This study has drawn our attention to the fact hat the main reason lies in STPM students being taught in more detail compared to the Matriculation-based students since the length of study for STPM is longer than that of the matriculation. On the other hand, [2-4] claim that students' performance in the university cannot be reliably predicted

This work is sponsored by Ministry of Higher Education of Malaysia through the fund of PTS-20 11-00 I.

from their pre-university performance. To rephrase, pre­university performance does not necessarily guarantee good performance at the university. A study by [5] shows that the significant difference exists between the grade levels of students in elementary schools and mathematics. On the academic performance based on gender, a study by [6] reveals that gender has no effect on students' mathematics achievement. However, in contrast to another study by [7], female students are evidently found to be more successful academically than their male counterparts.

There also seems to be the findings regarding student performance, gender and pre-university performance, which are based on different students in different countries. This study attempts to analyse the correlation between student achievement in engineering mathematics course and students' profile at Universiti Kebangsaan Malaysia. Student achievement is measured from the results of the Linear Algebra course they obtain, and students' profile is examined with respect to gender, department, programme, pre-university qualifications and exam results.

For the purpose, the data was analysed using the Rasch model. The Person and Items Distribution Map (PlDM) can give a precise overview of the student's achievement on a linear scale of measurement [8]. Several studies have reported that the Rasch model proves to be more meaningful [9] in a way that it provides better exploratory depth in understanding the ability of any groups of the person studied [10].

IT. METHODOLOGY

The final Linear Algebra results and students profiling of 215 students in semester 1 academic session 2010/211 from the Faculty of Engineering and Built Environment (FKAB), Universiti Kebangsaan Malaysia (UKM) were considered in this study. Prior to using the Rasch model, all the data were converted into a rating scale for more convenient analysis.

A. Students Profiling

978-1-4577-1259-3/11/$26.00 ©2011 IEEE 197

Data of students' profile consisting of sex, department, programme, pre-university qualification and results are classified to variable codes. The methods of coding used on these data are shown in Table 1.

TABLE I. CODING VARIABLE FOR STUDENTS PROFILING

Out of 215 students, there are 132 male students and 83 female students. The highest number of respondents which are 135 students are of the Malay race, followed by Chinese, 72 students and Indian, 8 students. In the FKAB, Matriculationbased students are 131 students, which constitute the highest number of entry into the engineering degree, followed by 59 STPM students, 19 Diploma students, and others 6 students. Another variable is the pre-university results, which are based on the cumulative grade point average (CGPA) of their preuniversity qualifications which were divided into three groups; CGPA 0-2.99 as group 1 (10 students), CGPA 3.00-3.49 as group 2 (63 students) and CGPA 3.50-4.00 as group 3 (142 students). Four major departments have been covered in this study, namely; the Civil and Structural Department (JKAS) (46 students), Mechanical and Material Department (JKMB) (66 students), Chemical and Process Department (JKKP) (47 students) and Electric, Electronic and System Department (JKEES) (56 students).

B. Linear Algebra The final questions of Linear Algebra consist of three

parts, which are Part A, Part B and Part C. Students were required to answer all questions in Part A and B, while Part C is an optional question. There are 19 questions including the sub-questions considered in this study. Since each question has different total marks, the standardization method was used. The formula for standardization is given in Eq. (1):

Xii -mrn.xj Zto = (1 )

where i is i'h students (i = 1,00. ,215) , j is /h questions (;= 1,00. ,19) , zij is the standardized marks for ith student and/h

question, ZlJ is marks th is student and/h question while min X; and max X J represents the minimum and maximum marks for jlh respectively. Responses from the student's exam results were analysed using a rating scale in which the students were rated according to their achievement. From Eq. (1),

z"XlO· A (2)

Then, A is classified to correspond to the rating scale in Table 2.

Table 2 shows the fmal marks corresponding to the rating scale. The highest rating (scale 5) is the highest marks that the students obtained.

Tn the Rasch Model, the probability of success can be estimated for the maximum likelihood of an event as;

(3)

where e refers to the base of natural algorithm or Euler's number of 2.7183, A represents student's ability while OJ IS an item or task difficulty.

m. RESULTS AND DISCUSSION

The analysis was run in Winstep; a Rasch analysis software to obtain the logit values. The Rasch analysis is divided into two parts, where the first part is called the "Person Measure" and the second part is the "Item Measure". The Person Summary (Fig. 1) reveals a fair Reliability of Cronbach Alpha = 0.74. The major fmding is the Person Mean, f1person = 0.1810git where the students were found to be performing well above the expected standard in answering the final questions. The results identified two groups of student separation (G=1.70) with only 69.77% (N=150) of the students found to be "good" students, and 30.23(N=65) "poor" students.

The Item Summary (Fig. 2) well summarises the very high Reliability of Cronbach Alpha = 0.97 and items separation (G=5.57). The value of items separation indicates that there are six groups classifiable from the questions, but for easy profiling of the items, five groups were considered as "Very Difficult", "Difficult", "Moderate", "Easy" and "Very Easy". The group's separation for the Person Summary and Item Summary is also provided in Fig. 3 .

Fig. 3 shows the Person-Item Distribution Map (PlDM) where the person; e.g students' profiling and the item; e.g final marks are plotted on the same logit scale. Based on the previous table of the summarised statistics, the maximum and minimum values give an indication of the person and item spread. The Linear Algebra questions were constructed based on Bloom's Taxanomy and for this particular course only two domains were used; application (AP) and comprehensive (CM).

Generally, about a quarter of the students found that half of the topics studied had been difficult to grasp; with C3-AP and C5-AP being most difficult to solve. It is also noted that, there is a huge gap between these questions denoted by ( ... � ) indicating the extent of difficulty that the students encountered in attempting the question. By looking at Fig. 3, 15 students had been considerably exceptional in this cohort (denoted by the circle), where 80% (12/15) are Male students, 20 % (2/15)

198

SUMMARY OF 2 1.5 MEASURED Per- s' ons

RAW MODEl I FIT O UTFIT 1 1 SCORE COUNT MEASIIJRE ERROR M SQ ZSTD M SQ ZSTD 1 1-----------------------------------------------------------------------------1 1 MEA 51.1 15.1 .18 .21 1.01 .0 1.01 .0 1 1 S.D. 11.1 .4 .44 .04 .31 .9 .45 .9 1 1 MAX. 7 5.0 16 . .0 1.44 .45 1.92 2.2 2.82 2.4 1 1 loll • 2.0 • .0 12 . .0 -1.21 .18 .32 -3.1 .23 -2.3 1

1 REAL RMSE .2 0! AD]. SO .38 I SEPARATION 1.70 " Person RELIABILITY .74 . 1----------------------------- ------------------------� 1 MOO E L RM SE • 21. AD] . SO . 39 SE PARA i ION! 1. I'E P@!il'Sdli kE LIABI LI i y • jj 1 S.E. OF Person � EA = . .03 1

.68

FigJ!llll!! 1. Slimmary .of 2] 5 P�rson Measllred.

SUMMARY OF 19 MEASIIJRED Items

1 RA� MODEL I FIT OUTFIT 1 1 SCORE COUNT M EASIIJRE ERRDR ,., SQ Z STD r" SQ Z STD 1 1-----------------------------------------------------------------------------1 1 MEAN 578.4 170.5, .00 .07 1.07 .1 1.05 .1 1 1 S.D. 262.5 65.2 .53 .04 .27 2.6 .27 1.7 1 1 MAX. 968.0 215,.0 1.47 .20 1.58 3.3 1.88 2.7 1 1 MI . 60.0 46. 0 - • 78 • 05 • 5 4 -7 • 6 • 5 7 -5 • 2 1 1---------------------------- -------------------------1 1 REAL RMSE . .09 ADJ. SO .52 I SEPARATION 5.5 d I U:em RELIABILITY ·97 II I MODEL RMSE . .08 ADJ. SO .52 sEPARATioN 6. 29 Item RELIABILITY .98 1

1 S. E. OF Item MEA = .12 1

UMEA NF • .o0'0' U SCA L E =1 . .o0'O Item RAW SCORE-TO -MEAS IIJR E CORRElATIO = -.53 1 3240' DATA POINTS. lO G-lI KElI HDOD C HI-SQUARE : 7368.48 w i t h 3004 dl. f. p= . .00'00

Figure 2. Summary of 19 Items Measured

""'$� - ,,� - ;l;t;; .... 5i �r;ep;.I"'TIIlr.,

Person M2IlI = 1.44

!"<:",on M�"n = 0.18

2

. 111 �� tilt"'. :I

� I

. "�I(I"� Ii e.lll-AP ".,,� w�... C :I. :I. -AP ·rQ!·II"��_1I' � � j!'=:� MJ �::� t!,U"II�� __ �� I "'-:l-AP

nJJfcJJiiiS F �UIJi ::::::;, !::� :n .. �

I CB-AP

_ . fl. �1 5 <;;�'<" .... C2'-,lil'

.. T CiI!-"''' tI A�.AIP'

Item MB� - 1 , 47'

30�� ( 11.11 5 ) ••• Millo =d. ... u. 21m. (2.(13) .,... F<DW.< 5tuoi"'3 "lith S� (13I B )

S1'PM ho ld.<.""a OGPh l:>e;",.em3.;(l-4.0(l! ;md O.l.3,,>(2115} =lIirulM.lt b.old.<£ ",ill CC'P'A 1I.f:l.nlKl1 1.OO.s.:so. 40� (.6J I .S) fn:m.: III. JX.MI! d\erlf1llOom. 20�� (3 .... 5) fl-o.:m.

B2-"'"

<: 1-"'" C ;;o - AP

on'

Item Me<ln - 0

• � Item Mki - - 1) , 78

� n . (lll) u< 1Ii!I.I. ....=. Ill. "UIl ,.......:======::::;------=�-__t"MIot.a..1orl.,., O ml flOll . , CGP-A bftWoon

Parson Min - -L2 1 3. 00·3-'0 ;m,jJ f,,,,,, tho JKEES a�t

AIoO<b.,. o . ))·� (113) rnLd.."", II. FOlllQIo

b� STPM lwla .... , O G.P;;' brT-\"'" 3.51}­

�.OO .. � hiJlil unJ:nr do . 1KM11 �'I=UJI.<D' -2 .. .... oe:�s:.ol ... -,r"eq'l.l)o;

ElL -jl' I5 2,

Figure 3. Person-Item- Distribution Map (PIDM)

199

GU'IC"EV� SCALO!;RAM O lf RE S;�SES, : iPe. r !!!ll DZl 1 j[ 1>e-m

1 lHH l l.l. 11

1.2 � a B 6 S l9� 2 5 6a 7 7 9B a l Pre- Pre- J�l�11 1 • 110 L S9 ..... .... Result

+�55 lB� � 55�55555 If � 2 .z ibn 2H +555 555�55555 55l If � a a lid\:

'E O .... '.!l 66� a5555555�55 5 � � a .2 ibn �B +�5 5,�55 a 55�55555 � "t a .2 ibn

l56 +55 5555 5 555�555l If "t 2 l l.!!!II'

215 +5 5,25 55�5la�llll l � mI> a a lI:h HB +a5 5�55 a llL2a5 1. 2l � mI> 2 .2 ibn l7� . +55 a�5a l 55aaHH If mI> 2 .2 ibn lB6 +55 5.25a � 5aa.zHH If mI> 2 .z ibn 2a6 +55 lla5 l 55a�ll5l � di a .2 ibn

'f r.l!!C:. :r:II =- I 7.2 +5l 5aa5 5 5HHHl �. mJt. a 2 lI:h

laa +a5 l525 5 l5l5aHl �. ,. 11; a � k· t:: la� +55 l5�� � .2lalall5 � mI> a l k". l57 +a5 ��55 5 ll2la2ll � ",.,. 2 l k".

l 6 +5 55a 2�5lll�.2l5l l � mJt. a 2 kk

lB +5ll .2.211lll2l .2al If di a � k ,. l.22 +5ll l25llllll .2l.l � di 2 � k1> l a B +5l lH5 l llL2HHl �. ,,11; a � k � 1 99 +a 2H �Hll. U HH l If mJt. 2 � 1.it; l�9 +l� lll2 l llllllll � mJt. .2 .2 ibn

1

1 lHH l l.l. l.l

1.2 � a B 65, l 9'�.2 5 6a 7 7 9B a l

Figure 4. Scalogram for Person and Items measured

are Female students with 87% (13/15) being STPM holders and had had COPA between 3.50-4.00 and 0.13% (2/15) are matriculation graduates, with the COPA between 3.00-3.50. The JKMB department seems to be the one with the highest percentage under this cohort, which is 40% (6/15) and the other departments JKAS, JKKP and JKEES have the same percentage, which is 20% (3/15) respectively. These groups of students have been found to perform well in Linear Algebra. Another group, which is stated to fall at the bottom part in the column person in Fig. 3 shows that 67% (2/3) are Male students with Matriculation certificates, having COP A between 3.00-3.50 and are currently under the JKEES department. Another 0.33% (1/3) students are Females being STPM holders, with COPA ranging between 3.50-4.00 and are also the JKMB department students. Based on this output, these groups of students cannot perform well in Linear Algebra, even when easy questions had been provided. The detailed information about these students is shown in Fig. 4 (15 top students and 3 bottom students).

The scalogram for both the Person and Item measures is next captured in Fig. 4. In the column labelled Person, the top number is considered as good students and the bottom number

viewed as poor students. This output also tells us which number of question is considered as difficult or easy, as hown in the column labelled Items. Based on Fig. 4, the top students (good students) come from the JKMB department, females with STPM certificates and surprisingly with the COPA ranging between 3.00-3.49. Meanwhile, the lowest-ranked students (poor students) also come from the n(MB department with the same level of COP A but Males with Matriculation certificates. Obviously, the 15 top students are STPMcertificate holders.

Fig. 5 displays the category probability curve. Based on this curve, there are only two groups of students categorized, either the student knows or do not know how to answer the questions, based on the values of' l' and '5'. This verifies the output of the summary statistics in the first part of analysis. Fig. 6 displays the number of good and poor students corresponding to their gender. A total of 150 of good students, where 85 students are male and 65 students are female have been discovered. Contrastingly, for the groups of poor students, 47 students are male and 18 students are female (out of 65 students). These values show that the males have exceeded the number of females for the two groups.

200

The overall summary for the scalogram is shown in Table 3. The capital 'G' refers to the percentage of 'Good students' and 'P' as the percentage of 'Poor students'. Based on Table 3, male students constitute the higher number of percentage with 40% and 22% for the two groups of students compared to female counterparts. Tn the column labelled the pre­university qualification, students with Matriculation certificates have the highest number of percentage (40% for the 'good' groups), followed by STPM (24% for the 'good' groups) compared to others qualifications. Among the good groups, students from the JKMB department perform well with the highest percentage 20% and 11 %, followed by the JKKP (I 8% for the good groups), the JKAS and JKEES (16% for the good groups) respectively. Tn contrast, the 'poor' group shows that the highest percentage also comes from the JKMB department with 11 %, followed by those from the JKEES (11%), JKAS (10%) and JKKP (4%) departments.

TV. CONCLUSION

Based on the analysis, students being STPM holders have demonstrated that they are more able to answer difficult questions in Linear Algebra compared to others even though students with matriculation certificates show the highest percentage in this study. This result is in tandem with the study by Haliza Othman (2009), where STPM students have better performance towards Linear Algebra compared to Matriculation students. This result shows that students with other pre-university qualifications need to be given due attention, especially students with matriculation certificates because the 'poor' group is represented by the highest number of students in this category. This identification serves as an input to the educators to improve the course content and perhaps, modity the teaching style for the particular topics, especially when it involves the difficult areas which will subsequently be asked in the examination. It also recommends that, the Basics of Linear Algebra should be introduced first to them as preparation to Linear Algebra. Another rather interesting finding shows that male students have been found to perform better than their female course mates.

It is recommended that in the future, a predefined set of tests is to be given to the students at the early stage of entry. The pre-test would identity groups of students with different ability levels and performance, and thus, enable the educator to tackle each group individually for better performance in mathematics. Some other factors should also be investigated

C ATEGORY PRO BABILITIES : MODES - Struct u re measures at i ntersect i ons p -+--------------+-------------- --------------+--------------+-R 1.0 + o 1111111 5555 B I 111111 555555 A I 111 5555 B .8 + 111 555 I I 11 55 L I 1 55 I I 11 5 T .6 T 1 5S Y I 1 5 o F

.5 + 11 5 I 1 55

.4 + 1 5 I 1 5 I 55 11

.2 + 33*33 33*3 I 333355 4******44

! R E S P o N S E

I 2*** .... 2 ...... 2 ........ 44 11 33 ...... 44444444 I 12222222 ................ 3 ........ 4444 2222222 .... 111 33333333"44444441 . 0 +.................................... 222 .................................... +

-+--------------+-------------- --------------+--------------+--2 -1 0 1 2

Person [MINUS] Item MEASLRE

90 80 70 60 50 40 ]0 20 1 0

Figure 5. Category probability curve

o �------------------------------� good'tU<ients PoorstudCflts

Group

Figure 6. Number of good and poor students which corresponds to gender

TABLE III. SUMMARY TABLE FOR STUDENTS PROFILING BETWEEN TWO GROUPS OF STUDENTS

REFERENCES

in order to find out the reasons behind the students' lack of [I] understanding in this course.

O. Haliza, et aI., "A comparative study of engineering students on their pre-university results with their first year performance at FKAB, UKM, " in Seminar Pendidikan Kejuruteraan dan Alam Bina (PeKA 09), 2009, pp. 289-300.

[2]

201

K. L. Todd, "An historical study of the correlation between G.C.E advanced level grades and the subsequent academic performance of well qualified students in a university engineering department, " Math. TODAY, vol. 37, pp. 152-156,2001.

[3] 1. Adamson and H. Clifford, "An appraisal of A-level and university examination results for engineering undergraduates, " Tnt. J of Mech. Eng. Educ., vol. 30, pp. 265-279, 2000.

[4] S. Alexander, et at., "Case studies in admissions to and early performance in computer science degrees, " ACM STGCSE Bulletin, vol. 35, pp. 137-147, 2003.203

[5] D. Kogce., C. Yildez., M. Aydin., and R. Altindag., "Examining elementary school students' attitudes towards mathematics in terms of some variables, " in Procedia Soc. Behav. Sci., 2009, pp. 291-295.

[6] T. Saritas and o. Akdemir, "Identifying Factors Affecting the Mathematics Achievement of Students for Better Instructional Design, " International Journal of Instructional Technology and Distance Learning, vol. 6, 2009.

[7] M. J. Zalizan and M. D. Hazadiah, "Gender and educational performance: The Malaysian perspective, " in Procedia Soc. Behav. Sci., 2010, pp. 720-727.

[8] A. A. Arzilah, M. Azlinah, A. Noor Habibah, Z. Sohaimi, and M. Saifudin, "Appraisal of course learning outcomes using Rasch measurement: A case study in information technology education, " Tnt. J Syst. Appt. Eng. Dev., pp. 164-171, 2007.

[9] A. A. Azrilah, et aI., "Development of rasch-based descriptive scale in profiling information professionals' competency, " in IEEE, 2008. [10] M. Saifudin, et aI., "Easier learning outcomes analysis using Rash model

in engineering education research., " Latest Trends Eng. Educ., pp. 442-447, 2011.

202