the method of everything

7/29/2019 the method of everything

1/12

Cypriot Journal of Educational

Sciences

4 (2009) 5869

www.world-education-center.org/index.php/cjes

Application of Ordinal Logistic Regression and Artifical Neural

Networks in a Study of Student Satistaction

Meral Yaya*, Eylem Deniz Akncb

a Assistant Professor at Mimar Sinan Fine Arts University, Department of Statististics, Turkey

b Assistant Professor at Mimar Sinan Fine Arts University, Department of Statististics, Turkey

Received December 20, 2008; revised May 11, 2009; accepted May 30, 2009

Abstract

Measuring student satisfaction is an important issue especially for university administration, in order to improvestudent services and opportunities. The major objective of this study is to provide a solution for this issue.Consequently, student satisfaction has been measured with an ordered five-point Likert scale. A student satisfactionquestionnaire was applied to a total of 314 university students, consisting of 208 female and 106 male students, andsatisfaction was measured by asking students to respond to 19 questionnaire items. Ordinal regression and artificalneural network methods were applied to the collected data which emphasized the differences between the two

methods in terms of the correct classification percentages.

Keywords: artifical neural networks; ordinal logistic regression; student satisfaction

2009 Academic World Education & Research Center. All rights reserved.

1. INTRODUCTION

The measurement of student satisfaction can be useful in universities, based on the notion thatstudents have needs and rights when it comes to participating in quality programmes and should

receive a satisfactory service. A true and thorough understanding of the complex learningexperience requires a knowledge of both student satisfaction and the factors which may detractfrom this satisfaction. The results from the questionnaire survey used in this research have been

*Corresponding Author. Meral Yay. Tel: +90 0 212 236 69 36 177, fax: +90 0 212 261 11 21

E-mail address: [email protected] (M.Yay), [email protected] (Eylem Deniz Aknc)
mailto:myay:@msgsu.edu.trmailto:myay:@msgsu.edu.trmailto:edeniz:@msu.edu.trmailto:edeniz:@msu.edu.trmailto:myay:@msgsu.edu.tr


2/12

Meral Yay & Eylem Deniz AkncI / Cypriot Journal of Educational Sciences 4 (2009) 58-69

59

used as feedback to help university administrators to enhance the quality of the universitysprogrammes and services.

Different statistical methods can be used to analyze satisfaction data. These methods includeregression models, multilevel modelling, decision trees and artifical neural networks. Regresssionmodels such as linear, logistic and ordinal regression are useful for analyzing the relationship

between multiple explanatory variables and student satisfaction results (Thomas & Galambos,2002). These methods also permit researchers to estimate the magnitude of the effect of theexplanatory variables on the outcome variable.

In this study, ordinal regression and artificial neural networks are used to analyze the studentsatisfaction data. Ordinal regression is a very useful and powerful method for satisfactionclassification. Compared to the other regression methods used in the literature, the ordinalregression method is the most suitable and practical technique for analyzing the effects ofmultiple explanatory variables on the ordinal outcome that cannot be assumed to be a continuousmeasure with normal distribution. Artificial neural networks are algorithms that can be used toperform nonlinear statistical modeling and to provide a new alternative to ordinal logistic

regression methods. Neural networks have a number of advantages, including requiring lessformal statistical training than other methods, the ability to implicitly detect complex relationshipsbetween dependent variables and explanatory variables, the ability to detect all possibleinteractions between predictor variables, and the availability of multiple training algorithms.

2. ORDINAL LOGISTIC REGRESSION

The ordinal logistic method is a generalization of the linear regression method. The ordinalregression method is used to model the relationship between response (outcome) variables and aset of explanatory variables, which can be either categorical or numerical (Sentas, Angelis,

Stamelos & Bleris, 2004).

In ordinal regression (OR) analysis, the major link functions, e.g. logit, complementary-loglog (continuation ratio or proportional hazard), negative log-log, probit and Cauchit are used tobuild specific models. There is currently no universal method to help the researcher choose whichlink function best fits a given dataset - only basic heuristics. Generally, the logit link is consideredsuitable for analyzing ordered categorical data evenly distributed among all categories; the clogloglink is often used to analyze ordered categorical data when higher categories are more probable(Cheng, 2007). In a negative log-log link function, lower categories are more probable and ourdata set is approprate for this function. Five different link functions can be explained briefly as in

Table 1.

The unique feature of the cumulative logit model is that the odds ratio for each predictor istaken to be constant across all possible collapsings of the outcome variable. When a testableassumption is met, the odds ratios in a cumulative logit model are interpreted as the odds of beinglower or higher on the outcome variable, across the entire range of the outcome. The wideapplicability and intuitive interpretation of the cumulative logit model are two reasons for its

being considered the most popular model for ordinal logistic regression.


3/12

Meral Yay & Eylem Deniz Aknc / Cypriot Journal of Educational Sciences 4 (2009) 58-69

60

The hallmark of the cumulative logit or POM is that the odds ratio for a predictor can beinterpreted as a summary of the odds ratios obtained from separate binary logistic regressionsusing all possible cut points of the ordinal outcome. Whereas a binary logistic regression models asingle logit, the cumulative logit models several cumulative logits (Hosmer & Lemeshow, 1989).Therefore, if the ordinal outcome has five levels (1, 2, 3, 4, and 5), four logits will be modeled, onefor each of the following cut points: 1 vs. 2, 3, 4, 5 ; 1, 2 vs. 3, 4, 5; 1, 2, 3 vs. 4, 5 and 1, 2, 3, 4 vs 5.

(Gameroff, 2005)

Let ( ) ( )1 i k ix ,...., xp p denote response probabilities at values for a set of explanatory

variables. First form cumulative probabilities are as follows:

( ) ( ) ( ) ( )k i i 1 i k iF x p Y k / x x ..... x , k 1, 2,...., K 1p p= = + + = - (1)

Cumulative logits are then formed as follows:

( )( )

( )k i

k k i

k i

F xL log it F x log , k 1, 2,....,K 1

1 F x

= = = - -

(2)

Letting ( ) ( )k i k iL x log it F x= , where ( )iF x is the cumulative probability up to, and

including, category k, the Proportional Odds Model (McCullagh, 1980) can be expressed as follows:

( )k i k k iL x x , k 1, 2,....., K 1a b= + = - (3)

The 1 K 1,.....,a a - parameters are nondecreasing in k and are known as the intercepts or

cutpoints. The parameter vector b contains the regression coefficients for the covariate vector

ix . Inherent in this model is the proportional odds assumption, which states that the cumulative

odds ratio for any two values of the covariates is constant across response categories. Itsinterpretation is that the odds of being in a category less than or equal to k is ( )1 2exp x xb - times higher at 1x x= than at 2x x= . The model constrains the K-1 response curves to have the

same shape, and therefore we cannot fit it by fitting separate logit models for each cutpoint. Wemust maximize the multinomial likelihood, subject to constraints. The model assumes that effectsof variables are the same for each cutpoint, k = 1,..., K-1.

In ordinal regression (OR) analysis, the major link functions, e.g. logit, complementary-loglog (continuation ratio or proportional hazard), negative log-log, probit and Cauchit are used tobuild specific models. There is currently no universal method to help the researcher choose whichlink function best fits a given dataset - only basic heuristics. Generally, the logit link is considered

suitable for analyzing ordered categorical data evenly distributed among all categories; the clogloglink is often used to analyze ordered categorical data when higher categories are more probable(Cheng, 2007). In a negative log-log link function, lower categories are more probable and ourdata set is approprate for this function. Five different link functions can be explained briefly as in

Table 1.


4/12


61

Table 1. Different Link Functions and Forms in Ordinal Regression

Function Form

Logit ( )

( )k i

k i

F xlog

1 F x

-

Complementary log-log ( )( )( )k ilog log 1 F x- -Negative log-log ( )( )( )k ilog log F x- -Probit ( )( )1 k iF x

-F

Cauchit ( )( )( )k itan F x 0.5p -

In this study, the lower categories are more probable. That is why, the negative log-log functionis used in the application of ordinal regression. In the negative log-log function, the form of thelink is defined as follows:

( ) ( ){ } ( ) ( ){ }k j k j j jlog it F x log log 1 F x log log P Y y / X / P Y y / X = - - = - - = > (4)

3. NEURAL NETWORKS

Classification is one of the most active research and application areas with regard to neuralnetworks. The literature is vast and growing. Neural networks have emerged as an important toolfor classification purposes. The recent research activities in neural classification have establishedthat neural networks are a promising alternative to various conventional classification methods.

The advantage of neural networks lies in the following theoretical aspects. Firstly, neural networksare data-driven self-adaptive methods in that they can adjust themselves to the data without anyexplicit specification of functional or distributional form for the underlying model. Secondly, theyare universal functional approximators in that neural networks can approximate any function witharbitrary accuracy. Since any classification procedure seeks a functional relationship between thegroup membership and the attributes of the object, accurate identification of this underlyingfunction is doubtlessly important. Thirdly, neural networks are nonlinear models, which makesthem flexible in modeling real world complex relationships. Finally, neural networks are able toestimate the posterior probabilities, which provide the basis for establishing classification rulesand for performing statistical analysis (Zhang, 2000).

Artificial neural networks (ANN) are relatively crude electronic networks of "neurons" based on

the neural structure of the brain. They process records one at a time, and "learn" by comparingtheir classification of the record (which, at the outset, is largely arbitrary) with the known actualclassification of the record. The errors from the initial classification of the first record is fed backinto the network, and used to modify the networks algorithm the second time around, and so onfor many iterations (Silva, 2006).

Roughly speaking, a neuron in an artificial neural network is;1. A set of input values (xi) and associated weights (wi),


5/12


62

2. A function (g) that sums the weights and maps the results to an output (y).

Figure1. Output Map

Figure 2. Hidden Layers

The input layer is composed, not of full neurons, but rather consists simply of the values in adata record, that constitute inputs to the next layer of neurons. The next layer is called a hiddenlayer; there may be several hidden layers. The final layer is the output layer, where there is onenode for each class. A single sweep forward through the network results in the assignment of avalue to each output node, and the record is assigned to whichever class's node had the highestvalue.

To use a neural network for classification, we need to construct an equivalent function

approximation problem by assigning a target value for each class. For a two-class problem we canuse a network with a single output, and binary target values: 1 for one class, and 0 for the other.The training of the network is commonly performed using the popular mean square error. For

multiclass classification problems ( )1 of K, where K 2- - > we use a network with K outputs, one

corresponding to each class, and target values of 1 for the correct class, and 0 otherwise. Sincethese targets are not independent of each other, however, it is no longer appropriate to use thesame error measure. The correct generalization is through a special activation function (thesoftmax) designed so as to satisfy the normalization constraint on the total probability. However,this approach does not retain the ordinality or rank order of the classes, and is not, thereforeappropriate for ordinal multistate classification problems ( Costa & Cardoso, 2005).


6/12


63

4. CASE STUDY

In this study, student satisfaction questionnaires were distributed to a total 314 senioruniversity students consisting of 208 females and 106 males. Satisfaction was measured by asking

the students to respond to 19 items. Student satisfaction is a response variable and it is measuredon a five-point Likert scale (very dissatisfied, dissatisfied, dont know, satisfied, very satisfied).Gender is coded by 1 female and 2 male. 19 questionnaire items were measured using a five-pointLikert scale (very dissatisfied = 1, dissatisfied = 2, dont know = 3, satisfied = 4, very satisfied = 5).

OR and ANN which are frequently used for analyzing satisfaction were applied to our data. TheOR model is anayzed using SPSS 15 and ANN is analyzed using Statistica.

Cronbach Alpha is calculated for 19 questionnaire items. According to Nunnely (1998), theCronbach Alpha of a scale should be greater than 0.70 for items to be used together as a scale.Thehigh internal consistency for the survey might be demostrated based on the Cronbach Alpha

reliability 0.894 (for all items combined).

The student respondents indicated that they were very dissatisfied (43.6%) and very satisfied(2.2%) with the overall university experience. The majority of respondents seemed to be verydissatisfied with the universitys programmes and services, regardless of gender. This is indicatedin Table 1.

Table 2. Different Link Functions and Forms in Ordinal Regression

frequency percent valid percent cumulative percent

very dissatisfied 137 43.6 43.6 43.6dissatisfied 102 32.5 32.5 76.1

dont know 42 13.4 13.4 89.5satisfied 26 8.3 8.3 97.8

very satisfied 7 2.2 2.2 100total 314 100.0 100.0

For OR and ANN analysis, 80% of the data were used as the training set, and the remaining 20%for the test set in order to confirm model validation.

The OR model is applied for a negative log-log model because the lower categories are moreprobable. The negative log-log model containing satisfaction items revealed a number of

interesting findings. The Pearsons chi-square ( 2 971.681c = and p=1.00) for the model with the

negative log-log link, indicates that the observed data are consistent with the estimated values inthe fitted model. The results of parameter estimates are indicated as in Table 2. According to this

table, student satisfaction is significantly associated with the four explanatory variables(department, facilities, activities and library). These four significant explanatory variables exhibitpositive regression coefficients.


7/12


64

Table 3. Explanatory Variables Associated with Student Satisfaction Based on Complete Model for NegativeLog-log Link

estimate significance

satisfied=1 3.401 0.000

satisfied=2 4.839 0.000satisfied=3 5.852 0.000satisfied=4 7.633 0.000

gender -0.060 0.743department 0.403 0.000

course -0.113 0.181facilities 0.208 0.021activities 0.290 0.004

union 0.002 0.981scholarship 0.056 0.561exchange 0.054 0.564

administration -0.033 0.785

election 0.064 0.496website 0.072 0.448computerization 0.006 0.946

language 0.015 0.861consultancy -0.004 0.965

clean -0.132 0.148library 0.284 0.002food -0.145 0.091cafe 0.114 0.184

sports 0.057 0.575toilet 0.163 0.058

The test of parallel lines is designed to make a judgement concerning the adequacy of themodel. The null hypothesis states that the corresponding regression coefficient is equal across alllevels of the response variable. The alternative hypothesis states that the correspondingregression coefficients are different across all levels of response variables. According to the test ofparallel lines results, there was no significant difference for the corresponding regressioncoefficients across the response categories, suggesting that the models assumption of parallel

lines is not violated in the complete model with the negative log-log link ( 2c =658.422 and

p=0.000). The model with the negative log-log link provide evidence that it satisfies the parallellines assumption.

The data is divided into two parts - train (250) and test (64) in order to select the model which isthe best classification among the logit link, the clog log link and the ANN model. The cross-tabulating method is used to categorize the classified and actual responses into a 5 by 5classification table at the negative log-log link and the ANN model. Table 4 displays the accuracy ofthe classification results for the satisfaction response categories of the train data at the logit linkmodel. The percentage of classification is 62.4% for all categories.


8/12


65

Table 4. Actual Satisfaction Category * Predicted Satisfaction Category Crosstabulationfor the Negative Log-log Link Model (train)

predicted response category

totalverydissatisfied

dissatisfied dontknow

satisfied verysatisfied

actualresponsecategory

verydissatisfied

9384.5%

1715.5

0.0%

0.0%

0.0%

110100%

dissatisfied 2832.2%

5664.4%

22.3%

11.1%

0.0%

87100%

dont know 618.2%

2163.6%

412.1%

26.1%

0.0%

33100%

satisfied 210.5%

1157.9%

421.1%

210.5%

0.0%

19100%

very satisfied 0.0%

0.0%

0.0%

0.0%

1100%

1100%

total 12951.6%

10542.0%

104.0%

52.0%

14%

250100%

Table 5 displays the accuracy of the classification results for the satisfaction response categories ofthe test data using the negative log-log link model. The percentage of classification is 65.6% for allthe categories.

Table 5. Actual Satisfaction Category * Predicted Satisfaction Category Crosstabulationfor the Negative Log-log Link Model (test)



dissatisfieddontknow

satisfiedvery

satisfied


verydissatisfied

2385.2%

414.8

0.0%

0.0%

0.0%

27100%

dissatisfied 426.7%

1173.3%

0.0%

0.0%

0.0%

15100%

dont know 0.0%

777.8%

111.1%

111.1%

0.0%

9100%

satisfied 0.0%

228.6%

228.6%

342.9%

0.0%

7100%

very satisfied 0.0%

0.0%

0.0%

233.3%

466.7%

6100%

total 2742.2%

2437.5%

34.7%

69.4%

46.3%

64100%

Table 6 displays the accuracy of the classification results for the satisfaction response categoriesof the train data using the ANN model. The percentage of classification is 70.4% for all thecategories.


9/12


66

Table 6. Actual Response Category* Predicted Response Category Crosstabulation for the ANNModel (train)




satisfiedvery

satisfied


verydissatisfied

10696.4%

32.7%

1.9%

0.0%

0.0%

110100%

dissatisfied 3542.7%

3947.6%

67.3%

22.4%

0.0%

82100%

dont know 1029.4%

720.6%

1544.1%

25.9%

0.0%

34100%

satisfied 522.7%

14.5%

29.1%

1463.6%

0.0%

22100%

very satisfied 0.0%

0.0%

0.0%

0.0%

2100%

2100%

total 15662.4%

5020.0%

249.6%

187.2%

2.8%

250100%

Table 7 displays the accuracy of the classification results for the satisfaction response categoriesof the test data using the ANN model. The percentage of classification is 65.6% for all thecategories.

Table 7. Actual Response Category* Predicted Response Category Crosstabulation for ANNModel (test)




satisfiedvery

satisfied


verydissatisfied

2488.9%

27.4%

13.7%

0.0%

0.0%

27100%

dissatisfied 840.0%

840.0%

15.0%

315.0%

0.0%

20100%

dont know 337.5%

0.0%

337.5%

225.0%

0.0%

8100%

satisfied 0.0%

0.0%

0.0%

4100%

0.0%

4100%

very satisfied 0.0%

0.0%

120.0%

120.0%

360.0%

5100%

total 3554.7%

1015.6%

69.4%

1015.6%

34.7%

64100%

The research findings indicate that the ANN model has better satisfaction classification than theordinal regression model with negative log-log link model for train and test data.


10/12


67

5. CONCLUSION

In this study, student satisfaction is measured with an ordered scale and analyzed using OR andANN in order to emphasize the difference of the two methods in terms of satisfactionclassification. The principle of classification with a negative log-log link function in ordinalregression and ANN is adopted to build the candidate models and to search for the best model.The ANN and the OR models are used for classifications in various fields. In this study, thesemethods are applied to student satisfaction data. As a consequence, the ANN model provides abetter classification for ordinal data. In other words, it has more correct percentage ofclassification than the ordinal regression method.

REFERENCES

Cheng, J. (2007). A Neural Network Approach to Ordinal Regression. Electrical Engineering andComputer Science, 1, 1028-1036.

Costa, J. P., & Cardoso, J. S. (2005). Classification of Ordinal Data Using Neural Networks (pp.690-697) Springer Berlin.

Fujimoto, K. (2003). Application of Multinomial and Ordinal Regressions to the Data of JapaneseFemale Labor Market, Unpublished Masters Thesis, 18, University of Pittsburgh.

Gameroff, M. J. (2005). Using the Proportional Odds Model for Health-Related Outcomes: Why,When, and How with Various SAS Procedures, New York State Psychiatric Institute, 205-30.

Hosmer, D.W. & Lemeshow, S. (1989).Applied Logistic Regression, John Wiley & Sons, NewYork.

MacCullagh, P. (1980). Regression Models for Ordinal Data.Journal of the Royal Statistical Society,42, 109-142.

Sentas, P., Angelis, L., Stamelos, I., & Bleris, G. (2005). Software Productivity and Effort Predictionwith Ordinal Regression, Information and Software Technology, 17-29.

Silva, D.J. (2006). Comparison of Artificial Neural Network and Regression Models in SoftwareEffort Estimation, Laboratory for Computing and Applied Mathematics, 47, 960-977.

Thomas, E.H., & Galambos, N. (2002). What Satisfies Students? Mining Student-Opinion Data withRegression and Decision-Tree Analysis, Stony Brook University, NewYork.

Zhang, G. P. (2000). Neural Networks for Classification: A Survey , IEEE Transactions on Systems,

30(4), 451-465.


11/12


68

APPENDIX

MIMAR SINAN FINE ARTS UNIVERSITY

STUDENT SATISFACTION EVALUATION FORM

Gender : Female Male

The measurement of student satisfaction (response variable):

Very dissatisfiedDissatisfiedDont knowSatisfiedVery Satisfied

Questionnaire Items1 2 3 4 5

1- My department met my expectations.

2- The number of elective courses is adequate.

3- Classroom, workplace, equipment and techniqual facilities are adequate.

4- There is enough scientific, social and cultural activities at the university.

5- The student union works efficiently.

6- The scholarship opportunities are adequate.

7- It is easy to benefit from the student exchange programmes.

8- The university administration believes that it is important to delegatedecision making to the students.

9- Im satisfied with the way in which student delegates are elected.

10- Im satisfied with my universitys website.

11- Im satisfied with my universitys computer facilities.

12- The foreign language education is adequate.

13- The course consultancy system works well.

14- The environment of university is clean.

15- The library oppurtinities are adequate.

16- The quality of food is good.17- The variety of food in the cafe is adequate.

18- The sports opportunities are adequate.

19- The toilets are clean.


12/12


69

Turkish Abstract

Ordinal Lojistik Regresyon ve Yapay Sinir Alarnn renci Memnuniyeti zerinde Uygulanmas

z: renci memnuniyetini lme, niversite ynetiminin renci olanaklarn ve frsatlarn

artrmalar iin nemli bir sorundur. almann temel amac, bu soruna bir zm salamaktr. Buama dorultusunda renci memnuniyeti beli Likert leinde lmtr. renci memnuniyetianket sorular 208i kz 106s erkek olmak zere toplam 314 renciye sorulmu ve memnuniyetrencilere yneltilen 19 soru ile llmtr. Toplanan verilere ordinal regresyon ve yapay sinira yntemleri uygulanm ve ardndan ile renci memnuniyetini doru snflandrma oran

asndan yntemler arasndaki fark vurgulanmtr.

Anahtar Kelimeler: Yapay sinir a, ordinal lojistik regresyon, renci memnuniyeti.

the method of everything

Documents

Transcript of the method of everything