Irene Martelli - PhD presentation

download Irene Martelli - PhD presentation

of 31

  • date post

  • Category


  • view

  • download


Embed Size (px)

Transcript of Irene Martelli - PhD presentation

  • 1.Multidimensional Item Response Theory Models with General and Specic Latent Traits for Ordinal Data Irene Martelli Tutor: Prof. Stefania Mignani Bologna, 15 Maggio 2014

2. Outline Aim and structure of the thesis Item Response Theory: a brief introduction Unidimensional Graded Response Model Multidimensional Graded Response Models Model implementation using OpenBUGS Simulation study and results Application to real data Conclusions 3. Aim and structure of the thesis Aim of the thesis: to propose a Markov chain Monte Carlo estimation of the multidimensional additive item response theory model for ordinal data, where the latent traits are allowed to correlate. Structure of the thesis: Ch.1 - An introduction to Item Response Theory (IRT) Ch.2 - Multidimensional IRT (MIRT) models: a review Ch.3 - Bayesian estimation of MIRT models Ch.4 - MIRT graded response models with complex structures Ch.5 - Simulation study Ch.6 - Application to real data Ch.7 - Conclusions 4. Item Response Theory: Aims and Fields of Application IRT has the nal aim to measure abilities and attitudes of individuals through the responses on a number of test items. By using IRT methods we wish to determine the position of the individual along some latent dimension. This latent trait represents the unobservable characteristic of the individuals. The latent trait is often called ability, because of the intense use of IRT in educational and psychological elds. New applications in medicine. New interesting application: extension of IRT models to Social Sciences (well-being, satisfaction, ...) Where the available information is typically collected by questionnaires with multiple ordinal items (i.e. Likert scales) 5. IRT: the concept of model and estimation methods Prob(response|latentvariable(s))Itemwith categoricalresponses Exploratory ConfirmatoryApproach: Multidimensionalmodels Unidimensionalmodels dimensions: oflatent Number Three Two One itemcharacteristics: describingthe Numberofparameters Polytomousresponses Binaryresponses ofthedata: Structure LogitProbit responseandtheexamineesability: Probabilitymodelusedtolinkthe (MCMCtechniques) approach Bayesian methods Likelihood Maximum Methods Estimation 6. Dierent models: literature review MCMC estimation of IRT Models 7. Dierent models: literature review MCMC estimation of IRT Models 8. Samejima unidimensional model for graded responses (1) The unidimensional graded response model (Samejima, 1969) is used for data collected on n subjects who have responded to each of p ordinal items. The response Yij of the i-th individual to the j-th item can take values in the set {1, . . . , Kj}. With ijk we denote the probability that the i-th subject will select the k-th category on item j. In order to construct the probability ijk we rst need to consider the cumulative probabilities Pijk that are expressed as a function of the predictor ijk, which depends on i and jk: ijk = f(i, jk) We focus on: i. Normal ogive models: Pijk = (ijk) 1 (Pijk) = ijk ii. jk = (j, jk), where j and jk are the discrimination and threshold parameters, respectively. 9. Samejima unidimensional model for graded responses (2) Each item has Kj 1 thresholds j1, . . . , j,Kj 1 that satisfy the order constraint j1 < < j,Kj 1. Pijk = P(Yij k|i, j, jk) = (jk ji) = jkji (z)dz The probability ijk that the i-th subject will select the k-th category on item j can thus be written as: ij1 = Pij1 ijk = Pijk Pi,j,k1 for k = 2, . . . , Kj 1 ijKj = 1 Pi,j,Kj1 10. Towards multidimensional models Advantages of unidimensional models: i. Fairly simple mathematical forms; ii. Numerous examples of applications; iii. Evidence of robustness to violations of assumptions. However, the actual interactions between persons and test items are not simple as implied by unidimensional models: i. Examinees are likely to bring more than a single ability to bear when responding to a particular test item; ii. The problems posed by test items are likely to require numerous skills and abilities to determine a correct solution. There is a need for more complex IRT models that more accurately reect the complexity of the interactions between examinees and test items Multidimensional IRT models. 11. MIRT Models: underlying latent structures AdditivestructureHierarchicalstructuresMultiunidimensionalstructure Itemj Let consider: n individuals; a test consisting of p ordinal items, divided into m subtests; the existence of m latent abilities i = (1i, . . . , mi) . We focus on the following models: Multiunidimensional Graded Response Model, where items in each subtest characterize a single ability. Additive Graded Response Model, where each item measures a general and a specic ability directly. 12. The multiunidimensional GRM Pvijk = P(Yij k|vi, vj, jk) = (jk vjvi) = jkvjvi 1 2 et2/2 dt vj and jk are item parameters representing the item discrimination and the threshold between categories k and k + 1; vi represents the i-th examinee ability in the v-th ability dimension; Probability vijk that the i-th examinee will select the k-th category on item j in subtest v: vij1 = Pvij1 vijk = Pvijk Pv,i,j,k1 for k = 2, . . . , Kj 1 vijKj = 1 Pv,i,j,Kj 1 13. The additive GRM Pvijk = P(Yij k|0i, vi, 0vj, vj, jk) = (jk 0vj0i vjvi) = jk0vj0ivjvi 1 2 et2/2 dt For each item j of the subtest v: 0vj, vj and jk; 0i represents the i-th overall ability and vi represent the specic abilities (with v = 1, . . . , m); The probability vijk that the i-th examinee will select the k-th category on item j in subtest v is obtained as in the multiunidimensional GRM; Both general and specic abilities are involved in determining the response probability by following a compensatory approach. 14. Identication issues In general, Bayesian item response models can be identied (Fox, 2010): By imposing restrictions on the hyperparameters. Via a scale transformation in estimation procedure. Then, for identication purposes, a multivariate normal prior distribution with a xed correlation structure is assumed for abilities: i = (0i, 1i, . . . , mi) , i Nm+1(0, ), i = 1, . . . , n with covariance matrix with diagonal elements being 1 and o-diagonals elements being the ability correlations. 15. Additive GRM implementation using OpenBUGS (1) Categorical distribution is assumed for responses: Yij| dcat(vij1, . . . , vijKj ) for j = 1, . . . , p i = 1, . . . , n P(Yij = k|) = [k=1] vij1 [k=2] vij2 . . . [k=Kj] vijKj Model specication according to additive GRM: Pvijk = (jk 0vj0i vjvi) v = 1, . . . , m PvijKj = 1 j = 1, . . . , p k = 1, . . . , Kj 1 vij1 = Pvij1 vijk = Pvijk Pv,i,j,k1 for k = 2, . . . , Kj 16. Additive GRM implementation using OpenBUGS (2) Normal prior distributions are assumed for item parameters: vj N(0, 1)(vj > 0) v = 1, . . . , m 0vj N(0, 1)(0vj > 0) j = 1, . . . , p jk N(0, 1) k = 1, . . . , Kj 1 {jk, . . . j,Kj1} = ranked{ j1, . . . , j,Kj1}1 Multivariate normal prior distribution is assumed for abilities: i = (0i, 1i, . . . , mi) , i Nm+1(0, ), i = 1, . . . , n with covariance matrix with diagonal elements being 1 and o-diagonals elements being the ability correlations. 1 In order to satisfy the order constraint on thresholds. 17. Simulation study (1) The aim is to evaluate the parameter recovery of the multiunidimensional and the additive graded response models under several conditions. We consider the bidimensional case m = 2. Ability correlations structures: 0.3 0.2 1 0.4 1 0.2 1 0.40.3 =B0 0 1 0 1 0 1 0 0 =A Additive:0.4 1 1 0.4 =B0 1 1 0 =A Multiunidimensional: General simulation conditions: 30,000 iterations (15,000 burn-in) to ensure stationarity; Two chains (to compute the R Gelman and Rubin diagnostic statistic for convergence); Sample sizes: n = 500 and n = 1000; 10 replications for each simulation. 18. Simulation study (2) Simulation conditions: Additivemodel(Blocks1and2)Multiunidimensionalmodel(Block1) For each of the 16th scenarios: median RMSE and median absolute bias for each set of item parameters, estimated ability correlations. Main results: Item parameters and ability correlations are well reproduced. Higher biases are noticed when sample sizes are smaller (as expected). Stimulating observation: relation between number of test items (p) and number of categories (K). 19. Simulation study results: Additive model (1) 20. Simulation study results: Additive model (2) 21. Simulation study results: estimated ability correlations 22. Perception of tourism impact on the quality of life (1) Ravenna Forl-Cesena Rimini Rep.di SanMarino Aim: to investigate Romagna and San Marino residents perception and attitudes toward the tourism industry. Perceptionofbenefits Perceptionofcosts Residents Tourism Industry Data analyzed: subset of results of a research on the subjective well-being, conducted by the University of Bologna (in the end of 2010). Criticitylevels Thresholds(item)parameters towardstourism Generalattitudetourismbenefits Perceptionof tourismcosts Perceptionof thetourismindustry todistinctaspectof onasetofitemsreferred Respondentsopinions Observedvariables 23. Perception of tourism impact on the quality of life (2) Sample size: n = 794. Answers on a 7-point scale, from strongly disagree to strongly agree. We focus (among the others) on items of the questionnaire concerning residents evaluations about costs and benets of the tourism industry: Itemsonbenefits Subtest1 Personal information (age, gender, nationality, residence and occupation) were also collected. 24. Tourism impact: results for the multiunidimensional GRM Public services (B3), job opportunities (B4) and cultural activities (B5) are the most informative items on the perception of the tourism advantages (high discrimination parameters). Trac (C4) and pollution (C5) are the most informative items on the perception of the tourism costs (high discrimination parameters). The main and immediate advantages of tourism are identied in the economic