DIFFERENCES IN READING STRATEGIES AND DIFFERENTIAL ... excelled, which were selected response items,...

Click here to load reader

  • date post

    02-Aug-2020
  • Category

    Documents

  • view

    0
  • download

    0

Embed Size (px)

Transcript of DIFFERENCES IN READING STRATEGIES AND DIFFERENTIAL ... excelled, which were selected response items,...

  • DIFFERENCES IN READING STRATEGIES AND

    DIFFERENTIAL ITEM FUNCTIONING ON PCAP

    2007 READING ASSESSMENT

    by

    Tanya Scerbina

    A thesis submitted in conformity with the requirements

    for the degree of Master of Arts

    Department of Human Development and Applied

    Psychology

    Ontario Institute for Studies in Education

    University of Toronto

    © Copyright by Tanya Scerbina 2012

  • ii

    DIFFERENCES IN READING STRATEGIES AND

    DIFFERENTIAL ITEM FUNCTIONING ON PCAP

    2007 READING ASSESSMENT

    Master of Arts 2012

    Tanya Scerbina

    Department of Human Development and Applied

    Psychology

    University of Toronto

    Abstract

    Pan-Canadian Assessment Program (PCAP) 2007 reading ability item data and contextual data

    on reading strategies were analyzed to investigate the relationship between self-reported reading

    strategies and item difficulty. Students who reported using higher- or lower-order strategies

    were identified through a factor analysis. The purpose of this study was to investigate whether

    students with the same underlying reading ability but who reported using different reading

    strategies found the items differentially difficult. Differential item functioning (DIF) analyses

    identified the items on which students who tended to use higher-order reading strategies

    excelled, which were selected response items, but students who preferred using lower-order

    strategies found these items more difficult. The opposite pattern was found for constructed

    response items. The results of the study suggest that DIF analyses can be used to investigate

    which reading strategies are related to item difficulty when controlling for students‟ level of

    ability.

  • iii

    ACKNOWLEDGMENTS

    My deepest gratitude goes to my supervisor, Dr. Ruth Childs, for her expertise,

    guidance and support throughout my Master‟s program, as well as for her insight and feedback

    in the process of completing this thesis. I would also like to thank Monique Herbert, the second

    member of my supervisory committee, for being an invaluable mentor to me in my program of

    study.

    The completion of this thesis would not have been possible without the statistical

    expertise of Olesya Falenchuk and professional input of Pierre Brochu; I thank both for their

    patience and ongoing assistance. I would also like to express my gratitude to the employees of

    the Canadian Ministers of Education, Canada, specifically Kathryn O‟Grady and Pierre Brochu

    for providing me with indispensable resources and services.

    Finally, I would like to thank all of the members of Datahost and my classmates;

    especially Amanda, Christie, Jayme and Marija for giving excellent advice. Special thank-you

    goes to my family and friends for their immense support and encouragement.

  • iv

    TABLE OF CONTENTS

    Abstract ..................................................................................................................................... ii

    Acknowledgments ................................................................................................................... iii

    Table of Contents ..................................................................................................................... iv

    List of Tables ........................................................................................................................... vi

    List of Figures ........................................................................................................................ viii

    1 Introduction............................................................................................................................ 1

    1.1 Differential Item Functioning .......................................................................................... 1

    1.2 Reading Process and Strategies ....................................................................................... 1

    1.3 Objectives ........................................................................................................................ 6

    2 Method ................................................................................................................................... 8

    2.1 Data .................................................................................................................................. 8

    2.2 Grouping Variable ........................................................................................................... 9

    2.3 Analyses ......................................................................................................................... 15

    3 Results and Discussion ........................................................................................................ 17

    3.1 Score Distributions ........................................................................................................ 17

    3.2 Classical Item Analysis.................................................................................................. 19

    3.3 Reading Strategies and Test Scores ............................................................................... 32

    3.4 DIF and DSF Analyses .................................................................................................. 33

    3.4.1 Dichotomous Items ............................................................................................... 33

    3.4.2 Polytomous Items and DSF .................................................................................. 36

    3.4.3 DIF with Scaled Matching Score, Dichotomous Items ........................................ 37

    3.4.4 DIF with Scaled Matching Score, Polytomous Items and DSF ........................... 40

  • v

    4 Implications and Conclusion ............................................................................................... 46

    4.1 Limitations and Future Directions ................................................................................. 48

    References................................................................................................................................ 50

    Appendix A. Factor Analyses for Booklet 1 and Booklet 2 .................................................... 54

    Appendix B. Means, Standard Deviations and Frequencies of Students‟ Questionnaire

    Responses for Grouping Variable Sample, Booklet 1 and Booklet 2...................................... 56

    Appendix C. The Relationship between Total and Scaled Scores for Booklet 1 and Booklet 2

    ................................................................................................................................................. 58

    Appendix D. Missing Item Data for Booklet 1 and Booklet 2 ................................................ 60

    Appendix E. Grouping Variable Sample: Item Statistics for Booklet 1 and Booklet 2 .......... 63

    Appendix F. Anchor Items Eliminated: Item Discrimination for Booklet 1 and Booklet 2 .... 69

    Appendix G. Item Analysis be Section for Booklet 1 and Booklet 2 ...................................... 72

    Appendix H. Chi-Square Analyses for Reading Strategies by Item, Booklet 1 and Booklet 2

    ................................................................................................................................................. 75

    Appendix I. DIF with Total Matching Score for Booklet 1 and Booklet 2 ............................. 78

  • vi

    LIST OF TABLES

    Table 1 Student Questionnaire: Assessment of Reading Strategies .......................................... 9

    Table 2 Means, Standard Deviations and Frequencies of Students’ Questionnaire Responses,

    Booklet 1 .................................................................................................................................. 10

    Table 3 Means, Standard Deviations and Frequencies of Students’ Questionnaire Responses,

    Booklet 2 .................................................................................................................................. 11

    Table 4 3-Factor Model: Reading Strategies .......................................................................... 13

    Table 5 2-Factor Model ........................................................................................................... 14

    Table 6 Item Statistics, Booklet 1 ............................................................................................ 20

    Table 7 Item Statistics, Booklet 2 ............................................................................................ 21

    Table 8 Distractor Analysis, Booklet 1 .................................................................................... 24

    Table 9 Distractor Analysis, Booklet 2 .................................................................................... 25

    Table 10 Item Discrimination by Subscores, Booklet 1 ........................................................... 27

    Table 11 Item Discrimination by Subscores, Booklet 2 ....................................