® Copyright © 2009 Educational Testing Service. 1 Observations from a Lifetime in Accountability...

®

Copyright © 2009 Educational Testing Service.

1

Observations from a Lifetime in Accountability Testing

Wendy M. YenETS

CERANovember 19, 2009

®

1. Everyone wants to help children learn.


2

• Opinions differ about how to help children.• There is conflict in educational policy.

• Opinions are strongly held about how to help children.

®

2. Everyone wishes that every student would learn a lot.


3

• All educators believe that their students’ test scores should be higher than they are.

• Many people work very hard every day to help students learn.

®

2. Everyone wishes that every student would learn a lot (cont.)


4

• There must be problems with the existing education system.• Every system can be improved.• If the existing system is changed, it will be improved.

• The amount that students are learning is never enough.

There seem to be general beliefs that:

®

3. Educational systems are very complex.


5

• Their social, economic, & political surroundings are constantly changing and being changed by education.• All the implications of any action or change cannot be foreseen.

• Educational systems are reactive and dynamic.

®

4. Within this complex system what can accountability testing contribute?


6

• AT can spur thought into the meaning and alignment of content standards, curricula, instruction, tests, and performance standards.

• AT can help define what we want our students to learn and increase focus on it.

®

4. What can accountability testing contribute? (cont.)


7

• AT can help us see what change is taking place & what change is not:

• From 2003 to 2009 the CST average increase per year in percent proficient was 2.5% in ELA and 1.8% in Math.

• Gaps in performance persist.

®

4. What can accountability testing contribute? (cont.)


8

• Historical AT information provides setting in which we can evaluate the difficulty of goals and the resources needed to reach them.

• Stretch goals that are attainable can be developed.

®

5. There can be unintended negative consequences of accountability testing:


9

• Everyone is stressed by AT.

• High standards are good but unrealistically high standards are counter-productive.

• AT can receive too much emphasis in evaluating educational effectiveness.

®

6. Within this ever-changing, imperfect system, what can a psychometrician contribute?


10

• Offer honest advice about what works and does not work, what is accurate and fair and what is not

• Ask lots of questions

• Attempt to communicate meaningfully; explain “why”

®

6. What can a psychometrician contribute? (cont.)


11

• Bias analyses

• A scientific basis for data analysis procedures, such as

• Scaling• Equating

• Documentation of those procedures

®



12

• Equating is what allows score comparisons from year to year.

• It’s the equating, Stupid.

• If scores “bounce around,” they are not useful.• Changes in test structure undermine good equating.

®



13

• Dedication to 100% accuracy

• Detailed focus on procedures, data, and results• Broad awareness of the possibility of the unexpected• Constant query: Do the results make sense?

®



14

• Humility, given that

• Respect for the unique knowledge, good intentions, and hard work of others

• many people are affected by AT

• others, not AT, are the direct cause of student learning

® Copyright © 2009 Educational Testing Service. 1 Observations from a Lifetime in Accountability...

Documents

Transcript of ® Copyright © 2009 Educational Testing Service. 1 Observations from a Lifetime in Accountability...