Test Security Guidebook

West Virginia: A State Perspective

An Important Addition

• A compilation of “Best Practices” • 3 Key Areas– Prevention– Detection– Investigation

• Key Implications:– Potential source to conduct a gap analysis– Preparation for any test security requirements

State Awareness Timeline

• Test security has always been of interest to SEAs

• Security breaches have thrust assessment issues into the spotlight

• SEAs, LEAs, and schools now have a dedicated interest in test security and assessment integrity

Assessment Integrity

Ensuring the validity and reliability of assessment results through standardized

and secure assessment conditions. ®

A State’s Potential Context

• The 3 Ps of Assessment Integrity– Policy– Politics– Practical Implications (i.e., reliability of results)

• What is the driver?– Timing + Severity + Pervasiveness + Climate =– Crapshoot

A State’s Potential Question

• How often have people in testing programs been asked to engage in a potentially controversial decision in advance of clear and distinct policy or practice?

• Drives the need for solid guidance to drive best practice.

WV’s Approach

Greatest Area of Emphasis

Primary Focus

• Prevention– Training– Security of Test Materials– Policy-based Definitions– Code of Ethics– Signed Agreements

Secondary Focus

• Investigation– Allegations – Pattern Analyses– Gains Analyses– Qualitative Analyses• Documentation• Interviews• Written/oral complaints or reports

Secondary Focus Cont’d

• Detection– Erasure Analyses– Reporting Protocols– Investigation Protocols

Newest Areas of Focus(West Virginia)

• Detection– Staff assigned to troll:• Twitter• Pinterest• Facebook

– New Technologies• Snapchat• http://www.snapchat.com/#What-is-snapchat?• Unmonitored photos that are automatically deleted

Result and Implications

• Uncovered more instances of test administrator violations – Kids sleeping– Student tweets (positive and negative)– A few cases of administrators sleeping

• What’s next?– Documenting assigned roles and responsibilities,

as well as the process.

Where are we now?

• Waiting for the next AJC article• (Quality?) Information everywhere about test

security practices, audits, and requirements.• The guidebook providing a comprehensive

look at the three major categories• Taking recommendations to create actionable

information that practitioners can use to revise practice and policy

Using the Guidebook

• Prevention– Documentation that can be used for a “best-practices

check” • Detection– Potentially a weak area for states– Need defensible analyses to mitigate Type 1 and 2 error– More sources for triangulation

• Follow-up Investigations– Establish policies and procedures to have something to

reference (e.g., invalidations)

Thank you

Contact Information

Juan D’BrotWest Virginia Department of Education

Executive DirectorOffice of Assessment and Accountability

Office of Researchjdbrot@access.k12.wv.us