Slide 1

An Overview ofVirginia Standards of Learning Item and Test Development

Division Director of Testing Regional Meeting Spring 20121Purpose of this PresentationDescribe the Item and Test Review Committees

Provide an overview of Standards of Learning (SOL) Item and Test Development

Division Director of Testing Regional Meeting Spring 20122Item and Test Review CommitteesOn an annual basis, Virginia educators from across the state participate in the development of the Virginia Standards of Learning (SOL) assessments.

Every summer, Item and Test Review Committees convene in the Richmond area to review test items and test forms for the SOL assessment program.

Division Director of Testing Regional MeetingSpring 20113Item and Test Review CommitteesContent committees are composed primarily of educators teaching the subject of the test, as well as instructional specialists or other curriculum staff.

Committee members represent a geographical cross-section of Virginia.

Division Director of Testing Regional MeetingSpring 20114Item and Test Review CommitteesNew members are added each year in order to provide for a balance of experienced educators and new members, and to bring new perspectives into committee meetings.

Division Director of Testing Regional MeetingSpring 20115Item and Test Review CommitteesCommittee members are selected based on the following criteria:grade-level experience and content area expertise; in-depth knowledge of the SOL; instructional/supervisory experience with students of varied learning styles, abilities, and aptitudes, including students with disabilities and students with limited English proficiency; and balanced regional representation.

Division Director of Testing Regional MeetingSpring 20116Item and Test Review Committees The Item and Test Review Committees play an important and essential role at three major stages of the Test Development Process:review of new field-test items;data review of field-tested items; andreview of test forms.

An overview of the Item and Test DevelopmentProcess follows.

Division Director of Testing Regional MeetingSpring 20117Overview of Steps in Test DevelopmentItem is Written: The test item is developed by Virginias testing contractor.Item Review: The new test item is reviewed by an Item and Test Review Committee, comprised of Virginia educators. If approved, the new item can be field-tested on a Standards of Learning (SOL) test.Field-Testing: The new item is placed on an SOL test and tried out with Virginia students.Data Review: The item with its field-test data is reviewed again by an Item and Test Review Committee, comprised of Virginia educators. If approved, the item can be used on a test as an item that counts towards a students score.Test Forms Review: Newly constructed tests are reviewed by the educators on the Item and Test Review Committee.

Division Director of Testing Regional Meeting Spring 20128Item Development Overview Item Writing Phase

The testing contractor develops items:Item writers are trained specifically to interpret the SOL while using the Curriculum Framework.

Division Director of Testing Regional Meeting Spring 20129Item Development Overview Item Writing Phase (continued)Items are reviewed by the testing contractor for:content accuracy and correct answer;match to the SOL; andbias and sensitivity (fairness for all groups): an item is free from prejudice, favoritism or stereotyping regarding gender, ethnic, religious, political, age, or socio-economic group; there is no advantage or disadvantage for any one group over other groups of students.

Division Director of Testing Regional MeetingSpring 201110 Item Review Phase Educators on the Item and Test Review Committee examine and judge each item for:

content accuracy and correct answer;its match to the SOL;appropriateness; andfairness to all groups.Item Development Overview

Division Director of Testing Regional Meeting Spring 201211Items accepted by the committee are eligible for field-testing.Newly approved items are tried out by Virginia students.Field-test items are included in the SOL tests but are not counted in student scores.Field-Testing PhaseItem Development Overview

Division Director of Testing Regional Meeting Spring 201212ngItem Development OverviewData Review Phase

Student performance on items is described by field-test statistical results.The Virginia Item and Test Review Committee examines field-tested items for a second time and reviews field-test data.The committee makes judgments about items based on:match to the SOL;appropriateness;fairness to all groups; andfield-test data.

Division Director of Testing Regional Meeting Spring 201213Test Development OverviewTest Form Construction Phase

The testing contractor uses items approved by the Item and Test Review Committee to construct new tests.

Newly constructed tests must match the test blueprint and meet statistical criteria.

The Item and Test Review Committee examines new tests before they are administered.

Division Director of Testing Regional Meeting Spring 201214Test Development Overview Tests are administered to Virginia students and scored using only the approved operational (non-field-test) items. All tests for a given SOL test are statistically equated and score results are produced. This is a very important step to ensure that students are required to demonstrate the same level of proficiency regardless of which test form they take. That is, scores carry the same meaning regardless of which test form a student takes. Only then are test results produced and sent to the divisions throughout the state.

Division Director of Testing Regional Meeting Spring 201215For further questions, contact student_assessment@doe.virginia.gov

Division Director of Testing Regional Meeting Spring 201216