Technology's Roles in the NCLB/ SOL Assessment Dance David Mott, Reports Online Systems and Tests...
-
Upload
richard-gibson -
Category
Documents
-
view
217 -
download
0
Transcript of Technology's Roles in the NCLB/ SOL Assessment Dance David Mott, Reports Online Systems and Tests...
![Page 1: Technology's Roles in the NCLB/ SOL Assessment Dance David Mott, Reports Online Systems and Tests for Higher Standards Presentation at the Virginia Society.](https://reader033.fdocuments.in/reader033/viewer/2022061305/551431475503466d1a8b5030/html5/thumbnails/1.jpg)
Technology's Roles in the NCLB/SOL Assessment Dance
David Mott, Reports Online Systems and Tests for Higher Standards Presentation at the Virginia Society for Technology in Education (VSTE), Virginia Beach, February 27, 2007
![Page 2: Technology's Roles in the NCLB/ SOL Assessment Dance David Mott, Reports Online Systems and Tests for Higher Standards Presentation at the Virginia Society.](https://reader033.fdocuments.in/reader033/viewer/2022061305/551431475503466d1a8b5030/html5/thumbnails/2.jpg)
Technology's Roles in the NCLB/SOL Assessment Dance
Technologists have vital roles to play in the current dance involving standards-based testing, the NCLB, AYP, school accreditation, and so on. Initially, technology was called upon to supply basic computer connectivity for internet-based, computer-delivered SOL tests. Having now largely accomplished this limited task, educators involved with technology are faced with a far more difficult task — supporting teachers and administrators in one of the highest profile sectors of the current era, assessment and testing. This support requires both IT knowledge and a broad understanding of the educational process itself. Automating parts of the assessment process can produce a very high return on investments in technological expertise and money. Properly deployed and implemented, IT-supported systems that gather and analyze test data, can give all levels of decision makers potent information. This session covers the specifics of how current technology can support data-informed instruction. We will talk about formative and summative assessment systems, the “right-sizing” of assessment technology, proper data collection and use, and the need for end-user training in all aspects of these new data based approaches. The context will be real-world, with specific examples using the Reports Online System with the Test for Higher Standards assessments. However, the general principals apply to any powerful automated assessment system.
![Page 3: Technology's Roles in the NCLB/ SOL Assessment Dance David Mott, Reports Online Systems and Tests for Higher Standards Presentation at the Virginia Society.](https://reader033.fdocuments.in/reader033/viewer/2022061305/551431475503466d1a8b5030/html5/thumbnails/3.jpg)
About me My childern –
The animals –
Colin
JessicaDylan
Zack
Puck
Awk
![Page 4: Technology's Roles in the NCLB/ SOL Assessment Dance David Mott, Reports Online Systems and Tests for Higher Standards Presentation at the Virginia Society.](https://reader033.fdocuments.in/reader033/viewer/2022061305/551431475503466d1a8b5030/html5/thumbnails/4.jpg)
About Me (so that you know where I’m coming from)
I’m an experimental psychologist. Was at the VA DOE for 17+ years, in what
is now called Assessment and Reporting. Supervisor of Test Development (and
psychometrics – test equating, etc.) Administrator of the VSAP (VA Student
Assessment Program, SAT-9) The division’s liason with the IT group In charge of procurements
![Page 5: Technology's Roles in the NCLB/ SOL Assessment Dance David Mott, Reports Online Systems and Tests for Higher Standards Presentation at the Virginia Society.](https://reader033.fdocuments.in/reader033/viewer/2022061305/551431475503466d1a8b5030/html5/thumbnails/5.jpg)
How did I get into this business test publishing and scoring?
After I left the VA DOE, what I was going to do for the rest of my life?
Got together with Stuart Flanagan to do some SOL Mathematics tests.
Then came SOL English, Science, and History & Social Science tests.
Then we developed ROS so that teachers didn’t have to hand score tests.
![Page 6: Technology's Roles in the NCLB/ SOL Assessment Dance David Mott, Reports Online Systems and Tests for Higher Standards Presentation at the Virginia Society.](https://reader033.fdocuments.in/reader033/viewer/2022061305/551431475503466d1a8b5030/html5/thumbnails/6.jpg)
Some Vital Roles for Technology and Technologists in the NCLB Era —
Supply connectivity to the LAN & Internet for: Administrative data bases Testing activities Student access Teachers’ uses
Training of: Administrators Teachers Students
![Page 7: Technology's Roles in the NCLB/ SOL Assessment Dance David Mott, Reports Online Systems and Tests for Higher Standards Presentation at the Virginia Society.](https://reader033.fdocuments.in/reader033/viewer/2022061305/551431475503466d1a8b5030/html5/thumbnails/7.jpg)
Some Vital Roles for Technology and Technologists in the NCLB Era —
Recommend, buy, install, and support hardware
Recommend, buy, install, and support software
Plan for future directions for technology Participate in the production of technology
related products Help school personnel use the available
technology
![Page 8: Technology's Roles in the NCLB/ SOL Assessment Dance David Mott, Reports Online Systems and Tests for Higher Standards Presentation at the Virginia Society.](https://reader033.fdocuments.in/reader033/viewer/2022061305/551431475503466d1a8b5030/html5/thumbnails/8.jpg)
Humor
Time flies like an arrow;
fruit flies like a banana.
![Page 9: Technology's Roles in the NCLB/ SOL Assessment Dance David Mott, Reports Online Systems and Tests for Higher Standards Presentation at the Virginia Society.](https://reader033.fdocuments.in/reader033/viewer/2022061305/551431475503466d1a8b5030/html5/thumbnails/9.jpg)
Humor
Time flies like an arrow;
fruit flies like a banana.
Time flies like an arrow – but an arrow flies in only one direction! The point is that whatever happens to education in the present era, education will never go back to the way it was.
The point
![Page 10: Technology's Roles in the NCLB/ SOL Assessment Dance David Mott, Reports Online Systems and Tests for Higher Standards Presentation at the Virginia Society.](https://reader033.fdocuments.in/reader033/viewer/2022061305/551431475503466d1a8b5030/html5/thumbnails/10.jpg)
A specific area: Data-Informed Decisionmaking
Why Data-Informed rather that Data-Driven?
Because data is not alive, it can’t be in the driver’s seat.
A person needs to be at the controls.
![Page 11: Technology's Roles in the NCLB/ SOL Assessment Dance David Mott, Reports Online Systems and Tests for Higher Standards Presentation at the Virginia Society.](https://reader033.fdocuments.in/reader033/viewer/2022061305/551431475503466d1a8b5030/html5/thumbnails/11.jpg)
Data-Informed Decisionmaking — continued 1
Bus Schedules — Data % ontime lengths of route in miles # of students transported on bus
Bus Schedules — Questions with Answers Are the shorter routes ontime more often? What is the average student’s route distance? and so on.
![Page 12: Technology's Roles in the NCLB/ SOL Assessment Dance David Mott, Reports Online Systems and Tests for Higher Standards Presentation at the Virginia Society.](https://reader033.fdocuments.in/reader033/viewer/2022061305/551431475503466d1a8b5030/html5/thumbnails/12.jpg)
Data-Informed Decisionmaking — continued 2
School Lunches — Data # of lunches served in a day Menu items SOL testing day or not a testing day
Bus Schedules — Questions with Answers Which menus are most popular? Are more lunches served on SOL testing days,
and is that dependent on the menu that day? and so on.
![Page 13: Technology's Roles in the NCLB/ SOL Assessment Dance David Mott, Reports Online Systems and Tests for Higher Standards Presentation at the Virginia Society.](https://reader033.fdocuments.in/reader033/viewer/2022061305/551431475503466d1a8b5030/html5/thumbnails/13.jpg)
Data-Informed Instruction — Testing
Used as an example because it can be one of the heaviest and most mission-critical uses of technology – and because I know the most about it.
Also, there is much payback to be had from properly using testing data to inform instruction – which is, after all, the business we are in.
![Page 14: Technology's Roles in the NCLB/ SOL Assessment Dance David Mott, Reports Online Systems and Tests for Higher Standards Presentation at the Virginia Society.](https://reader033.fdocuments.in/reader033/viewer/2022061305/551431475503466d1a8b5030/html5/thumbnails/14.jpg)
The ExaminationThe Examinationas shown by the Ghost of Testing Pastas shown by the Ghost of Testing Past
The ExaminationThe Examinationas shown by the Ghost of Testing Pastas shown by the Ghost of Testing Past
![Page 15: Technology's Roles in the NCLB/ SOL Assessment Dance David Mott, Reports Online Systems and Tests for Higher Standards Presentation at the Virginia Society.](https://reader033.fdocuments.in/reader033/viewer/2022061305/551431475503466d1a8b5030/html5/thumbnails/15.jpg)
The Parts of an Assessment System
Student Information Database Test Generating System Test Presentation System Test Response Recording System Test Scoring System Test Reporting System Actionable Information
![Page 16: Technology's Roles in the NCLB/ SOL Assessment Dance David Mott, Reports Online Systems and Tests for Higher Standards Presentation at the Virginia Society.](https://reader033.fdocuments.in/reader033/viewer/2022061305/551431475503466d1a8b5030/html5/thumbnails/16.jpg)
The Parts of an Assessment System Simple example, 7 parts – Teacher’s
Gradebook System
Student Information Database Test Generating System Test Presentation System Test Response Recording System Test Scoring System Test Reporting System Actionable Information
![Page 17: Technology's Roles in the NCLB/ SOL Assessment Dance David Mott, Reports Online Systems and Tests for Higher Standards Presentation at the Virginia Society.](https://reader033.fdocuments.in/reader033/viewer/2022061305/551431475503466d1a8b5030/html5/thumbnails/17.jpg)
Choice Point
Assessment systems In general
ROS Online Testing
ROS specifically, with reports
?
![Page 18: Technology's Roles in the NCLB/ SOL Assessment Dance David Mott, Reports Online Systems and Tests for Higher Standards Presentation at the Virginia Society.](https://reader033.fdocuments.in/reader033/viewer/2022061305/551431475503466d1a8b5030/html5/thumbnails/18.jpg)
The Parts of an Assessment System More complex example same 7 parts –
Reports Online System with content from Tests for Higher Standards
Student Information Database Test Generating System Test Presentation System Test Response Recording System Test Scoring System Test Reporting System Actionable Information
![Page 19: Technology's Roles in the NCLB/ SOL Assessment Dance David Mott, Reports Online Systems and Tests for Higher Standards Presentation at the Virginia Society.](https://reader033.fdocuments.in/reader033/viewer/2022061305/551431475503466d1a8b5030/html5/thumbnails/19.jpg)
Assessment System – Reports Online System w/ Tests for Higher Standards
Student Information Database Extract from SASI or other student information system Name, ID, NCLB categories, Other info.
Test Generating System Test Presentation System Test Response Recording System Test Scoring System Test Reporting System Actionable Information
![Page 20: Technology's Roles in the NCLB/ SOL Assessment Dance David Mott, Reports Online Systems and Tests for Higher Standards Presentation at the Virginia Society.](https://reader033.fdocuments.in/reader033/viewer/2022061305/551431475503466d1a8b5030/html5/thumbnails/20.jpg)
Assessment System – Reports Online System (ROS) w/ Tests for Higher Standards (TfHS)
Student Information Database Extract from SASI or other Name, ID, NCLB categories, Other info.
Test Generating System Paper test using MS Word & TfHS content Teacher-produced test District-produced benchmark test Textbook, coachbook, or other test
![Page 21: Technology's Roles in the NCLB/ SOL Assessment Dance David Mott, Reports Online Systems and Tests for Higher Standards Presentation at the Virginia Society.](https://reader033.fdocuments.in/reader033/viewer/2022061305/551431475503466d1a8b5030/html5/thumbnails/21.jpg)
Assessment System – Reports Online System (ROS) w/ Tests for Higher Standards (TfHS)
Paper test using MS Word & TfHS content Teacher produced test – printed, overhead,
oral, etc. District-produced benchmark test Textbook, coachbook, or other test
Test Presentation System Printed page, overhead projector,
whiteboard, oral, etc. Computer display
![Page 22: Technology's Roles in the NCLB/ SOL Assessment Dance David Mott, Reports Online Systems and Tests for Higher Standards Presentation at the Virginia Society.](https://reader033.fdocuments.in/reader033/viewer/2022061305/551431475503466d1a8b5030/html5/thumbnails/22.jpg)
Assessment System – Reports Online System (ROS) w/ Tests for Higher Standards (TfHS)
Test Presentation System Printed page, overhead projector, whiteboard, oral, etc. Computer display
Test Response Recording System Student written responses – notebook
paper, etc. Bubbled-in student answer sheets Responses on computer Responses on “clicker” (ROS doesn’t support this yet.)
![Page 23: Technology's Roles in the NCLB/ SOL Assessment Dance David Mott, Reports Online Systems and Tests for Higher Standards Presentation at the Virginia Society.](https://reader033.fdocuments.in/reader033/viewer/2022061305/551431475503466d1a8b5030/html5/thumbnails/23.jpg)
Assessment System – Reports Online System (ROS) w/ Tests for Higher Standards (TfHS)
Responses on computer Responses on “clicker” (ROS doesn’t support this yet.)
Test Scoring System ROS website servers w/ underlying database
One-time scoring vs. Active scoring (ROS is active.)
Cut points and variable item weights Active keys
Test Reporting System
![Page 24: Technology's Roles in the NCLB/ SOL Assessment Dance David Mott, Reports Online Systems and Tests for Higher Standards Presentation at the Virginia Society.](https://reader033.fdocuments.in/reader033/viewer/2022061305/551431475503466d1a8b5030/html5/thumbnails/24.jpg)
Assessment System – Reports Online System (ROS) w/ Tests for Higher Standards (TfHS)
Test Reporting System Reports on screen and on paper generated by
ROS website servers w/ underlying database
Item Analysis Report shows per-option response counts
Matrix Report compares performance on each item, standard, reporting category, and more
Progress Report outlines performance over time on multiple tests
![Page 25: Technology's Roles in the NCLB/ SOL Assessment Dance David Mott, Reports Online Systems and Tests for Higher Standards Presentation at the Virginia Society.](https://reader033.fdocuments.in/reader033/viewer/2022061305/551431475503466d1a8b5030/html5/thumbnails/25.jpg)
Assessment System – Reports Online System (ROS) w/ Tests for Higher Standards (TfHS)
Matrix Report compares performance on each item, standard, reporting category, and more
Progress Report outlines performance over time on multiple tests
Actionable Information A person (a teacher or administrator) needs to
combine the dry data in the reports with personal experience about students into real information.
Differentiated instruction needs to follow.
![Page 26: Technology's Roles in the NCLB/ SOL Assessment Dance David Mott, Reports Online Systems and Tests for Higher Standards Presentation at the Virginia Society.](https://reader033.fdocuments.in/reader033/viewer/2022061305/551431475503466d1a8b5030/html5/thumbnails/26.jpg)
ROS Website
Transition to the ROS website
http://ros.tfhs.net/
http://rosworks.com/
![Page 27: Technology's Roles in the NCLB/ SOL Assessment Dance David Mott, Reports Online Systems and Tests for Higher Standards Presentation at the Virginia Society.](https://reader033.fdocuments.in/reader033/viewer/2022061305/551431475503466d1a8b5030/html5/thumbnails/27.jpg)
For more information
ROS website – rosworks.com David Mott – [email protected]
866.724.7997
804.355.5489 TfHS website – www.tfhs.net