Ready for Online Assessments? Help is Here.Call-in number: 571-392-7703 | PIN: 635 397 980 649

 October 2013

Poll #1

A. SETDA/Assess4ed Promotion

B. Connected Educators Month Promotion

C. Twitter/Facebook

D. A Colleague

How did you hear about this event?

About SETDA

• Ten-year old DC-based, national, non-profit member association

• Serve, support, and represent U.S. state and territorial directors (SEA leadership) for educational technology

• Forum for:– Research and best practices – Inter-state collaboration– Professional development– Public-private partnerships– State-federal relations

SETDA Resources for Online Assessment Readiness

• assess4ed.net• Technology Requirements for Large-scale

Computer-based and Online Assessment• Technology Readiness for College and Career

Ready Teaching, Learning and Assessment• SEPC• Implementing Online Assessments – Pathways to

Success

For access to SETDA’s resources to go -

http://www.setda.org/web/guest/assessment

Agenda

• Welcome– Geoff Fletcher, Deputy Executive Director, SETDA

• Implementing Online Assessment Website Overview District Sharing– Mike Nelson, Director of Curriculum and assessment of Coeur d"Alene

School District 271, ID– Patches Hill, Technology Systems Manager of Indian River School

District, DE– Scott Smith, Chief Technology Officer of Mooresville Graded School

District, NC

• Q&A, Geoff Fletcher, SETDA

Poll #2

A. Technology contact

B. Assessment contact

C. I am driving the whole show

D. I am doing nothing to help prepare – yet

E. I will implement it when it happens

Chose the statement that best describes your role in helping your state/district prepare for online assessments.

Presenters

Scott Smith

Chief Technology Officer

Mooresville Graded School District, NC

Patches Hill Technology Systems ManagerIndian River School District, DE

Mike Nelson Director of Curriculum and Assessment Coeur d’Alene School District 271, ID

Effective Use Online

Platforms for Formative & Summative

Assessment

Michael S. NelsonDirector of Curriculum and Assessment

Coeur d’Alene School District (Idaho)msnelson@cdaschools.org

Coeur d’Alene School District 271http://www.cdaschools.org

Public school district of 10,300 students in Idaho 530 teachers in 17 schools (7 of last 14 State teachers of the year)

11 elementary; 3 middle schools; 2 traditional high schools 6 magnet or charter schools

Part of the Smarter Balanced Assessment Consortium Highest performing large district in Idaho 90GB of bandwidth coming into our district

17 mobile laptop/iPad labs available if needed Blended learning for several courses

District Background

Professional Devel. Approach

We empower faculty through:•Job embedded staff collaboration

– Weekly on Mondays– 1 day per month district-driven

•“Top 40” Professional Devel. Team– Stiggins’ Assessment Training Institute

•Building Leadership opportunities

Our Assessments are used for…

• Diagnostic to determine the student’s pre-knowledge and

identify strengths & weaknesses

• Formative and Interim to provide the student with feedback on their

progress during a course

• Summative to estimate performance at the end of a course

and ‘grade’ the student’s work

Smarter Balanced Questioning Types

Our Assessment Quandary   Discovery

Education PaperScannable

OMR Schoolnet SkywardDelivery Options

Clicker Delivery       XX  

Online Delivery XX     XX XX

Paper Delivery XX XX XX   XX

Question TypesSelected Response

XX XX XX   XX

Constructed Response

XX XX   XX XX

Technology Enhanced

XX        

Performance Tasks

  XX      

End of Exam

Item Analysis     XX XX  

Reporting Features

    XX XX XX

Import to Grades         XX

Costs

Ease of Use XX XX      

Administration Ease of Use

XX        

Annual Maintain and/or Upgrades

XX     XX  

Other Considerations

Ties to Standards       XX  

Connects to Rubrics

      XX  

We Chose Skyward…

After evaluating options, we found that our student management system did the best in:

•test factual knowledge effectively

•include web-based resources, media objects

•track student attempts

•provide detailed, specific feedback

•mark automatically, quickly, consistently

•provide question banks, randomized ordering

•produce statistical information for test items

•Export to gradebook and statistical packages

Areas of Note• construction of good objective tests still depends on

instructors

• building up question pools is time-consuming

• specialized mathematics and scientific notation still cumbersome to incorporate

• provision of secured test environment

• review or grading essays and project-based assessment/performance tasks

• detect plagiarism in essays

• robust infrastructure is required to avoid failure during exams

Students Taking

Assessments

Getting to the Exam

Click on “Open Student Access” on the upper left corner of the screen.

Student “assignment” will be listed at the top of the screen along with their current grades and academic calendar.

Click on “Take Assignment”

Ending the Exam

When choosing “Save and Complete Exam” a summary screen will appear asking you to review your work and ask whether you want to complete the assessment.

Click “Yes” to end.

Essay Questions

• Read the prompt carefully.• Create a well-crafted essay that fully answers

the prompt.

• You can expand the response box by dragging down on the bottom-right corner of the window.

Selected Response Stems

• These are NOT traditional multiple choice.• Each question may have multiple correct

answers.• Read the question carefully AS WELL AS

EACH answer choice.• Place an X in each of the boxes that correctly

answer the question correctly.• There may be an image, document or other

information available for answering your question.

• Look for the button

X

X

X

Post-Assessment Analysis

Post-test, we have the ability to analyze each assessment by question stem or by teacher distribution.•Report-based or chart form

Final Thoughts

Organizational Support Structure

Phase-In • Work with provider• Pilot work at different levels• Constantly communicate

Provide Supports in a variety of methods

• How to handouts• PowerPoint presentations• Short instructional videos• Top 40 Team• Technology Leaders• Hand holding may be

necessary

Provide Timely Results and Successes

• Empower faculty to manipulate and use results

• All assessment should inform instruction

• Triangulate results with other works

Thank You.

Michael S. NelsonDirector of Curriculum and Assessment

Coeur d’Alene School District (Idaho)msnelson@cdaschools.org

Delaware, IRSD and Online Assessments

Summary of levels A multi-step process at multiple levels

◦ State infrastructure implementation◦ State assessment implementation – summative standardization ◦ Personalized learning and formative standardization

History and Background Delaware has only three counties that encompass 19 school districts (16 traditional districts and 3 county-wide vocational/technical districts).

The move to adopt and fully implement Common Core State Standards (CCSS)-aligned assessments in 2014 builds on Delaware’s nearly twenty-year history of planning toward better information sharing across the state.

Faced with standardized testing contract expiration after the 2009-2010 school year, the state formed a taskforce with two priorities for the next iteration of tests: immediate scoring, and the ability to track data over time. These priorities led to the group’s ultimate recommendation for a shift to online assessment.

Delaware wanted to emphasize testing flexibility in how the test is administered and within the assessment. The Delaware Comprehensive Assessment System (DCAS) provides multiple opportunities for students to demonstrate proficiency and provides academic achievement including a measure of fall-to-spring and year-to-year individual student growth.

Delaware has a law requiring all state-mandated and high-stakes assessments to be administered digitally.

Key Success Factors Centralized statewide network that provides Internet connectivity to each school.

Statewide pupil accounting system that tracks accommodations used by every district already in place.

Annual school technology survey to assist in determining readiness.

Stakeholders involved starting very early in the field testing process and focused on technology capacity and hardware issues, particularly the logistics of setting up schools.

What districts did Prior to field testing in 2010, districts and schools were using technology for instruction, but administering paper-and-pencil assessments.

Assessment no longer meant entire buildings would have to shut down for several days while all students inside completed the exams because of the move to broad assessment windows.

Technology Readiness – Including network, bandwidth and computers readiness.

Teacher/Staff Assessment Readiness – Including training of district technology coordinators, as well as training of school and district test administrators.

Student Readiness – All students viewed a presentation that introduced the DCAS field test and completed both a computer skills assessment and a training test.

Scheduling – Districts developed careful schedules to complete in time allowed.

What districts did Displaced some instruction because there aren’t enough laptops with particular features to meet all demand scenarios.

Establishing Wi-Fi in every school to assure secure, reliable, redundant connections. Currently, all but two buildings are wireless at 100% coverage of facility.

PLCs provided a structure for decentralized decision making about the timing and configurations for online assessment that work best.

Despite the considerable flexibility offered by broad testing windows, most schools continue to follow the more traditional approach of holding shorter testing windows comprised of a few days during which nearly all students are tested.

Four schools at IRSD (2 high schools, 1 middle school and 1 elementary school) are participating in the spring 2013 SMARTER Balanced pilot test.

Key Factors for Future Systems Adoption at IRSD

As we move to more personalized learning it will prove key to build systems that follow open standards.

Joined BRINC(State) and IMS GLC (International).

IMS Global and its members have helped us significantly in understanding the total interoperability of our data and how they should integrate with other state and local systems.

Only choosing systems that follow QTI and APIP at district level to keep impact of assessment database longevity and interoperability from being a concern. Also focusing on Common Cartridge (CC) and Learning Tools Interoperability (LTI).

IMS Standards Prove Key @ IRSD

QTI - Question and Test Interoperability is a technical specificationthat describes formats & services for exchange of digital assessment items and tests among assessment system software components.

◦ XHTML based◦ Permits inclusion of other mark-ups, e.g., MathML, ChemML◦ Supports alternative accessibility representations◦ Future version will merge with HTML5 & digital book formats

APIP Accessible Portable Item Protocol is a technical specificationthat describes formats & services for exchange of accessible digital assessment items and tests among assessment system software components.

◦ APIP v1.0 uses a precisely defined subset of QTI v2.1 ◦ Expands the accessibility content feature in QTI◦ Uses accessibility definitions from IMS AccessForAll specifications (Metadata and Personal

Needs and Preferences - PNP)

How does APIP relate to QTI in terms of origin?

QTI has been implemented and evolved over 10+ years of experience of leading assessment, publisher and learning platform providers worldwide.

APIP arose out of seed funding from the U.S. Department of Education to a project led by a consortium of U.S. states.

Conformance Profiles

Additional LinksDistrict Website

http://www.irsd.net/

Materials Used

http://assessmentstudies.setda.org/resources/

IMS Overview

http://www.youtube.com/watch?v=corV3-WsIro

Open Infrastructure

http://www.imsglobal.org/IMSSupportsK12.pdf

Mooresville Graded School District Scott S. Mooresville Graded School District Scott S. SmithSmithDigital ConversionDigital Conversion Ed.DEd.D

Mooresville Graded School District:Mooresville Graded School District:DemographicsDemographics

Number of Schools

8

Certified Staff ~ 400

Non Certified Staff ~ 300

Total StudentsTotal StudentsK-12K-12 59005900

Asian 2%

Multi Racial 3%

Hispanic 8%

African American

17%

Caucasian 70%

Students receiving Free/Reduced Lunch has increased in MGSD...

Why a Digital Why a Digital Conversion?Conversion?• Close the digital divideClose the digital divide

• Relevant instructionRelevant instruction

• 21st century readiness21st century readiness

• Real world experienceReal world experience

• Instructional practiceInstructional practice

• Improved academic Improved academic achievementachievement

• Moral imperativeMoral imperative

IngredientsIngredients

• Building the cultureBuilding the culture

• 21st century tools and 21st century tools and online resourcesonline resources

• Data rich & intensive – Data rich & intensive – with immediate with immediate feedback and resultsfeedback and results

• Capacity buildingCapacity building

• All in!!!All in!!!

The Tech SpecsThe Tech Specs

• Over 5500 MacBook Airs for every teacher and student Over 5500 MacBook Airs for every teacher and student grades 3-12grades 3-12

• Online content available virtually 24 / 7Online content available virtually 24 / 7

• Ubiquitous wirelessUbiquitous wireless

• 802.11n Access Point in every classroom (at 802.11n Access Point in every classroom (at minimum)minimum)

• High availability connectivityHigh availability connectivity

• 500 MB Pipe for the district500 MB Pipe for the district

• GB Fiber between locationsGB Fiber between locations

• Cultural shift / personalized instructionCultural shift / personalized instruction

• Support for online assessmentsSupport for online assessments

Online AssessmentsOnline Assessments

• Daily online assessments in Angel (Learning Daily online assessments in Angel (Learning Management System)Management System)

• Implementation of Lock Down Browser in conjunction Implementation of Lock Down Browser in conjunction with Angelwith Angel

• Quarterly Assessments with ThinkGateQuarterly Assessments with ThinkGate

• End of Grade and end of Course Exams onlineEnd of Grade and end of Course Exams online

Meeting Bandwidth NeedsMeeting Bandwidth Needs

• Prior to Digital Conversion Prior to Digital Conversion

• 40MB Connection40MB Connection

• Year 1 (2008 – 2009)of Digital Conversion (~2200 Year 1 (2008 – 2009)of Digital Conversion (~2200 laptops)laptops)

• 100 MB Connection 100 MB Connection

• Year 2 (2009 – 2010) of Digital Conversion (~4000 Year 2 (2009 – 2010) of Digital Conversion (~4000 laptops)laptops)

• 250 MB Connection 250 MB Connection

• Year 4 (2011-12)of Digital Conversion (~5000 laptops)Year 4 (2011-12)of Digital Conversion (~5000 laptops)

• 500 MB Connection500 MB Connection

• Year 6 (2013-14) of Digital Conversion (~5500 laptops)Year 6 (2013-14) of Digital Conversion (~5500 laptops)

• 500 MB Connection (500 MB Connection (watchingwatching closelyclosely))

Online AssessmentsOnline Assessments

• Average week – 274 MB of a 500 MB pipe (Sept 16 – Average week – 274 MB of a 500 MB pipe (Sept 16 – Sept 20, 2013)Sept 20, 2013)

June 3, 2013 – First day of Online June 3, 2013 – First day of Online ExamsExams

• Average 173 MB of a 500 MB pipeAverage 173 MB of a 500 MB pipe

• Honor School of ExcellenceHonor School of Excellence• High GrowthHigh Growth• Met 100% AMO TargetsMet 100% AMO Targets

Mooresville High SchoolMooresville High SchoolGrades 9 - 12Grades 9 - 12

Composite increase of 23 Composite increase of 23 percentage points over 6 percentage points over 6

yearsyears

Overall Composite for Students Testing ProficientOverall Composite for Students Testing Proficienton State End-of-Course Examson State End-of-Course Exams

Mooresville Graded School DistrictMooresville Graded School DistrictGetting students across the line!Getting students across the line!

4-Year Cohort Graduation Rate4-Year Cohort Graduation Rate

MGSD is 3rd MGSD is 3rd in the state in the state out of 115 out of 115

districts for districts for its 2012 its 2012

graduation graduation rate!rate!

Mooresville Graded School DistrictMooresville Graded School DistrictDistrict AchievementDistrict Achievement

16%16%Growth!Growth!

16%16%Growth!Growth!

Mooresville Graded School DistrictMooresville Graded School DistrictDistrict AchievementDistrict Achievement

DistrictDistrictPercent Proficient Percent Proficient

using EOG Reading, using EOG Reading, Math, Science; EOCMath, Science; EOC

Per Pupil Per Pupil ExpenditureExpenditure

(state ranking)(state ranking)

District District Enrollment Enrollment NumbersNumbers

11 Camden CountyCamden County 90%90% 7989 (73)7989 (73) 18911891

22Mooresville Mooresville

Graded School Graded School DistrictDistrict

89%89% 7463 (100)7463 (100) 56225622

Polk CountyPolk County 89%89% 9604 (22)9604 (22) 23632363

33Chapel Hill Chapel Hill CarrboroCarrboro 88%88% 8979 (39)8979 (39) 11,50411,504

MGSD Ranked MGSD Ranked 2nd in the 2nd in the

State State 2011-122011-12

Formula: Number of Proficient Test Scores on Reading Grades 3-8, Math Grades 3-8, and EOCs divided by the total number of tests Formula: Number of Proficient Test Scores on Reading Grades 3-8, Math Grades 3-8, and EOCs divided by the total number of tests taken in school year 2011-2012.taken in school year 2011-2012.

Questions?

Webinar Archive & Resources

http://www.setda.org/web/guest/connectededucators

http://assessmentstudies.setda.org

More information: Geoff Fletcher, gfletcher@setda.org

Claim your CEM Digital Badge

• Go to http://badges.connectededucators.org• Click the “Have claim code?” button• Enter the code: rere124• Click “Claim”

NOTE: Participants will have to create a Mozilla account and backpack to claim a badge if they do not have an account already.

CEM Activities for State Leadership

http://connectededucators.org/cem-for-states/

http://connectededucators.org/events/#/audience=249

Webinar Resources

Webinar Archive & Resources

http://www.setda.org/web/guest/connectededucators

http://assessmentstudies.setda.org

More information: Geoff Fletcher, gfletcher@setda.org