Colorado Online Testing Site Readiness Fall Field Test 2013.

Colorado Online Testing Site Readiness Fall Field Test 2013

Attendance

• Please take a moment and submit a quick attendance form http://goo.gl/15G73

• You may also send an email to Collin Bonner at [email protected] include:– Full Name– District– Title or role– Training session attended

• A record of who attends will help Pearson & CDE identify and reach out to districts who have not been trained

2

http://goo.gl/15G73

mailto:[email protected]

Agenda• Site Readiness Training Introduction

– Online Testing Components– Site Readiness Checklist

• Preparing for Testing– Firewalls & Content Filters– Proctor Caching– System Check– TestNav– Configuring Proctor Caching in PearsonAccess

• Providing Support During Testing– Early Warning System

• New Technologies– Virtualization & Online Testing– Emerging Devices & Online Testing

• Resources

3

Site Readiness Training Introduction

Testing Overview

5

6

Site Readiness - High Level Checklist

1. DTC identify School Technology Coordinators2. STC view Site Readiness Technical Training3. Configure Firewall/Content filter4. Identify testing labs and Proctor Caching Machines5. Install Proctor Caching6. Complete System Check verification on workstations and wireless

testing labs7. Update workstations to comply with TestNav hardware/software

requirements8. Verify Assessment Environments in the PearsonAccess Training

Center.9. Enter TestNav Configuration(s) in PearsonAccess Production. 10.Notify CDE’s Collin Bonner at [email protected] of the

results of step 8 for each district testing site.

mailto:[email protected]?subject=Site%20Readiness%20Training%20Attendance

Configure Firewall & Content Filter

Configure Firewall/Content Filter Takeaways

• How should I configure our Firewalls & Content Filters?

• Section Resources– Hardware and Software Requirements link for TestNav 7– TestNav 7 Technology Guidelines

Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved. 8Step 3

Firewalls, Content Filters & Spam Filters

The following domains and ports must be allowed to pass through your Internet firewalls, content filters or spam filters.

*testnav.com 80, 443*pearsonaccess.com 80, 443

Note: Districts using IP-based content filters must update the allowed IP addresses as part of this change. These districts will need to contact our Technical Support team to receive the allowed IP addresses and change a setting in PearsonAccess so that the correct URL prints on the Student Authorization Tickets.

Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved. 9

Preparing for Testing: Proctor Caching for TestNav Overview

Proctor Caching Takeaways

• What is Proctor Caching?• Where is Proctor Caching placed within the Network?• What are the Proctor Caching Requirements?• What are the steps to setup Proctor Caching?• What information is provided on the Proctor Caching Diagnostics

screens?

• Section Resources– Hardware and Software Requirements link for Proctor Caching– TestNav 7 Proctor Caching User Guide– Proctor Caching media training

Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved. 11Steps 4 & 5

Proctor Caching

Proctor Caching is Pearson-supplied software that is used in conjunction with TestNav to reduce bandwidth requirements and accelerate the delivery of test content.

• Allows you to pre-cache test content to your local network before a test

• Reduces the burden on your ISP by eliminating redundancy in requests for test content

• Stores an encrypted local copy of all pre-cached tests


Diagram of How Proctor Caching Works

13Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Proctor Caching Requirements

• Runs on Windows, Mac OS X• Does not require an underlying server-based operating system• Proctor Caching hardware requirements can be found at

http://www.pearsononlinetesting.com/TestNav/7/requirements_proctorcaching_2013_1.html

• TCP Ports: 80 (Internet), 4480, & 4481 (Local Network)• Proctor Caching requires a fixed internal IP address




Proctor Caching Setup

• Download Proctor Caching installer from PearsonAccess. – Log into PearsonAccess and click on Support->Downloads


• Run the installer and Start Proctor Caching if not already started by the installer.

• Use System Check from client computer to verify that Proctor Caching is functional.

• Create a Proctor Caching setup or add Proctor Machine within PearsonAccess at District or School level.

Proctor Caching Diagnostics

• Contents tab for content visibility• Pearson On-Line Testing integration• Client List tab for monitoring client computer connectivity


Contents Tab


• Content is displayed at the Test and Form level

• Functionality is available to Refresh, Reload or Purge selected test content

• Clicking on a test name will display the Content Details screen which displays individual items in a test

Contents Tab – Status

Ok – Content is successfully cached

Not Loaded – Content is not cachedWaiting… - Content is waiting to be loadedLoading… - Content is currently loading

Failed to load content – There was a failure to load contentMD5 Check Invalid – MD5 comparison could not be

completedMD5 Mismatch – MD5 comparison was completed but files

did not match

In general, any status in red is a critical problem. Any content with a red status should be checked and Reloaded. If the problem is not resolved contact Pearson Support


Client List Tab


• Clients which have connected to TestNav are listed by IP address

• Clicking on a computer will display the Client Details screen which displays information regarding a particular computer connecting to Proctor Caching

Client tab – Status

Ok – Client is active

Idle – 5 to 30 minutes since client was activeLong Idle – 30 minutes to 12 hours since client was active

Clients are removed after 12 hours of inactivity


Additional Technical Notes

• For setting up an upstream proxy refer to the Windows or Mac “Tips on Proctor Caching” section in the TestNav 7 Proctor Caching User Guide

• Use System Check for TestNav capacity planning in your network environment


Preparing for Testing: System Check


System Check Takeaways

• What is System Check?• How do I load System Check?• What functionality is provided by System Check?• How do I use the TestNav 7 tab?• How do I use the Network User Capacity tab?• How do I use the Testing Volume Calculator tab?

Section Resources– System Check User Guide

Step 6

System Check for TestNav Overview

• System Check was developed as a tool for customers to use to validate their computers to ensure that TestNav software could run on their computers.

• Support can also use System Check to validate the computer for any technical problems.


• To gather realistic results System Check must be run from a student testing machine on the same day of week/time of day when you anticipate performing online testing.

Loading System Check

• System Check can be loaded from: http://www.pearsononlinetesting.com/Colorado and clicking on the link for the System Check tool

• System Check is Java based and requires that you run and accept an applet


http://www.pearsononlinetesting.com/Colorado

http://www.pearsononlinetesting.com/Colorado

System Check Tabs Functionality

• TestNav 7 Tab– Validates testing workstations for TestNav

• Network User Capacity Tab– Bandwidth testing for Internet and Proctor Caching connections

• Testing Volume Calculator– Testing Volume Calculator for capacity planning


TestNav 7 Tab


• Validates computer workstation to ensure that minimum requirements are met

Network User Capacity Tab


• Internet bandwidth testing to the Pearson & Akamai distributed content network

• Click Add Caching Computer to add your potential Proctor Caching machine for a local network bandwidth test

Network User Capacity Tab: Add Caching Computer


• Add a name for the caching computer, the IP address and port (4480)

Network User Capacity Tab: Results


• Click on Start to begin the test• Based on the bandwidth test System Check can provide the number

of students that can test concurrently.

Testing Volume Calculator Tab

• Input number of students testing concurrently.– How many testing labs and student testing workstations are

available? • Modify network links and network utilization sliders as necessary.


Testing Volume Calculator Tab - Interpretation

• The graph will display the results based on your inputs. The background should remain white to ensure your planned testing volume can be supported by your available network capacity.


Planned testing is feasible with available network capacity

Planned testing is dangerously close to available network capacity

Planned testing exceeds the available network capacity

Preparing for Testing:TestNavOnline Testing Platform

TestNav Takeaways

• What are the TestNav Technology Dependencies?• What are TestNav Best Practices and Recommendations?• What do I need to know about wireless testing?• What are Student Response Files (SRFs)?• What directory permissions are needed for TestNav?

• Section Resources– Hardware and Software Requirements link for TestNav 7– TestNav 7 Technology Guidelines

Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved. 34Step 7

TestNav Technology Dependencies

• Firewalls, content filters and spam filters must allow the Pearson domains

• Minimum Screen Resolution of 1024 x 768 is required• Browser Dependencies:

– Adobe Flash Player version 10 and higher– Java runtime plugin version 1.5 and higher– Windows Firewall configured to allow javaw.exe to communicate– Allow pop-ups for Pearson sites– Accept Java applet– Allow Local File access to home directory

• Complete requirements can be found at http://pearsononlinetesting.com/TestNav/7/

Note: Pearson continually monitors and evaluates the hardware and software requirements. As vendors release newer versions of their products, we update the recommended requirements accordingly.


http://pearsononlinetesting.com/TestNav/7/

Best Practices and Recommendations

• Performance:– Use Proctor Caching for best network performance.– Use System Check extensively to test Proctor Caching and

network environment before live testing.– Use System Check to test wireless network capacity if using

wireless clients.• Testing Computers:

– Use System Check (TestNav Verification Test) to validate that TestNav will load on machines.

– Freeze browser, Adobe Flash Player, & Java environment if possible.

– Energy Saving features can interrupt testing, so the recommendation is to set those to not trigger during the length of the test.



• Testing Computers:– Applications that launch automatically

Any applications that may automatically launch on a computer should be configured not to launch during testing sessions. Common applications that may launch automatically include:

– Anti-virus software performing automatic updates – Power management software on laptops warning of low

batteries – Screen Savers– Email with Auto message notification


For wireless testing:

• Wireless access points – more is better• Distance – closer is better• Limit the number of computers per wireless access point• Try a small number first for proof of concept

– Do not roll out a few hundred on the first attempt (baby steps)• Limit obstructions

• Use the System Check Utility to verify that you have the appropriate amount of bandwidth for online testing



TestNav Directories

• TestNav creates a Pearson directory in the home directory of the testing computer:– Windows XP C:\Documents and Settings\username\Pearson\...– Windows Vista and 7 C:\Users\username\Pearson\...– Mac //Users/username/pearson/...

• Student Log files:– Pearson/student/logs

• SRF file default location:– Pearson/student/srf

• TestNav and test content is temporarily stored locally:– Pearson/temp

• Pearson recommends that the Pearson directory be excluded if desktop restoration tools such as DeepFreeze are used


Student Responses

TestNav will save student responses to an encrypted backup file so the student can either continue testing or exit the system without losing response data.

– TestNav requires at least one working save location– TestNav by default saves student response data to:

<home dir>/Pearson/student/srf– Create a TestNav Config to specify custom Primary and Alternate

file locations Pearson recommends designating two Response File Save Locations Best Practice: Primary Response File Location should be on the

student workstation. The alternate Response File Location should be on a shared network location

– Students must have full access to designated Response File Locations


• Response file will be saved to both the individual student’s testing workstation and a network location

• Student may resume on any workstation that has access to the shared network save location

41

Student Response Files (SRF)SRF file configured for network and local save locations

Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Student Response Files (SRF)SRF file configured for local save locations only

• All response files will save only to each individual student’s testing workstation

• Student must resume on the same workstation or SRF files must be moved to the workstation where the student will resume the test

• If the SRF files are not placed in the same location on the new workstation TestNav will prompt the student to navigate to where the SRF files are saved when they resume testing


• Response files will save to one or more network locations• Student may resume on any workstation that has access to the

shared network save location(s)

43

Student Response Files (SRF)SRF file configured for network save location(s) only


Using PearsonAccess to Configure Proctor Caching

Proctor Caching Takeaways

• How do I use PearsonAccess to Configure TestNav for Proctor Caching Setup?

• Section Resources– TestNav 7 Proctor Caching User Guide– Proctor Caching media training

Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved. 45Steps 9

Using PearsonAccess to Configure Proctor Caching


Providing Support during Testing: Early Warning System& Quick Reference Guide

Early Warning System Takeaways

• What is the Early Warning System?• What are the TestNav Early Warning System Triggers?• How do I resolve Early Warning System alerts?

• Section Resources– Early Warning System User Guide


Early Warning System (EWS)

The Early Warning System (EWS) is integrated functionality that gives TestNav a high degree of fault tolerance and provides additional fail-safes in the event of unexpected network disruptions during online testing.


Early Warning System Design

• Minimal interruptions for students• Multiple physical file backups to safeguard against data loss• Notification Screens designed for Test Proctors• Early Warning System notifications often involve IT staff so an

understanding is necessary to correct any issues during testing


Student Responses – SRF Files

• The EWS writes continuously in the background to the student response files. Both the Primary and Alternate files are written to at the same time.

• The SRF file has a response data threshold that once reached will trigger TestNav to send response data to Pearson.

• Uploading of response data is continuous. If an upload fails TestNav will cycle and attempt another upload.

• If the response data upload is successful, TestNav will create a new empty SRF file and begin the process again. TestNav will only delete an SRF file once it is successfully uploaded.


Student Responses - Identification

• A combination of the test ticket ID combined with the session token is used to uniquely identify a Student Response File (SRF).

• TestNav can identify the correct student response file if a test is successfully resumed.

• The SRF can only be used the next time the student resumes a test.


TestNav Early Warning System - Triggers

Scenario 1:TestNav determines that neither Saving Response File Location is

viable.– Will result in immediate Test Proctor Click Here notification.

Scenario 2:TestNav is unable to download portions of the test.

– Will result in immediate Test Proctor Click Here notification.

Scenario 3:TestNav is unable to upload student responses to Pearson.

– Will result in Test Proctor Click Here notification at the time of test submission.


TestNav EWS – Test Proctor Click Here Screens

One of the following screens will appear when scenario 1, 2 or 3 occurs.

Students should ALWAYS raise their hand when presented witheither of the Test Proctor Click Here screens. They should NEVER click the Test Proctor-Click Here button.


TestNav EWS – Scenario 1

In this scenario there is an issue with both of the Response File Locations.

Either the student does not have full access to the location(s), OR the location(s) are temporarily unavailable.


Resolve by entering and saving a viable Response File Location. If the issue is temporary, reselect the original location. Then, click the Save and Continue button.

Best Practice: Contact your local Technology Coordinator to determine the appropriate course of action.


In this scenario, TestNav cannot access the test content on the student work station.

If Proctor Caching is being utilized, either the Proctor Caching Server is not functioning properly, OR the local network and Pearson are not communicating.


The ONLY way to resolve this situation is to click the “Close TestNav” button. The student’s testing status will remain as STARTED in PearsonAccess™.

Once communication is restored, Resume the student in PearsonAccess and have the student log back into TestNav.

Best Practice: Your local Technology Coordinator should be contacted to determine the appropriate course of action. TestNav makes several attempts to access the test content. This message indicates all attempts to retrieve test content have failed.


In this scenario not all of the student responses have been received by the Pearson testing server. However, responses not received by Pearson have been saved locally to the Primary Response File Location.


You may try to resolve this scenario by clicking the Retry button until the submission of responses is successful or click on Close TestNav

Best Practice: If clicking the Retry button does not resolve the situation, contact your local Technology Coordinator to determine the appropriate course of action.

TestNav makes several attempts to communicate with the Pearson Testing Server. This message indicates attempts to submit some or all of the student responses have failed.

Quick Reference Guide

• The Quick Reference Guide should be available to all test proctors to help support students as they test.

• The Guide provides a list of all the warning messages that TestNav may show a student and how to resolve them.

• Available through PearsonAccess.com->Colorado->Support->Reference Guides->Online Test Administrator Quick Reference Guide


New Technologies: Virtualization & Online Testing

Virtualization Takeaways

• What is virtualization?• What are the concerns with virtualization and online testing?


• What is it?– Using your monitor/keyboard/mouse to look at and run software

on some other computer

– “Thin Clients” are one type of virtualization– VDI (Virtual Desktop Infrastructure) is becoming popular as a

way of turning aging computers into terminals, instead of retiring them

– There are almost as many different solutions as there are solutions providers

Virtualization

• Virtualization often appeals to schools looking to lower the student to computer ratio within limited IT budgets

• Virtualization presents unique security considerations when used for high-stakes online testing– Schools & districts may already be using virtualization

technology without Department of Education knowledge

Virtualization


Virtualization Network

• Virtualized environment runs on a central server and streams the user interface to workstations


Virtualization Security Concerns


Virtualization Concerns • Virtualization security settings on the server are typically unable to

be accessed or controlled by the test delivery engine• Streamed data between the virtualization server and workstations

may not be encrypted and could be intercepted by “man in the middle” exploits– Secure test content– Student keystrokes and interactions with TestNav– Student responses

• Virtualization on student workstations may not properly lock down the environment and provide a secure online testing environment– Unable to prevent student use of other applications– Unable to prevent capturing screen and test content– Lack of TestNav visibility to virtualization “modal” windows

• Virtualization performance issues could negatively impact students’ ability to test

VDI – Keeping Aging Computers In Service



A Worst-Case VDI Scenario


TestNav and Virtualization

• Pearson is working with individual virtualization vendors to qualify their solutions for use with TestNav

• Approved virtualization solutions will be listed on: www.TestNavQualified.com

• Non-approved virtualization solutions may expose online testing to exposure of secure content

http://www.TestNavQualified.com/

New Technologies:Emerging Devices & Online Testing

Takeaways

• What technology changes are taking place in schools?• What does comparability mean in online testing?


Online Testing & Technology Innovation

Technology Markets – Always Changing– Innovation drives continual change in

technology markets– As technology evolves, the testing

environment also changes Incremental updates: OS’s,

browsers, Flash, Java, HTML New hardware & devices:

netbooks, tablets, smartphones New network architectures: thin

clients, virtualization, cloud computing

New human/computer interfaces: mouse, touch, speech recognition, motion-controlled interaction (eg, Wii, Xbox, Kinect)

– A transition to technology-based assessments is not a one-time change

Diversity in the School Technology Environment

Comparability & “Standardized” Testing

Comparability between Paper and Online Tests

– Decades of research looking at differences in experience — and impact to student performance — has resulted in:

Scaled score adjustments Content & design changes Certain technology requirements

– Typically has treated online as the same experience for all students


New Comparability Research Focus

Comparability between Devices– Differences between devices are

becoming more pronounced– New research is beginning to

look at differences within the “online” condition

Areas of Research– Display size– Text input – keyboards vs. on-

screen typing– Interface – mouse vs. touch

Cursor-based feedback Control & fine-point precision Ability to see through/around

cursorCopyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Display Size


Text Input

Mouse- & Touch-based Interfaces

Managing “Ongoing Change” in Testing

Supporting Research– Only way to know comparability impact is

through research studies and field tests– Expect that technology adoption will always

be slowest in the realm of “high-stakes” testing

Communication Always Helps– Don’t hesitate to contact Pearson with

questions– Please share what is happening with

technology in your schools– Help educate your local technology staff

about the critical connection between technology at the student level and statewide comparability


Resources


Documents and Links

• DTC Academy: https://sites.google.com/a/elearningbydesign.net/cde-dtc-field-test/home or http://tinyurl.com/goDTCs

• CDE DTC webpage: http://goo.gl/1sM6QC• PearsonAccess Colorado Support Page

– System Check User Guide– System Check tool link– Hardware and Software Requirements link for TestNav 7– Hardware and Software Requirements link for Proctor Caching– TestNav 7 Technology Guidelines– TestNav 7 Proctor Caching User Guide– Proctor Caching media training– Early Warning System User Guide

https://sites.google.com/a/elearningbydesign.net/cde-dtc-field-test/home

https://sites.google.com/a/elearningbydesign.net/cde-dtc-field-test/home

http://tinyurl.com/goDTCs

http://goo.gl/1sM6QC


Support Contacts

• Collin Bonner– CDE Assessment Technology Specialist– 303-866-6752– [email protected]

• Pearson Support– 888-687-4759– [email protected]– Districts/Schools should use Pearson as the first contact for

support

Questions & Answers?

CDE Site Readiness Training Evaluation 2013

Please take a moment to complete the evaluation of this training:

http://goo.gl/e74GLS or

http://tinyurl.com/CDEeval

http://goo.gl/e74GLS

http://tinyurl.com/CDEeval

Colorado Online Testing Site Readiness Fall Field Test 2013.

Documents

Transcript of Colorado Online Testing Site Readiness Fall Field Test 2013.