Computer Based TestingComputer Based Testing Security & AdministrationSecurity & Administration2007 North American Pesticide Applicator Certification & Safety Education Workshop

       Portland Maine 2007     

CBT Security & AdministrationCBT Security & Administration10 Minute10 Minute ReviewReview

Metro Institute BackgroundMetro Institute Background

Testing Security BasicsTesting Security Basics

CBT System OverviewCBT System Overview

Step-by-Step ProcedureStep-by-Step Procedure

System ReportsSystem Reports

Concluding Statements & QuestionsConcluding Statements & Questions

Metro Institute BackgroundMetro Institute Background

Training Institute - 2000Training Institute - 2000– Initial Applicator Training & CEU’sInitial Applicator Training & CEU’s

Computer Based Testing – 2003Computer Based Testing – 2003– 3GB Technologies @ USU Research Center 3GB Technologies @ USU Research Center – Began Providing CBT for Arizona SPCCBegan Providing CBT for Arizona SPCC

Present Goals – 2007Present Goals – 2007– CBT & Web CBT & Web Training WorldwideTraining Worldwide– Implement CBT Nebraska, Alabama, N. Carolina, S. Implement CBT Nebraska, Alabama, N. Carolina, S.

CarolinaCarolina– National Pesticide Applicator Core Exam – CBT ConversionNational Pesticide Applicator Core Exam – CBT Conversion

Testing Security BasicsTesting Security Basics Server Physical SecurityServer Physical Security

– Hosted by Dedicated ISP ProviderHosted by Dedicated ISP Provider– Offsite Backup Performed Every HourOffsite Backup Performed Every Hour

Test Data SecurityTest Data Security– Dual Server System – Web & DataDual Server System – Web & Data– ISP Firewalls & Port Blocking & PasswordISP Firewalls & Port Blocking & Password– MD5 Login EncryptionMD5 Login Encryption– SSL Data Transfer EncryptionSSL Data Transfer Encryption– No ActiveX or Java Applets EmployedNo ActiveX or Java Applets Employed

Testing Security BasicsTesting Security Basics Agency Data SecurityAgency Data Security

– Test Applications to Metro via FTP Test Applications to Metro via FTP – Test Scores Returned to Agency Via FTPTest Scores Returned to Agency Via FTP

Testing Sites SecurityTesting Sites Security– Proctored & MonitoredProctored & Monitored– Photo ID Required, Photo TakenPhoto ID Required, Photo Taken

Testing Computer SecurityTesting Computer Security– Hardened Kiosk - Stores No Data EverHardened Kiosk - Stores No Data Ever

CBT System OverviewCBT System Overview

Step-by-Step ProcedureStep-by-Step Procedure

ApplicationApplication Server OperationsServer Operations Testing AppointmentTesting Appointment Arrival at Testing SiteArrival at Testing Site Accessing Test QuestionsAccessing Test Questions Testing ProcessTesting Process Test CompletionTest Completion Final Data Delivery to AgencyFinal Data Delivery to Agency

ApplicationApplication

Application CompleteApplication Complete

Reviewed & Reviewed & ApprovedApproved

Agency Data EntryAgency Data Entry

Letter to ApplicatorLetter to Applicator

Cued for Daily FTPCued for Daily FTP– Name & ID #Name & ID #– Date ApprovedDate Approved– Tests ApprovedTests Approved

Server OperationsServer Operations

Web Server solicits Web Server solicits & receives data & receives data from Agency Serverfrom Agency Server

Web Server submits Web Server submits password to Data password to Data Server for transferServer for transfer

Data transferred Data transferred from Web Server to from Web Server to Data Server thru Data Server thru firewallfirewall

Testing AppointmentTesting Appointment

Applicant calls Applicant calls Metro with ID Metro with ID numbernumber

Metro queries Web Metro queries Web Server for authority Server for authority to offer testto offer test

Approval confirmed Approval confirmed & test set for & test set for specific date & site specific date & site thru Web Serverthru Web Server

Arrival at Testing Site Arrival at Testing Site

Photo ID is requested & Photo ID is requested & identification confirmedidentification confirmed

Digital photo taken & Digital photo taken & sent to test-taker filesent to test-taker file

Test-taker proctored Test-taker proctored into specific test, into specific test, computer IP address computer IP address confirmedconfirmed

Test-taker gets color Test-taker gets color scratch paper, enters scratch paper, enters ID#, agrees to terms, ID#, agrees to terms, begins testingbegins testing

Accessing Test QuestionsAccessing Test Questions Six events must be Six events must be

completed in completed in sequence to receive sequence to receive test question:test question:

– Application ReceivedApplication Received– Test AppointmentTest Appointment– Day & LocationDay & Location– Proctor ID’s, Photos & Proctor ID’s, Photos &

Releases TestReleases Test– IP Address ConfirmedIP Address Confirmed– Proctor Signs in test-Proctor Signs in test-

taker to workstationtaker to workstation

Testing ProcessTesting Process

Test questions/answers Test questions/answers randomly generated & randomly generated & sent in multiple packetssent in multiple packets

Packets may contain Packets may contain one or two wordsone or two words

Packets may or may not Packets may or may not take same routetake same route

Answers return to Web Answers return to Web Server as single Server as single numbernumber

Test CompletionTest Completion

Tester may review Tester may review questions & answers questions & answers until timeoutuntil timeout

Once “Finish” button Once “Finish” button selected (or timeout) selected (or timeout) test is recordedtest is recorded

Completed test can Completed test can not be alterednot be altered

Score sheet is printedScore sheet is printed

100%100%

Final Data Delivery to Final Data Delivery to AgencyAgency

Test-taker scores & Test-taker scores & pictures sent from pictures sent from Metro Web Server Metro Web Server to Agency File to Agency File Server via FTP Server via FTP dailydaily

Agency issues Agency issues License to all test-License to all test-takers with passing takers with passing scoresscores

ReportsReports Histogram – Score DistributionHistogram – Score Distribution

Discrimination IndexDiscrimination Index

Test Question – Sort by % CorrectTest Question – Sort by % Correct

Test Results – Sort by LocationTest Results – Sort by Location

Average Time to Complete TestAverage Time to Complete Test

Applicator Test Look-upApplicator Test Look-up

Applicator Photo Look-upApplicator Photo Look-up

Test Questions & AnswersTest Questions & Answers

HistogramHistogram

Discrimination IndexDiscrimination Index

Test Question % CorrectTest Question % Correct

In ConclusionIn Conclusion Metro Online CBT System Attributes:Metro Online CBT System Attributes:

– More SecureMore Secure - Paper-based & computer - Paper-based & computer transmitted testing systems vulnerabletransmitted testing systems vulnerable

– Allows FlexibilityAllows Flexibility - Timely question changes - Timely question changes

– Secure & ProvenSecure & Proven - Procedures & protocols - Procedures & protocols taking advantage of latest technologytaking advantage of latest technology

– ReportsReports – Any analysis provided – Any analysis provided

– Free to AgencyFree to Agency - Saves money allowing - Saves money allowing staff & office space reduction or staff & office space reduction or reassignmentreassignment