CMAS Colorado Measures of Academic Success

Colorado Measures of Academic Success (CMAS)Training for SACs

Training for SACs and admins so they can train their proctors1AgendaPearson AccessOnline test management

SAC ResponsibilitiesCommunicationSchedulingTest Security

Proctor TrainingAccommodations

We will discuss each of these in more detail, but up front wanted you to have a list to reference2Windows, grades, content areasAssessment WindowApril 14-May 2, 2014

Science grades 5 and 8Social Studies grades 4 and 7

3FrameworksThe frameworks list Percent representation and number of score points for each of the reporting categories/standard areas that will appear on the summative assessmentsEvidence Outcomes eligible for inclusionTopics explicitly identified in the Social Studies standards are legitimate content for the assessment in addition to the standards' concept and skills

http://www.cde.state.co.us/assessment/newassess-sum Changes..Online Assessment vs Paper PencilWill be a shift for teachers, proctors, students, parents, administrators-communicate regularly Using Pearson Access to manage your testingAccessibility features vs AccommodationsLearning tech readiness for you and your buildingMore rigorous, higher DOKThis will be different and new, and some ideas of where those difference lie that we will cover. We will cover all of this today and anything not addressed please let me know. 5Similarities Test security is important for valid and reliable scoresEthical administration and procedures and ensuring standardized conditionsScheduling will have some hurdles just as when we first scheduled TCAP Active Proctoring is a must

So there are some similarities in big concepts. Like test security is still a consideration, it is the underlying work that will change to make sure they are secure. Scheduling is obviously still important, but there will be changes to how we schedule because it is an online assessment.6Each grade level and content area consists of three sections

The assessments are composed of the following item types:Selected ResponseConstructed ResponseComputer-enabled/technology-enhancedSimulations (science)Performance Events (social studies)Test StructureTechnology enhanced items, 2 types: drag and drop and hot spot3/13/20147What are the Responsibilities of School Assessment Coordinators?

So, lets get into what the responsibilities of the SAC are with this new assessment8SAC Responsibilities includeCommunicationSchedulingTest Security and Ethics

TechnologyPearson AccessProctor TrainingOnline Test prep

Again, big picture responsibilities are the same for the most part with the notable addition of technology and pearson access9Understand all CMAS procedures and requirementsEmploy school level security planServe as liaison between DACs and Test AdministratorsRead all communications from DAC and distribute as appropriateNEW for CMAS: during testing, school level personnel will have direct communication with Pearson

Key SAC Responsibilities THROUGHOUT TestingParticipate in district trainings and review all manualsWorking with STC, create the testing schedule for your school, including accommodated computer-based and paper-based testing groups and make-up sessionsDetermine school specific procedures (make-up testing, communication plan)Determine school level security planProvide and ensure completion of all training by Test Administrators and anyone else you have authorized or expect to come in contact with secure materials or environmentsReceive, secure, and distribute all test materialsManage and return security and training compliance documents

Key SAC Responsibilities BEFORE TestingPearsonAccessSet up user accounts for all necessary personnelIdentify and maintain a list of students requiring the text-to-speech form and accommodations (paper-based and computer-based administrations).Verify that PearsonAccess accounts for students needing:Paper forms of the assessment, including braille and large printOral scripts (paper and online)Request from your DAC additional secure and non-secure materials as neededAssign all students to a test sessionAssign students requiring special forms for the computer-based assessment to the appropriate formRegister/add new students

Key SAC Responsibilities BEFORE Testing-PearsonAccessEstablish the testing environments

PROCTOR CACHE!!!-simply choose the correct proctor caching server from the down menu when creating test sessions in pearsonaccess

Prepare studentsMeet with students to review school testing policies and expectations, including that test results should be reflective of individual effort.Ensure all students are familiar with TestNav 8.0ePats!Key SAC Responsibilities BEFORE TestingCharters will have to proctor cache themselves but other schools district IT is doing the actual proctor caching, they just have to choose the correct proctor caching server from the drop down13Ensure that all students are assessed in accordance with CMAS policies and proceduresRequest additional secure and non-secure materials from DAC as neededMONITOR TESTING ACTIVITYDistribute and collect paper testing materials dailyReport suspected testing irregularities (misadministrations, security breaches, technology glitches). Work with DAC to investigate and declare misadministrations.Account for the assessment of all students and ensure that all students have completed testing. Administer make-up sections as needed.

Key SAC Responsibilities DURING TestingPearsonAccess:Account for the assessment of all students and ensure that all students have completed testingVerify all testing sessions have been completedVerify that PearsonAccess has been updated to reflect the accommodations used during testing, including students taking paper-based assessmentsTranscribe student responses from large print and braille test books into scannable test books. Collect, sort, secure, and return all test materials to DAC by district determined dateManage, submit and retain compliance documents

Key SAC Responsibilities AFTER TestingSAC Responsibilities:Communication16CommunicationLead contact for questions in their building Always available to school proctors during all testing sessions in case of tech or test administration question or concernLead communication with building technology staffBe part of communication with building tech staff and district ITBuilding communication to DAC

Who to contact with questionsTest administrationProctor to SACSAC to DACDAC to CDE

Tech questionsBuilding tech to district IT-inform SAC of any tech issues so they are aware

Assessment Technology

CDEThe only time SACs or proctors would contact Pearson directly is during an actual test session if there is a tech issue call Pearson directly and tell them you are in a live testing session and they will put you the front of the line/prioritize your call and needs. Or for login issues, or issues navigating the pearson access system, ie how to create a session, how to add a new student, etc19SAC responsibilities-SchedulingDefinitionsPearsonAccess Test Session: Virtual data grouping of students who are grouped together within PearsonAccess for test management purposes. Each PearsonAccess test session consists of one grade level and one content area (three sections). Students must be placed in a scheduled PearsonAccess test session before they can log in to a test.

Test Section: The parts of a CMAS: Science and Social Studies assessment. Each grade and content area of the CMAS: Science and Social Studies assessments include three sealed test sections.

Physical Testing Group: Students who test within a physical testing environment at the same time. Need to know these before we can talk scheduling21SchedulingCreate school wide testing scheduleincluding accommodations and make ups

Proctor scheduling: Suggest proctor and tech savvy person both be in room so if students are stuck with tech issues proctor isnt stuck with one student

Does not specifically need to the tech person, but someone who is tech savvy22Each grade level and content area consists of 3 sections

Test sections can only be administered to students in consecutive order

One grade level and one content area at a time except for make-ups

To the extent possible, all students within a grade should be assessed at the same time. Should that not prove possible, all students should be assessed within the shortest timeframe practicable (i.e., in back-to-back sections, within the same day or consecutive days if more sections are needed than can be administered in a day, etc.)

CMAS Scheduling Considerations23Scheduling considerations contTechnologyNumber of compatible devicesBandwidthWireless access pointsHow many concurrent testing sessions can you run?-use the site readiness toolExtra computers

Drag and drop on laptop vs ipad vs desktop computerFine motor considerations?External Mouse available? Vs touch pad

May not want to schedule younger students to work on ipads or laptops if you think there may be fine motor skill deficiency in using drag and drop? Just something to consider24Section Timing90 minutes per sectionIncludes 10 min. for login, directions, sample items80 minutes of actual test time for every student (required)55 minutes minimum required3 sections in Science3 sections in Social Studies

1.5 hours includes time for directions and practice items, only need to schedule more for students with double time25CMAS: Test Structure and TimingCMAS: Science and Social Studies Section Administration Time for Elementary and Middle SchoolTotal Section Time

Time for Set up and Transition-approximately-may take more or lessRequired Testing Time for All Students

RequiredAdditional Testing Time for all students

90 minutes10 minutes55 minutes25 minutes

26Start up TestNav 8.0Log inTest transfer (proctor cache)Give directionsReview sample itemsTime for Set up and Transition Includes.

Best Practices-Recommendations2 extra devices in each testing roomMinimum: Two proctors per each testing roomNo more than 2 test sections per day (this year)Do not start first thing in the morning-make up sessionStudents remain in room for at least 55 minutes, up to 80 minutes if at least one student is still working

Strongly suggest that you do no dismiss students once they are done, all must remain in testing environment. Suggest that they always have at least one or two computers extra in case something goes wrong a student needs to move to another computer. Strongly suggest that they have two proctors in the room- one for questions/tech issues and one to maintain active proctoring.

Required: Students can read when done (school choice if allowing reading), but may not write, not work on computer or electronic device of any kind.

28Schedule first group/first day with more of a time cushion than other groups if you can

If you have to do multiple sessions, schedule smallest group first

Plan for accommodations

Recommendation: Schedule students for only one section per day (exception: make-ups)

Remember: This is Colorado

RecommendationsSchool CMAS schedulesEmail to Kim Terry no later than March 31st Also send security of testing materials form

Priority: minimize risk of prior exposure to test content that could result in an invalidationMake-Up Testing

Student completes missed test section during make-up testing time in advance of re-joining original physical testing groupMake-Up Testing Sample Make-Up Testing Schedule Day 1Day 2Day 3Regularly Scheduled Make-Up Testing Time (beginning of morning)-Absent Student Takes Section 1-Physical Testing Group Testing Time (scheduled after initial morning make-up testing time)Physical testing group takes test section 1Absent Student re-joins physical testing group for test section 2Physical testing group takes test section 3All absent students can be in one session and proctor (s) will have o give all dif student the dif directions and sample items (only the first section has sample items). Proctor (s) must also ensure each absent student uses the correct Seal Code to get into the test and erase seal codes after a student uses them, do not leave them up on the board. Proctor will need all absent students authorization tickets and must return all when done. Do not have to move students in Pearson Access just do NOT stop any pearson access test session until the end of the testing window.33How do device and facility availability affect scheduling?

What other impacts to scheduling do you see?

How does make-up testing affect scheduling?How to schedule for students who miss mutiple days?

Scheduling Discussion Guiding Questions3/13/201434CMAS or CoAltCSAP-A decision not based on:Category of disabilityTime in special educationProgram placement

Must be documented by IEP team

Contact Barb Rice @ x21931 if you need clarification35

CoAlt remains paper based, proctors will enter the scores online rather than on a rating form. Training dates for this are: 35SAC responsibilityTest and materials securityTest SecurityTesting environment-screen security and number of computersHow far apart to sit studentsCan students view from front to back the rows of screens in front of them?Student view side to sideSt authorization ticketsStudent pre exposure to items

37Test securityNo cell phones in testing roomCollect (baggies with names) so students do not have them during testingNo student tablets allowed in testing roomIpads are allowed for TEST ADMINISTRATORS onlyNo recordingNo pictures of anything in test environment ever

Be careful of collecting phones that you dont lose them or have another student take someone elses phone because you open yourself up to liability if that happens. No photos allowed, not of error messages, not by proctors, not by anyone.38Security of Student Authorization tickets-one approachSAC print the day before the test Put all into an envelope labeled for that testMaybe with a class list on the outside of it, All used tickets will be collected by the proctor and put back in that envelopeHave another empty envelope labeled MAKE UP TESTING REQUIRED Proctors put any unused authorization tickets

Show samples of manila envelopes for this. Then proctors return both envelopes to SAC. To training bring sample envelopes to show as visual example39Chain of CustodyCount in/out and test administrator sign for number of student authorization tickets each timePaper formsOral scriptsScratch Paper (official CMAS scratch paper, not any other paper)Secure central storage locationNot stored in classrooms

CMAS scratch paper is secure.40Maintaining Security of CMAS

All secure test materials must be secured while in the Test Administrators possession No duplication of secure CMAS materials is permissibleNo cell phones or other communication, reproduction or recording devices are allowed in the classroom

41Unauthorized Visitors and the MediaOnly students, Test Administrators/Examiners, and authorized school, district, state personnel or state-sanctioned test monitors may be in testing areas during administration

Media are not allowed to have access to the tests before, during, or after test administration, or take pictures or video of testing materials or testing students42Accessibility Features and AccommodationsOverviewTestNav 8 Embedded features: available to all students to increase accessibility

Embedded accessibility features that must be requested ahead of time: available to all students to increase accessibility

Additional embedded and non-embedded accommodations available to students consistent with their IEP, 504, or English learner plansIncreased AccessibilityBullet 2: SAC must go in and turn on prior to the test and after theyve created the test session in pearsonaccess44Tool BarNavigationBack pageNext pageReviewFlagPointerHighlighter (Caution: in development)Ruler (cm in 5th and 8th grade, inch in all grades)Can also be used as line guideCalculator (4 function for 5th grade, scientific for 8th grade)Eliminate answer choiceEmbedded Features for All Students Drop DownMagnifier

ZoomDependent on device

Writing tools for constructed responseBold, italics, and underlineBullet and numberCut, copy, and pasteUndo and redoEmbedded Features for All Students Selected by formText-to-speech with volume controlContinue to playClick to hear recommend students are taught to default to thisColor contrast (accommodation)Text-to-speech with color contrast (accommodation)Oral script (accommodation)Samplers for text-to-speech and text-to-speech with color contrast will be available in the second week of February

Embedded Accessibility Features and Accommodations that Must Be Requested Ahead of Time for Computer-based AdministrationSACs will collect a list from their schools of which students need these. Can just use TCAP accommodations list. Numbers 2, 3 and 4 above are accommodations which means only for students with IEP/504 or ELL plan. Item 1 is available for any student, be sure to use site readiness tool with the tab for text to speech to see how many you can run at once at your school, be sure to turn on this feature for students by going in to pearsonaccess and selecting it for the students who will have it.47Must be based on an individual need documented in the students approved IEP, 504, or English learner plan

A master list of all students and their accommodations must be maintained by the SAC. These will need to be recorded in PearsonAccess. Please include students using text-to-speech.

Accommodations are available for computer-based and paper-based administration.Paper-based form is itself intended to be an accommodated form

AccommodationsPaper Pencil or Online?Paper-based Assessments: A paper-based form of the assessments will only be available for students who: Havea neurological disorder, a condition that causes seizures or another health condition that prevents the student from accessing the computer Need abraille form with tactile graphics IEP or 504 plan requires assistive technology, such as speech-to-text, that may not be compatible with the test engine. www.pearsononlinetesting.com/at Will provide written responses in a language other than English or Spanish

Order by March 14th in pearsonaccess49PresentationEnglish oral script used for:On-site translations into languages other than Spanish*On-site sign language presentation*

Spanish oral script

Color contrast, including high contrast and reverse contrast

External cover overlays

Word-to-word dictionaries (English/native language)

Assistive technology devices compatibility will need to be established

*Due to the number, length and complexity of sources in the Social Studies assessment, it is recommended that this accommodation be provided individually when practicable.

Accommodations for Computer-based Administration50ResponseWritten responses in Spanish

Scribe (must be individually administered)EnglishOther languages

Assistive technology devices compatibility will need to be established

Accommodations for Computer-based AdministrationSettingSmall groupIndividualTiming Extended time greater than time and a half Time of day Multiple breaks (requires separate setting)

Accommodations for Computer-based Administration52Devices with demonstrated compatibility with TestNav8:

www.pearsononlinetesting.com/at

Assistive TechnologyThe paper-based form of the assessment is itself considered an accommodation.

Use of the paper-based assessments is expected only to be available to students who:Have a neurological disorder, a condition that causes seizures, or another health condition that prevents the student from accessing the computer*

Need a braille form with tactile graphics

Have a IEP or 504 plan that requires assistive technology, such as speech-to-text, that may not be compatible with TestNav 8

Will provide written responses in a language other than English or Spanish

*Students with disabilities that prevent them from utilizing devices safely may meet this criterion.Accommodations for Paper-based AdministrationPresentationBraille with tactile graphicsLarge print (Reminder: computer-based has zoom and magnifier)External cover overlaysEnglish oral script (text-to-speech in computer-based)*On-site translations into languages other than SpanishOn-site sign language presentationSpanish oral script (also computer-based)*Word-to-word dictionary (English/native language)

*Due to the number, length and complexity of sources in the Social Studies assessment, it is recommended that this accommodation be provided individually when practicable.

Accommodations for Paper-based Administration55ResponseBraille note-takerScribe (also for computer-based)Must be administered individuallySpeech-to-textAssistive technology devices (word prediction)Setting (same as computer-based)Small groupIndividualTiming (same as computer-based)Extended time greater than time and a half Time of day Multiple breaks (requires separate setting)

Accommodations for Paper-based AdministrationSchool Technology ResponsibilitiesMight be two different people, SAC and tech, or might be one person, depends on school57School techJava updatesshortcut or LAN school set up using login given by IT that locks out other programs, puts test nav screen on every computerSite readiness tool so # of devices to run at one time is known http://systemcheck.co.testnav.com/systemCheck.jsp?acc=co_chk

Refer to district IT tech training for more informationSystem checkhttp://systemcheck.co.testnav.com/ If you have the specifications for which browsers and versions work with which operating systems, etc you may not need this toolResults from system check-Failed

System check-Passed

Without proctor caching- 26 students

With proctor caching

With proctor caching- 466 students

Test Administrator TrainingAdministering the testSACs can pull these next slides out and use them to train their test administrators. Anyone who will be in the room while testing is happening and will talk to students must be trained. Otherwise they shouldnt be in the room or talk to students. 65SAC ResponsibilitiesAdministration Training Read Procedures Manual in its entirety after todays training and prior to training your buildingA face to face proctor training is required by you to your building proctors prior to the start of the testing windowUse the Proctor Training ppt. provided, it includes ethics and test administrationKeep all signed confidentiality agreements onsite for all of your trained CMAS proctors6666Individual at school level responsible for directly administering the CMAS: Science and Social Studies assessmentsMust be employed by the school/districtBe trained and have read the Test Administrators Manual prior to any assessment administrationBe trained in any specific accommodations they are providingMust sign security agreement before assessment administration beginsRead the directions to students exactly as they appear in the Test Administrators ManualTest session management (school decision)Administer sections actively

CMAS Test AdministratorActive AdministrationActive Administrators:Administrators May Not:

Provide feedbackClarify test questionsAnswer content related questionsInterfere with the students demonstration of skillsInteract with students in any way that would impact student responsesEngage in other tasks during test sectionsRead sources, items, or student responsesUse any electronics during testing (other than to monitor in pearson access if sac deems it necessary)Ensure they have all necessary materials for each sessionEnsure a standardized testing environmentFollow all scripts exactly as writtenMove throughout the room during testingRead directions to students when askedUse proximity to keep students on task68Prepare the test environmentPick up student authorization tickets/sign out from SACMake sure all students from list are present or marked absent on envelope with absent students tickets put into the absent envelopeStart session in PearsonAccess-or SAC can do-school decisionFollow directions and read script in the Test Administrators ManualHand out student authorization tickets and scratch paperAssist students in logging on to TestNavPut Seal Code on white board or screen

Administration StepsTired to put these in order of how they would actually need to complete things.

If seal code is written on chart paper, that paper is now secure materials so must be turned in. 69Administration Steps-contComplete directions to students and practice items if there are practice itemsActively administer the test sectionCollect student authorization tickets and scratch paperErase Seal Code from board completelyClose session in PearsonAccess after third section-SAC can do at end of test window because if any student absent/not marked complete will need to either take all sections or have the SAC mark complete in PearsonAccess

Test Administrators-helpful hints from field testAdministrators should not hand out authorization tickets until students are seated in the labHave students log in in wavesOnce students are logged in, proctors should direct them to put hands off of keyboard while directions are readDirections should be that students do not touch keyboard until proctor says so

Another good reason to have 2 proctors so that students do not begin or go past practice items 71Secure Materials- Test AdministratorsCollect back student authorization tickets immediately after students loginPut student authorization tickets into correct envelopesReturn envelopes to SACs immediately after testingSign in the number of student authorization tickets being returned15 minute rule of thumb for returning 72Test EnvironmentNo food or drink on desksNo content posted that would give students an advantagePut up Do Not Disturb signNo musicSame rules as TCAPBathroom-dont log students out or pause their tests just put something (blank) in front of screen so no one can see itNo one should be able to see the students screen anyways, but as a precaution, if a student leaves to go to the bathroom put something in front of their screen. It will not time them out.73Contact Pearson directly If a technology issue arises during an actual live test session test administrators should contact Pearson directly-after informing SAC of the issueThis is the only time test administrators need to/should contact Pearson directlyTell them you are in a live testing situation and they will bump you to the front of the linePearson: 1-888-687-4759Testing IrregularitiesFill out form online if any testing irregularityMisadministrationsSecurity breachesTechnology glitchesEmergencies/unforeseen circumstancesMisadministrationsStudents moving onto next sectionReally only able to do if in a make up section and they have access to the seal code to the next section

Student receives help from anyone on a test itemStudents uses a cell phone, calculator, electronic deviceStudent is not given accommodations they requireSAC: call Kim Terry if any misadministration has occurred

Breach of secure test materialsDiscussing, reproducing, transmitting by any means any secure test materialsStudent obtaining or sharing secure test materialsSecure test materials include:Student authorization ticketsOfficial CMAS test scratch paperSeal codesEmergenciesStudent becomes illLog that student out-directions on next screenRecord amount of time left in sectionRecord onto the absent student list that the student left and the amount of testing time left To allow ill student to finish later

79EmergenciesEmergency requiring the test to be stopped:Safety first, evaluate and respond to the emergencyIf possible without risk to students, note time left in sessionStudents can have time left in session

When students are doneStudents all remain in room until all students are done -recommendedStudents must be allowed to have 55 minutes minimum-requiredIf all students are done before 55 minutes, all students must remain in room for 55 minutes and then can all be dismissed-requiredAll students are allowed 80 minutes, so if any student is still working after 55 minutes, ALL students must remain in testing room until ALL students are done or 80 min. has passed-recommended81How students submit their testsThey should choose that they are finished and want to submit their final answersThis means they cannot go back into that sectionThey can practice this step in the epats alsoSee pdffor how students submit tests for directions for sections 1, 2 and 3.82Technology issues

Lets acknowledge that the potential is there, so what plans does your school have in place to deal with potential issues? I recommend seating with leaving 2 open computers, having an extra proctor in the room. Dont schedule too tightly this first time. Schedule your smallest group first if possible.Who to contact? What to prepare in advance?83E PatsPractice TestsThe epats will be available Feb. 27th . The teacher guide that goes with them is highly recommended to use. It purposefully walks teachers and students through the use of the tools and accessibility features.84Test preparation for online assessmentsePATs ePat guides for proctors are so helpfulFamiliarize students with tools, give opportunity to usePractice with scrolling, drag and drop items, simulations

Schedule time for students to use the epats to become familiar with tools and features so they arent using test time to discover these.The best test prep is everyday teaching that follows the pacing guides/supports the standards. But we understand the want to have students familiar with the format of online assessments.85ePats: Practice Testswww.pearsonaccess.com/co Important for students to practice with tools so they arent overly distracted by them at test time.86Receiving and Returning Test Materials87Online and PaperTest Administrators ManualScratch PaperCMAS Secure Return EnvelopeSchool Security ChecklistOnline OnlyStudent Authorization TicketsEnglish and Spanish Oral Scripts for Online FormOnsite Translations into Languages other than English/SpanishPaper OnlyTest Books, including Braille and Large PrintSource Book (Social Studies Only)English and Spanish Oral Scripts for Paper FormOnsite Translations into Languages other than English/SpanishTest Materials88Receiving Test MaterialsMaterials will arrive via district warehouse-date tbd

Purple labels on all boxesGreen labels on CMAS boxesPink labels on CoAlt boxes

Keep your boxes to return in. All st. auth. Tickets will need returned and accounted for also.89Receiving Test MaterialsSAC to verify materials received with the School Packing List

90SACsCollect all secure materials and group as follows:Secure return envelopesScorable test booksUnused test booksAccommodated materialsElectronic materials must be removed from any devices after testing and verification sent to the DACPrepare materials for packagingCheck and deliver CMAS scorable test books to the DAC Check and deliver CMAS non-scorable materialsto the DACReturn materials with original boxes

Returning Test Materials-SACs

A return schedule will be sent3/13/201491

Questions92