Caveon Webinar Series: What you Should Know about High Stakes Cheating in Your Schools
Caveon Webinar Series - Lessons Learned at the European Association of Test Publishers Annual...
-
Upload
caveon-test-security -
Category
Education
-
view
150 -
download
3
description
Transcript of Caveon Webinar Series - Lessons Learned at the European Association of Test Publishers Annual...
![Page 1: Caveon Webinar Series - Lessons Learned at the European Association of Test Publishers Annual Conference and The Conference on Test Security October 2014](https://reader033.fdocuments.in/reader033/viewer/2022060121/5594323e1a28abba7e8b45f7/html5/thumbnails/1.jpg)
Caveon Webinar Series
1
Lessons Learned at
The Conference on Test Security
& the European Association of
Test Publishers Annual
ConferenceRachel SchoenigAssistant Vice President,
Head of Test Security
ACT
John Fremer
President
Caveon Consulting Services
www.caveon.com
October 16, 2014
![Page 2: Caveon Webinar Series - Lessons Learned at the European Association of Test Publishers Annual Conference and The Conference on Test Security October 2014](https://reader033.fdocuments.in/reader033/viewer/2022060121/5594323e1a28abba7e8b45f7/html5/thumbnails/2.jpg)
Agenda for Today
• EATP
• Sample Session John Fremer
• COTS
• What is COTS? Rachel Schoenig
• John Fremer Takeaways
• A Test Security Framework, Rachel Schoenig
• Q&A
2www.caveon.com
![Page 3: Caveon Webinar Series - Lessons Learned at the European Association of Test Publishers Annual Conference and The Conference on Test Security October 2014](https://reader033.fdocuments.in/reader033/viewer/2022060121/5594323e1a28abba7e8b45f7/html5/thumbnails/3.jpg)
Overview of EATP
3www.caveon.com
![Page 4: Caveon Webinar Series - Lessons Learned at the European Association of Test Publishers Annual Conference and The Conference on Test Security October 2014](https://reader033.fdocuments.in/reader033/viewer/2022060121/5594323e1a28abba7e8b45f7/html5/thumbnails/4.jpg)
EATP
• Annual Meeting of European Testing
Professionals and Others
• 2014 Theme – “Growing Talent in Europe:
Gaining Advantage Through Assessment”
• Primary Focus is Industrial/Organizational
Psychology
• Role of Test Security is being Increasingly
Recognized
4www.caveon.com
![Page 5: Caveon Webinar Series - Lessons Learned at the European Association of Test Publishers Annual Conference and The Conference on Test Security October 2014](https://reader033.fdocuments.in/reader033/viewer/2022060121/5594323e1a28abba7e8b45f7/html5/thumbnails/5.jpg)
EATP Session – Learning from the
Past
• Presenters:
– Steve Addicott – Caveon
– Ardeshir Geranpayeh – Cambridge Assessment
– John Fremer – Caveon
• Lessons drawn from Many Centuries
– “The more things change, the more they remain
the same”
5www.caveon.com
![Page 6: Caveon Webinar Series - Lessons Learned at the European Association of Test Publishers Annual Conference and The Conference on Test Security October 2014](https://reader033.fdocuments.in/reader033/viewer/2022060121/5594323e1a28abba7e8b45f7/html5/thumbnails/6.jpg)
Learning from the Past (cont)
• Cheating on the Imperial China Selection Exam could lead to Execution
– This helped keep cheating to “only” about 60 per cent of participants
• In the Ancient Olympic Games, Cheating Athletes
– Fined
– Monies used to build statues of Zeus
– Each statue - offense, athlete’s name and city
6www.caveon.com
![Page 7: Caveon Webinar Series - Lessons Learned at the European Association of Test Publishers Annual Conference and The Conference on Test Security October 2014](https://reader033.fdocuments.in/reader033/viewer/2022060121/5594323e1a28abba7e8b45f7/html5/thumbnails/7.jpg)
Learning from the Past (cont)
• As is the case now, cheating was discouraged
and punished
• Rules about Testing were Disseminated
• Special Efforts Devoted to Proctoring
• Some Cheating Tools seem “Modern,” e.g. a
cheating cloak containing impermissible info
• Publicity about Misbehavior Used as a
Deterrent
7www.caveon.com
![Page 8: Caveon Webinar Series - Lessons Learned at the European Association of Test Publishers Annual Conference and The Conference on Test Security October 2014](https://reader033.fdocuments.in/reader033/viewer/2022060121/5594323e1a28abba7e8b45f7/html5/thumbnails/8.jpg)
Session: The Future of Test Security
8
• Presenters– Fremer and Addicott
– Phil Dickison, NCSBN
• Key issues– Technology Based Testing
– Growth in Testing International
Career Readiness
– Privacy
www.caveon.com
![Page 9: Caveon Webinar Series - Lessons Learned at the European Association of Test Publishers Annual Conference and The Conference on Test Security October 2014](https://reader033.fdocuments.in/reader033/viewer/2022060121/5594323e1a28abba7e8b45f7/html5/thumbnails/9.jpg)
Security – The Validity Paradigm
The New Paradigm
Security
9www.caveon.com
![Page 10: Caveon Webinar Series - Lessons Learned at the European Association of Test Publishers Annual Conference and The Conference on Test Security October 2014](https://reader033.fdocuments.in/reader033/viewer/2022060121/5594323e1a28abba7e8b45f7/html5/thumbnails/10.jpg)
The Conference on Test Security
10www.caveon.com
![Page 11: Caveon Webinar Series - Lessons Learned at the European Association of Test Publishers Annual Conference and The Conference on Test Security October 2014](https://reader033.fdocuments.in/reader033/viewer/2022060121/5594323e1a28abba7e8b45f7/html5/thumbnails/11.jpg)
11
• Presenters
Wayne Camara, ACT – the New Standards
Ray Nicosia, ETS – Management of a Major
Test Security Unit
Phil Dickison, NCSBN – Focus on Test
Validity
John Fremer, Caveon – Growth of our
Profession and Industry
Plenary – How has our approach to
Test Security Evolved
www.caveon.com
![Page 12: Caveon Webinar Series - Lessons Learned at the European Association of Test Publishers Annual Conference and The Conference on Test Security October 2014](https://reader033.fdocuments.in/reader033/viewer/2022060121/5594323e1a28abba7e8b45f7/html5/thumbnails/12.jpg)
COTS – Working a Case
Best Practices in Test Security investigations
• Presenters:
– A. Benjamin Mannes – ABIM
– Rachel Watkins Schoenig – ACT
– Marc Weinstein – Dilworth Paxton
– John Fremer – Caveon
12www.caveon.com
![Page 13: Caveon Webinar Series - Lessons Learned at the European Association of Test Publishers Annual Conference and The Conference on Test Security October 2014](https://reader033.fdocuments.in/reader033/viewer/2022060121/5594323e1a28abba7e8b45f7/html5/thumbnails/13.jpg)
The Call….
• Parent called the State DOE and Test Publisher hotline
• 7th grade daughter said her teacher gave them answers to questions on their year-end statewide assessment tests
• Another teacher was present in the room when this occurred
The Case:
13www.caveon.com
![Page 14: Caveon Webinar Series - Lessons Learned at the European Association of Test Publishers Annual Conference and The Conference on Test Security October 2014](https://reader033.fdocuments.in/reader033/viewer/2022060121/5594323e1a28abba7e8b45f7/html5/thumbnails/14.jpg)
Consider what outcome is desired:
• Ensuring score validity
• Deterring and stopping inappropriate future
behavior
• Maintaining public confidence in test sponsor
• Determining individual culpability and/or
accountability
• Taking action from employment, education,
certification and/or licensure perspective
Case – Investigative Goals?
14www.caveon.com
![Page 15: Caveon Webinar Series - Lessons Learned at the European Association of Test Publishers Annual Conference and The Conference on Test Security October 2014](https://reader033.fdocuments.in/reader033/viewer/2022060121/5594323e1a28abba7e8b45f7/html5/thumbnails/15.jpg)
So You Flagged a Cheater Now
What
• Presenters
– Aimee Hobby Rhodes - CFA Institute
– Kellie Early - National Conference of Bar
Examiners
– A. Benjamin Mannes - ABIM
– Joy Matthews Lopez - NABP
• Panel Discussion of Key Questions
15www.caveon.com
![Page 16: Caveon Webinar Series - Lessons Learned at the European Association of Test Publishers Annual Conference and The Conference on Test Security October 2014](https://reader033.fdocuments.in/reader033/viewer/2022060121/5594323e1a28abba7e8b45f7/html5/thumbnails/16.jpg)
Flagging a Cheater (cont)
• Importance of Very Tight Candidate
Agreements
• Collect and Deliver Information in Writing
Only
• Not Enough to have very Good Data
Forensics Information – You have to
explain it (Like “how do you get to Carnegie
Hall – Practice, Practice, Practice”)
16www.caveon.com
![Page 17: Caveon Webinar Series - Lessons Learned at the European Association of Test Publishers Annual Conference and The Conference on Test Security October 2014](https://reader033.fdocuments.in/reader033/viewer/2022060121/5594323e1a28abba7e8b45f7/html5/thumbnails/17.jpg)
Certified Exam Security Professional
• Certified Exam Security Professional (CESP)
• A “certification in the making”
• Route to a Credential that recognizes Test
Security Expertise
• Series of Credentials Envisioned beginning
with “CESP Generalist”
• Curriculum Targets and Learning Materials
and References
18www.caveon.com
![Page 18: Caveon Webinar Series - Lessons Learned at the European Association of Test Publishers Annual Conference and The Conference on Test Security October 2014](https://reader033.fdocuments.in/reader033/viewer/2022060121/5594323e1a28abba7e8b45f7/html5/thumbnails/18.jpg)
Session: Protecting our Tests – the
Past, Present & Future of Test Security
19
• Presenters
– Steve Addicott
– Rachel Schoenig
• Takeaways
– Privacy
– Career Readiness
– New Designs
– Test Security Training & Development
www.caveon.com
![Page 19: Caveon Webinar Series - Lessons Learned at the European Association of Test Publishers Annual Conference and The Conference on Test Security October 2014](https://reader033.fdocuments.in/reader033/viewer/2022060121/5594323e1a28abba7e8b45f7/html5/thumbnails/19.jpg)
What is happening?
Testing is
GROWING…incredibly
20www.caveon.com
![Page 20: Caveon Webinar Series - Lessons Learned at the European Association of Test Publishers Annual Conference and The Conference on Test Security October 2014](https://reader033.fdocuments.in/reader033/viewer/2022060121/5594323e1a28abba7e8b45f7/html5/thumbnails/20.jpg)
Why? We Have the Gold Bars
21www.caveon.com
![Page 21: Caveon Webinar Series - Lessons Learned at the European Association of Test Publishers Annual Conference and The Conference on Test Security October 2014](https://reader033.fdocuments.in/reader033/viewer/2022060121/5594323e1a28abba7e8b45f7/html5/thumbnails/21.jpg)
Issues Across Industries
0
2
4
6
8
10
12
14
16
Music Sales
1999
2009
22www.caveon.com
![Page 22: Caveon Webinar Series - Lessons Learned at the European Association of Test Publishers Annual Conference and The Conference on Test Security October 2014](https://reader033.fdocuments.in/reader033/viewer/2022060121/5594323e1a28abba7e8b45f7/html5/thumbnails/22.jpg)
0
200
400
600
800
1000
1200
Counterfeit & Pirated Products
2008
2015
Issues Across Industries
ICC 2011 and OECD 2009 reports
23www.caveon.com
![Page 23: Caveon Webinar Series - Lessons Learned at the European Association of Test Publishers Annual Conference and The Conference on Test Security October 2014](https://reader033.fdocuments.in/reader033/viewer/2022060121/5594323e1a28abba7e8b45f7/html5/thumbnails/23.jpg)
Protecting our gold bars
24www.caveon.com
![Page 24: Caveon Webinar Series - Lessons Learned at the European Association of Test Publishers Annual Conference and The Conference on Test Security October 2014](https://reader033.fdocuments.in/reader033/viewer/2022060121/5594323e1a28abba7e8b45f7/html5/thumbnails/24.jpg)
Protecting our gold bars
Reputation
25www.caveon.com
![Page 25: Caveon Webinar Series - Lessons Learned at the European Association of Test Publishers Annual Conference and The Conference on Test Security October 2014](https://reader033.fdocuments.in/reader033/viewer/2022060121/5594323e1a28abba7e8b45f7/html5/thumbnails/25.jpg)
Holistic Test Security
Framework
How to Protect our Tests
26www.caveon.com
![Page 26: Caveon Webinar Series - Lessons Learned at the European Association of Test Publishers Annual Conference and The Conference on Test Security October 2014](https://reader033.fdocuments.in/reader033/viewer/2022060121/5594323e1a28abba7e8b45f7/html5/thumbnails/26.jpg)
Testing Process “Silos”
Test
Development
Test
Publication
Test
Delivery
Score &
Report
Insert Test Security Here
27
![Page 27: Caveon Webinar Series - Lessons Learned at the European Association of Test Publishers Annual Conference and The Conference on Test Security October 2014](https://reader033.fdocuments.in/reader033/viewer/2022060121/5594323e1a28abba7e8b45f7/html5/thumbnails/27.jpg)
Holistic Test Security Framework
Deter
DetectDecide (now and for the future)
28www.caveon.com
![Page 28: Caveon Webinar Series - Lessons Learned at the European Association of Test Publishers Annual Conference and The Conference on Test Security October 2014](https://reader033.fdocuments.in/reader033/viewer/2022060121/5594323e1a28abba7e8b45f7/html5/thumbnails/28.jpg)
How to Protect our Tests
Development
Publication
AdministrationScoring
ReportingDeter
Detect
Decide
29www.caveon.com
![Page 29: Caveon Webinar Series - Lessons Learned at the European Association of Test Publishers Annual Conference and The Conference on Test Security October 2014](https://reader033.fdocuments.in/reader033/viewer/2022060121/5594323e1a28abba7e8b45f7/html5/thumbnails/29.jpg)
Thank you!
30
Follow Caveon on twitter @caveon
Check out our blog www.caveon.com/blog
LinkedIn Group “Caveon Test Security”
Rachel SchoenigAssistant Vice President,
Head of Test Security
ACT
John FremerPresident, Caveon Consulting ServicesCaveon Test Security
www.caveon.com