NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016:...
Transcript of NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016:...
![Page 1: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/1.jpg)
NEAS Conference 2016:
Understanding & Engaging With Computer-Based Testing Scoring Engines Louise FitzGerald
![Page 2: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/2.jpg)
Presentation overview
1. Automated testing
2. PTE Academic - selected items
3. Scoring and scoring engines
4. Questions
![Page 3: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/3.jpg)
Pedagogic disruption
![Page 4: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/4.jpg)
High stakes tests – Study and immigration
![Page 5: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/5.jpg)
Automated testing
![Page 6: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/6.jpg)
PTE Academic
![Page 7: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/7.jpg)
PTE Academic - Item types
Speaking and Writing
Personal introduction • Read aloud • Repeat sentence • Describe image • Re-tell lecture • Answer short question • Summarize written text • Write essay
Reading
• Multiple-choice, choose single answer • Multiple-choice, choose multiple answers • Re-order paragraphs • Reading: Fill in the blanks • Reading & writing: Fill in the blanks
Listening
• Summarize spoken text • Multiple choice, choose multiple
answers • Fill in the blanks • Highlight correct summary • Multiple choice, choose single answer • Select missing word • Highlight incorrect words • Write from dictation
![Page 8: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/8.jpg)
PTE Academic – Scored practice tests
![Page 9: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/9.jpg)
PTE Academic – Scored practice tests Offer
![Page 10: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/10.jpg)
[S] Item 3: Describe image
No. of tasks: 6-7 Preparation time: 25 seconds Speaking time: 40 seconds
![Page 11: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/11.jpg)
Scoring – Describe image
Enabling Skills and Traits
Content 5 Describes all elements of the image and their relationships, possible developments, conclusions or implications 4 Describes all the key elements of the image and their relations, referring to their implications or conclusions 3 Deals with most key elements of the image and refers to their implications or conclusions 2 Deals with only one key element in the image and refers to an implication or conclusion. Shows basic understanding of several core elements of the image 1 Describes some basic elements of the image but does not make clear their interrelations or implications 0 Mentions some disjointed elements of the presentation
Pronunciation 5 Native-like, 4 Advanced, 3 Good, 2 Intermediate, 1 Intrusive, 0 Non-English
Fluency 5 Native-like, 4 Advanced, 3 Good, 2 Intermediate, 1 Limited, 0 Disfluent
Communicative Skills: Speaking
Enabling Skills and Traits
Content 5 Describes all elements of the image and their relationships, possible developments, conclusions or implications 4 Describes all the key elements of the image and their relations, referring to their implications or conclusions 3 Deals with most key elements of the image and refers to their implications or conclusions 2 Deals with only one key element in the image and refers to an implication or conclusion. Shows basic understanding of several core elements of the image 1 Describes some basic elements of the image but does not make clear their interrelations or implications 0 Mentions some disjointed elements of the presentation
Pronunciation 5 Native-like, 4 Advanced, 3 Good, 2 Intermediate, 1 Intrusive, 0 Non-English
Fluency 5 Native-like, 4 Advanced, 3 Good, 2 Intermediate, 1 Limited, 0 Disfluent
![Page 12: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/12.jpg)
[W] Item 6: Summarize written text
No. of tasks: 2-3 Length of text: Up to 300 words Response word limit: 5-75 words Writing time: 10 minutes
![Page 13: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/13.jpg)
Scoring – Summarize written text
Enabling Skills and Traits
Content 2 Provides a good summary of the text. All relevant aspects mentioned 1 Provides a fair summary of the text but misses one or two aspects 0 Omits or misrepresents the main aspects of the text
Formal requirements
1 Is written in one, single, complete sentence 0 Not written in one, single, complete sentence or contains less than 5 or more than 75 words. Summary is written in capital letters
Grammar
2 Has correct grammatical structure 1 Contains grammatical errors with no hindrance to communication 0 Has defective grammatical structure which could hinder communication
Vocabulary 2 Has appropriate choice of words 1 Contains lexical errors with no hindrance to communication 0 Has defective word choice which could hinder communication
Communicative Skills: Reading and Writing
![Page 14: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/14.jpg)
Scoring – Summarize written text
Enabling Skills and Traits
Content 2 Provides a good summary of the text. All relevant aspects mentioned 1 Provides a fair summary of the text but misses one or two aspects 0 Omits or misrepresents the main aspects of the text
Formal requirements
1 Is written in one, single, complete sentence 0 Not written in one, single, complete sentence or contains less than 5 or more than 75 words. Summary is written in capital letters.
Grammar
2 Has correct grammatical structure 1 Contains grammatical errors with no hindrance to communication 0 Has defective grammatical structure which could hinder communication
Vocabulary 2 Has appropriate choice of words 1 Contains lexical errors with no hindrance to communication 0 Has defective word choice which could hinder communication
Communicative Skills: Reading and Writing
![Page 15: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/15.jpg)
Summarize written text
Test takers need to use just one compound or, preferably, complex sentence to summarize the main point and briefly mention the supporting detail.
Test takers need to check they have only written one sentence of between 5 and 75 words, with a capital letter at the beginning and only one full stop at the end.
1. ‘Just-in-time’ production relates goods produced to demand for goods and reduces the amount of wasted time, costs and materials. (19 words)
2. ‘Just-in-time’ manufacturing minimises production time and costs and stored stock is kept low because each stage is finished just before the next one is started. (25 words)
3. ‘Just-in-time’ is a method of manufacturing, which minimises time, costs and stock kept in factories, as each stage of production is completed before the next is started. (27 words)
![Page 16: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/16.jpg)
[W] Item 7: Write essay
No. of tasks: 1-2 Input text : Up to 4 sentences Word limit: 200-300 words Writing time: 20 minutes
![Page 17: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/17.jpg)
[W] Item 7: Write essay
Subskills tested
• Writing for a purpose (to learn, to inform, to persuade)
• supporting an opinion with details, examples and explanations
• organizing sentences and paragraphs in a logical way
• developing complex ideas within a complete essay
• using words and phrases appropriate to the context
• using correct grammar
• using correct spelling
• using correct mechanics
• writing under timed conditions
Scoring
Communication skill:
Writing Enabling skills:
Content
Form
Development, structure and coherence
Grammar
General linguistic range
Vocabulary range
Spelling
![Page 18: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/18.jpg)
Write essay
Students need to analyze the essay prompt so they know how to answer the question. Note key words, limiting words and task words, e.g. discuss.
Practise producing plans for a variety of persuasive essay topics.
Make sure they’ve practiced typing within the time and word limits. In the test, they’ll lose marks if they don’t!
Make sure your students know to leave a few minutes at the end to edit their work. If they know their common errors, they can just concentrate on those. They won’t have time to much more than that.
![Page 19: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/19.jpg)
[L] Item 19: Highlight incorrect words
No. of tasks: 2-3 Length of recording: 15-50 seconds No. of errors: up to 7
![Page 20: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/20.jpg)
Scoring – Highlight incorrect words
Communicative Skills: Listening and Reading
Partial credit scoring * Points deducted for incorrect options chosen 1 Each correct word -1 Each incorrect word 0 Minimum score
![Page 21: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/21.jpg)
Scoring and scoring engines
![Page 22: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/22.jpg)
Score alignment with IELTS
IELTS PTE Academic
4.5 30
5 36
5.5 42
6 50
6.5 58
7 65
7.5 73
8 79
![Page 23: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/23.jpg)
Score alignment with TOEFL iBT
![Page 24: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/24.jpg)
Item Scoring
PTE Academic Score Guide
http://pearsonpte.com/test-takers/results/
![Page 25: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/25.jpg)
Item Scoring Example: Write essay
+ Content (if 0, no further scoring)
+ Form (if 0, no further scoring) + Other traits
Enabling skills scores
+ Vocabulary Vocabulary
+ Spelling Spelling
+ Grammar Grammar
+ Development, structure and coherence Written discourse
+ General linguistic range
= Total item score
Communicative skills score: Writing
Overall score
![Page 26: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/26.jpg)
Scoring and Results
Automated scoring: objective and reliable
![Page 27: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/27.jpg)
Automated Scoring: Writing YouTube - Automated Scoring FAQs: how the machines are trained
![Page 28: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/28.jpg)
Automated Scoring: Writing YouTube – Latent Semantic Analysis
![Page 29: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/29.jpg)
Automated Scoring: Speaking YouTube - Automated Scoring FAQs: how the system is trained
![Page 30: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/30.jpg)
Fascinating Facts: Scoring Engines No.2 Individual,
discrete features of a response can
be analysed. independently.
No.1 The system is not
‘distracted’ by language-
irrelevant factors, such as poor handwriting.
No.3 The scoring engine
for writing is trained to score any traits that humans can
reliably score.
No.4 The scoring engine has been trained for
every new item on the test by human raters before it goes live.
No.5 All standard
conventions for spelling are
accepted, but test takes must use the same convention throughout their
response.
![Page 31: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/31.jpg)
Automated scoring
The automated system acts like a human rater when assessing test takers’ language skills, but does so with the precision, consistency, and objectivity of a machine.
![Page 32: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/32.jpg)
Automated scoring
For more information on Automated Scoring: White paper: http://pearsonpte.com/institutions/scores/ YouTube: https://www.youtube.com/user/PearsonPTE/featured
![Page 33: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/33.jpg)
Official Guide to PTE Academic with CD-ROM
![Page 34: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/34.jpg)
References https://www.ets.org/Media/Tests/TOEFL/pdf/10855_ellguide.pdf
https://www.ets.org/s/toefl/pdf/toefl_ibt_insight_s1v6.pdf https://www.ets.org/Media/Tests/TOEFL/pdf/SampleQuestions.pdf https://www.ets.org/toefl/ibt/scores/get/ https://www.ets.org/Media/Tests/TOEFL/pdf/SampleQuestions.pdf https://www.ets.org/s/toefl/pdf/toefl_ibt_research_insight.pdf http://www.hagitegas.gr/2013/05/%CE%B7-%CF%80%CF%81%CE%BF%CF%83%CE%B5%CF%85%CF%87%CE%AE-%CF%84%CF%89%CE%BD-%CE%B5%CE%BE%CE%B5%CF%84%CE%AC%CF%83%CE%B5%CF%89%CE%BD/ http://www.startupsmart.com.au/advice/growth/why-disruption-is-about-experiences-not-technology/#.Vx72QHMU8v0.linkedin
![Page 35: NEAS Conference 2016: Understanding & Engaging With ... · PDF fileNEAS Conference 2016: Understanding & Engaging With Computer- ... PTE Academic – Scored practice tests Offer [S]](https://reader033.fdocuments.in/reader033/viewer/2022052709/5a7944087f8b9a0b118c55d6/html5/thumbnails/35.jpg)
35
Questions?
Learn more at future webinars and PTEA teacher training events.