Translation Error Rate Metric
-
Upload
westbrook-sterling -
Category
Documents
-
view
15 -
download
0
description
Transcript of Translation Error Rate Metric
![Page 1: Translation Error Rate Metric](https://reader036.fdocuments.in/reader036/viewer/2022071807/56812e03550346895d936699/html5/thumbnails/1.jpg)
March 10, 2005 METEOR 1
Translation Error Rate Metric
• Reference: “the Iraqi weapons are to be handed over to the army within two weeks”
• MT output: “in two weeks Iraq’s weapons will give army”
![Page 2: Translation Error Rate Metric](https://reader036.fdocuments.in/reader036/viewer/2022071807/56812e03550346895d936699/html5/thumbnails/2.jpg)
March 10, 2005 METEOR 2
Step-1: Remove Stop Words
• the Iraqi weapons are to be handed over to the army within two weeks
• in two weeks Iraq’s weapons will give army
![Page 3: Translation Error Rate Metric](https://reader036.fdocuments.in/reader036/viewer/2022071807/56812e03550346895d936699/html5/thumbnails/3.jpg)
March 10, 2005 METEOR 3
Step-2: Word Align with Matcher
• the Iraqi weapons are to be handed over to the army within two weeks
• in two weeks Iraq’s weapons will give army
![Page 4: Translation Error Rate Metric](https://reader036.fdocuments.in/reader036/viewer/2022071807/56812e03550346895d936699/html5/thumbnails/4.jpg)
March 10, 2005 METEOR 4
Step-3: Count Flips (Reorder)
• the Iraqi weapons are to be handed over to the army within two weeks
• in two weeks Iraq’s weapons will give army
• Reordered:• in Iraq’s weapons will give army two weeks• Flip count = 4
![Page 5: Translation Error Rate Metric](https://reader036.fdocuments.in/reader036/viewer/2022071807/56812e03550346895d936699/html5/thumbnails/5.jpg)
March 10, 2005 METEOR 5
Step-4: Count Insertions/Deletions/Substitutions
• the Iraqi weapons are to be handed over to the army within two weeks
• in two weeks Iraq’s weapons will give army
• Reordered:• in Iraq’s weapons will give army two weeks• Flip count = 4• Substitutions: 2• Insertions (from Ref): 5• Deletions (from MT): 1
![Page 6: Translation Error Rate Metric](https://reader036.fdocuments.in/reader036/viewer/2022071807/56812e03550346895d936699/html5/thumbnails/6.jpg)
March 10, 2005 METEOR 6
Step-5: Score
• the Iraqi weapons are to be handed over to the army within two weeks
• in two weeks Iraq’s weapons will give army
• Reordered:• in Iraq’s weapons will give army two weeks• Flip count = 4• Substitutions: 2• Insertions (from Ref): 5• Deletions (from MT): 1• TER = flips + subst + ins + del / ref length =
4+2+5+1/12 = 12/12 = 1.0
![Page 7: Translation Error Rate Metric](https://reader036.fdocuments.in/reader036/viewer/2022071807/56812e03550346895d936699/html5/thumbnails/7.jpg)
March 10, 2005 METEOR 7
Another Example
• Ref: north korea is prepared to allow washington to verify that it does not manufacture nuclear weapons
• MT: North Korea prepared to allow Washington to the verification of that is to manufacture nuclear weapons
• Flips: 0• Substitutions: 3• Insertions: 1• Deletions: 0• TER Score = 0+3+1+0/16 = 4/16 = 0.25