EAMT Workshop 2015 - KantanMT
-
Upload
kantanmt -
Category
Data & Analytics
-
view
109 -
download
2
Transcript of EAMT Workshop 2015 - KantanMT
Computational Challenge of SMT
Language Model
Translation Model
Optimal Search
• Bi-Lingual Data
• Glossary• NTAs• Stock Engines
• Mono-lingual Data
Types of Training
Bi-Lingual Data
Terminology
Mono-lingual Data
Training Data
* Translation Memories
* PDFs, DOCX, TXT
* TBX, XLSX
Stock Training Data
* 5 Billion Words available
Factors to Consider
• Training Data - Three main factors:Quality
• The linguistic quality of the training material is crucially important
Relevance to domain• A high quality MT system has good domain
knowledge• Similar to the way you’ve always worked with
Translation Memories and CAT tools
Quantity• The more training data you use to build your
engine the better its capacity to generate translations that mimic your translation style and terminology
• F-Measure - Recall & Precision
Automated Measurements
Reference Translation
MT Output
Precision
correct
MT-Len
66%
Recall
correct
Ref-Len
80%
F-Measure
Precision * Recall
(Precision + Recall) /2
73%
• BLEU Score• Put simply – measures how many words overlap, giving
higher scores to sequential words
• High correlation between BLEU and human judgement of translation quality
Automated Measurements
Reference Translation
MT Output
Automated Measurements
• TER (Translation Error Rate)• Min number of edits to transform output to match reference
• Levenshtein distance measure• General indicator of Post-Editing Effort
Reference Translation
MT Output
TER
Substitutions + insertions + deletions
Reference-length
Comparative Measurements
• F-Measure Score• Recall & Precision calculation
• Closely linked to the relevancy of word selection for MT systems
Kantan BuildAnalytics™
Comparative Measurements
• TER Score• A method to help predict the post-editing effort
• TER is quick to use and correlates highly with actual post-editing effort
Kantan BuildAnalytics™
Comparative Measurements
• BLEU Score• Improvement upon F-Measure
• Takes word-order into consideration
• Linked to a sense of translation ‘fluency’
Kantan BuildAnalytics™
Comparative Measurements
Kantan BuildAnalytics™
• Useful for • Engine Development
• Baseline measurements
• Determination of ‘possible’ engine quality and relevancy
• Reference set of comparative translations required
• Does not work on unseen translations
• Of limited use in determining • PE effort
• Resources
• Costs