Criteria for Setting MT Project Unit Prices

TOIN Corporation Takatsugu Kato

Masaru Kawahara

Factors that Determine MT Fees

Objective for Using MTLevel of PEMT Quality

Decide: Objective for

using translation?

Targetdocument

Assess based on MT evaluation criteria

Can your objective be served by MT

alone?

MT unit price(Level 1)

Assess based on Light PE evaluation criteria

Light or

Full?

Assess based on Full PE evaluation criteria

Light PE unit price

(Level 5)

Full PE unit price

(Level 5)

Yes

No

Decide What Your Objective Is I want instant translations and/or to minimize fees MT only I want as close to human translation quality as possible MT+PE

MT Translation QualityMT quality is inconsistent, it diers according to various factors. Corpus volume Quality of source les Type of MT engine etc.

Perform sample translation and evaluate MT translation

Without evaluating MT... If MT output is good quality without PE, then PE is a useless extra cost Doing PE on low-quality MT translations increases the burden on Post-editors etc.

With a proper evaluation... You get MT+PE at a reasonable unit price A reasonable unit price keeps the client, the LSP, and Post-editors happy! etc.

Evaluating Quality when Using MT Alone With MT alone there is no human intervention, so unit price is the same However, to decide whether MT translation alone will serve your purpose requires an evaluation

How to Evaluate MT Output

Criterion DescriptionGood - MT translation is perfectly

understandable as-is- Some mistakes (roughly, up to 1 per 10 words), but captures the core meaning of the source

Poor Anything other than Good

MT evaluation criteria

Apply the following criteria to each segment, and divide them into Good and Poor. Then evaluate the percentage of Good.

Number of Segments to Evaluate Evaluating the entire target document takes too much time, so use basic statistical techniques to get samples, and calculate the number of segments

n = N/[(/())2*{(N-1)/(1-)}+1]No. of parameters (N)Precision ()Reliability (())Statistical percentage ()

Parameters Samples100 80

200 132

500 217

1000 278

5000 357

10000 370

100000 382

Set PE LevelLevel of post-editing Quality criteriaLight PE Accuracy

Full PE Accuracy+Readability+Style

There are two guidelines for PE level

PE Guidelines

PE evaluation criteriaCriterion DescriptionGood - MT translation can be used as-is,

without editing- Following items need small edits (roughly, up to 1 per 10 words):Word order (not including editing of particles/usage), placement, additions, deletions

Poor Anything other than Good

How to Evaluate MT Output Same as MT alone However, evaluation criteria are dierent

Variety of Evaluation CriteriaUse dierent evaluation criteria to suit your ultimate objective for using MT

When using PE

Use PE evaluation criteria Quantify how much eort it will take for PE Sliding unit price depending on evaluation result (Level 5*2)

When using MT alone Use MT evaluation criteria

What level of quality does MT output represent as-is?

Why We Use Separate Evaluation Criteria PE evaluation criteria measure how much eort will be required for PE with MT translation MT evaluation criteria measure how accurate and uent MT translation is

Each of these "rulers" is dierent

Concrete ExampleSource: You will then be able to use business rule sets and rules to validate that the objects are valid to be processed.

Correct translation:

MT translation PE criterion MT criterion

Just requires changing to armative sentence to match the blue text:Good

The meaning is opposite, so:Poor

Table of Unit Prices Set the unit price according to the percentage of Good in the target document (where HT is 100%)

% of Good Light PE Full PE71% or higher 30% 60%

61% to 70% 40% 70%

51% to 60% 50% 80%

31% to 50% 60% 90%

30% or lower 70% 100%

TOIN Corporation Takatsugu Kato

Masaru Kawahara