Dr. Constanza Gerding Salas Based on Sonia Colina’s Translation Teaching (2003 )

Translation assessment

Universität Leipzig -Universidad de Concepción, May 2012

Dr. Constanza Gerding SalasBased on Sonia Colina’s Translation Teaching (2003)

Leipzig -Concepción, 2012

Evaluation vs. assessment (Colina, 2003)

Evaluation refers to the quality of the translation product regardless of the method and procedures used.

Assessment uses the translation product to determine the translator’s competence.

Assessment today


Qualitative statements can be made on the basis of a comparison between TT and ST, bearing in mind the brief and the situational context (House, 1997).

A communicative approach to the evaluation of translations should consider error assessment focused on the communicative function of the word, phrase or sentence in question (Kussmaul, 1995).

Evaluators should restrict themselves to the effect the error has on the target reader (Kussmaul, 1995).

Quality assessment


Summative vs. formative assessment (Hatim & Mason, 1997).

Summative assessment: to establish whether a translator (student) is fit to be accepted in a translation program, move on to the next level, practice as a professional translator, etc.

Formative assessment serves as an aid to the learning process by providing continuous feedback on the development of translator competence.

Purpose of evaluation


Achievement testing (formative) determines whether the goals and requirements specified in the syllabus have been met.

Proficiency testing (summative) measures ability in relation to that required for a particular course of action, e.g. work as a professional translator (Colina, 2003).

Achievement vs. proficiency testing


Testers should consider the following questions:1. What is translation competence?2. What am I looking for in assessing translation

competence?3. Can multiple translation sub-competences be

distinguished?

A working definition of competence must be established prior to test elaboration.

Teaching and testing materials must be designed on the basis of such definition (Colina, 2003).

Testing translation


Separate componential scoring criteria could be used for testing multi-componential translation skills (Bachman, 1990; Cao, 1996).

E.g. language competence criterion, knowledge structures criterion, translation strategies criterion, etc.

Assessment frequently shows these weaknesses:• No brief• No relationship between test and class practice• Test selection based on text difficulty only

Componential model


A good test should meet at least these four criteria (Carroll, 1980):

1. Economy: obtaining the maximum amount of information about student’s competence with a minimum of effort on part of the test designer.

2. Relevance: a match between the test and the curriculum and course goals.

3. Acceptability: the learners’ satisfaction that the test evaluates their progress.

4. Comparability: similar scores across learners for different tests for the same group of learners .

Test format


Does it provide a brief and/or pragmatic information? Does it include format specifications? Does the test have a stated purpose? Has it been designed accordingly? Does it specify what competences are to be tested? (Not

necessarily on the test itself). Is it economical, relevant, acceptable and comparable? Does it state clear grading criteria? Does it test what was taught and how it was taught?

Checklist for a quality translation test


Nord (2009: 236) proposes a definition of error in translation based upon the principles of translation difficulties and translation problems.

There are no standards for every translation case. In functional translation, the normative framework to assess quality is given by the translation brief.

Errors are identified by comparing the student’s TT with the profile of an ideal translatum derived from a brief and considering specific translation problems (Nord, 2009).

Errors in translation


• A translation error is any violation of the standards stipulated in the translation brief with respect to specific functional aspects (Nord, 2009).

• A translated text can only be evaluated in relation to a given translation aim. Translators must know that aim.

• A translation task must be feasible, i.e. in accordance with student’s knowledge level.

• Time restrictions and quality standards must correspond to task’s degree of difficulty (Nord, 2009).

Defining translation error


• Most translation problems are identified during pre-translation analysis, having situation, pragmatics, culturemes, and linguistic structures in mind.

• The horizon of expectations should consider only those translation problems whose solution is to be assessed.

• Each adequate solution is given a number of positive points. The addition of these points makes up the score that should cover the solution to all translation problems included in the horizon of expectations.

• The student’s mark should be calculated on the basis of the percentage of adequate solutions given (Nord, 2009).

Identifying errors (Nord, 2009)


According to Nord (2009), the degree of difficulty of a translation task is determined by:

• Textual difficulties: complexity y specificity of ST

• Technical difficulties: number and quality documentation sources available

• Professional difficulties: translation brief (TT functions, readers, means, perfection degree demanded for the TT

• Competential difficulties: students’ a) linguistic, cultural and translational level and b) general and specific knowledge

Factors determining difficulties (errors)


Nord (2009) proposes a broad error taxonomy based on the translation problem categories and the inadequate solution or the absence of a solution in relation to a given brief. The categories are listed in order of importance:

PROBLEM ERROR1. pragmatic PTP = PTE 2. cultural CTP = CTE 3. linguistic LTP = LTE 4. extraordinary ETP = ETE

Nord’s error categories


The weight of an error will depend on how much it affects the intelligibility of the TT in relation to the brief and the communicative situation.

According to Nord (2009), cultural errors do not always impede understanding but they only “hinder” communication.

Error weight


• Grading is not simple• Should pragmatic errors weigh more heavily?• How far-reaching is the mistake?• Does it distort meaning?• Does it hinder/destroy communication?

Pym (1992: 279) proposes ‘binary’ and ‘non-binary’ errors:• Binary error: clearly wrong choices• Non-binary errors: based on the choice from potential TT

variants (‘It’s correct, but…)

Both authors recommend the non-binary error approach

Factors affecting grading (Kussmaul, 1995)


1. Cultural adequacy2. Situational adequacy3. Speech acts (what was said & implied, & its effect)4. Meaning of words (misinterpretation of text)5. Language errors: Do they impede comprehension or

bring discredit on the author?6. ST defects. Can meaning be inferred?

Error category for evaluation (Kussmaul, 1995)


Colina (2003: 136) proposes a descriptive profile, i.e. a numerical score that is described according to rating criteria:

• 5 = TT reveals careful consideration of brief and very appropriate translation solutions.

• 3 = TT reveals consideration of brief and acceptable translation solutions.

• 1 = TT reveals no consideration of brief and poor orinconsistent translation solutions.

(Model does not provide explicit description of 2 and 4).

In componential rating, each skill tested is graded separately and final mark is obtained from weighted average.

Grading criteria


Colina (2003) proposes a 100-point grading criteria rubric based on five components:

Multi-componential grading criteria

For component descriptions, see Colina 2003: 137-138.


Hatim & Mason (2007) based their model on Bachman’s paradigm (1990).

It consists of two types of skills:

1. ST & TT processing skills (40 points)• Intertextuality & rhetorical structure (genre, text type)• Texture & structure (lexical choice, syntax, cohesion)• Situationality (context)

2. Transfer skills (60 points)• Effectiveness• Efficiency• Relevance

Translation competence grading criteria


A. Translation Language competence (30 pts.)1. Organizational: grammatical/textual2. Pragmatic

B. Knowledge structures (20 points)1. Cultural2. Topical

C. Strategic competence (50 points)

Criticism: this model assigns between 25% and 30% of the score per category even if competence is assessed as ‘deficient’ or ‘lacking’.

Translation competence grading criteria: Cao’s model (1996)


1. ST dependence/adequacy: 25 points2. TT adequacy: 25 points3. Vocabulary, specialized knowledge, research (15

points)4. Functional & textual aspects (25 points)5. Revision process (10 points)

Description of all categories given by author.

Colina’s scale for evaluating translation competence (2003: 143-144)


1. Find out the exact function or purpose of your translation2. Classify your translation problems3. Use the parallel-activity technique to remove mental blocks4. Remember there is divergent thinking and flexibility5. Verify if SC & TC coincide or diverge about that problem6. Use parallel texts to find out about TT conventions7. Do not stick to formal equivalents; be creative8. Use notions, pictures and experiences in your memory9. Make use of situational analysis * Informants are the best dictionaries

Useful strategies (Kussmaul, 1995: 150-152)


Student’s checklist Translation brief and text formalities Text types and genres Pragmatic factors in ST & TT Decision-making Research and documentation Parallel text analysis Translation • ST analysis and comprehension• Reformulation• Revision

Elaborating tests for the CTC


Sample Test

A. Instructions• Read the whole text before translating. • Type your name in full on the top right corner of a new blank document of

Word.• Save the document at once under your name, not “Test 5”.• Format: margins, font type and size, spacing and page size similar to those of

the original. • Fit the target text on one page only.• Time assigned for the whole task: 90 minutes (589-word text).• Draft translation should be finished in half the period so as to have time to

revise TT. • When ready, close the file and send it to [email protected] .• Do not include source text, instructions or brief in your document.


B. Commission• This editorial is a complete text unit• Translate the whole text into Spanish• The target text is intended to be published in El Mercurio, Santiago,

Chile, in the form of an opinion column. The text should be source-oriented.

C. Pragmatic information• Source: The Opinion Pages of The New York Times • Means: http://www.nytimes.com/2012/01/09/opinion/haitis-slow-recovery.html?scp=2&sq=editorial

%20haiti&st=cse

• Target readership: mainly Chilean adults• Type of readership: conservative, educated adult readers• Register: in accordance with readership and publication• Text function: informative, right-wing focus

Test text commission


• Cloze passages (open or multiple-choice) to supply the missing word(s)

• Brief writing• Précis writing• Short answer, multiple-choice or T/F questions on content

discussed in class, e.g.: What is an editorial? Who writes the editorial in a newspaper?

(Text type and function) What is the difference between ‘un editorial’ and ‘una

editorial’? (SL culture; language issue) Are there any differences between newspaper editorials in the

USA and in Chile? (Cross-culture) What are the main translation problems of editorials?

(Translation competence)

Other possible test items


Thank you!

Dr. Constanza Gerding Salas Based on Sonia Colina’s Translation Teaching (2003 )

Documents

Transcript of Dr. Constanza Gerding Salas Based on Sonia Colina’s Translation Teaching (2003 )