Translation Quality -- Subjectivity and Hard Numbers

"Quality" is a much-abused word. In the language services industry, most quality assurance or quality control systems seem to be based on models from the manufacturing world and to depend heavily on strictly defined procedures and checklists. While these systems are good at isolating and preventing "errors" (in meaning, grammar, spelling, etc.), a translation can be "error-free" and still be of little value if it doesn't function well as a target-language document. But to a considerable extent, that suitability is determined by factors such as readability and conformity to (or effective divergence from) document-type conventions that are largely subjective and difficult to control through conventional QA techniques.

But subjective doesn't necessarily mean unquantifiable or unmanageable. A human reviewer can easily express his or her subjective assessment of a text's quality numerically and in multiple dimensions, and the numbers derived from these subjective assessments can then be put through the same kinds of rigorous processing and analysis as any other numerical data.

At Blue Danube we assess the quality of translation assignments with reference to four criteria:

• Source-language comprehension
  Translation accurately reflects the content of the source text


• Subject-matter expertise
  Correct and consistent use of terminology


• Target-language style
  Translation reads smoothly and naturally and is idiomatically appropriate for the document type


• Target-language technical proficiency
  Completeness, grammar, spelling, punctuation, formatting, fonts/charsets, untranslatables, etc.

Being subjective, any given assessment by one evaluator of one translation may be biased or skewed in any number of ways. The key to success is in continually evaluating virtually every translation and recording and monitoring the data for both translators and evaluators. If a translator gets a bad (or good) evaluation once, it may not mean much. But if the translator consistently gets high or low marks from a number of different evaluators, that's a pretty reliable indicator of his or her performance. Conversely, if an evaluator consistently gives a certain translator higher or lower marks than other evaluators, or scores one dimension more or less strictly, that's also information we can use.

The evaluation data becomes even more valuable when used as part of a comprehensive system of feedback loops for every level of the process.