Best method to evaluate translations from others
Thread poster: rma (X)

rma (X)
Apr 19, 2010

Hi everybody,

Can you please tell me what are the best methods to evaluate the translations of others? Can you please give me some hints or some ideas in this direction?

Thank you all in advance.

Best regards,

Marco


 

Attila Piróth  Identity Verified
France
Local time: 07:03
Member
English to Hungarian
+ ...
Specifications Apr 19, 2010

The topic of translation quality assessment is extremely vast, so please be more specific. Do you want to commission translation quality assessment from someone and you need to specify your requirements? Or do you want to evaluate the quality of someone else's translation?

The conference session Editing and translation quality assessment, which was to be held in Birmingham last weekend (but had to be canceled because of the volcano eruption) would have provided a brief introduction into this topic. To establish a QA protocol is a lot of work, you need to clearly define the the precise requirements of the specific job, as QA is about ensuring fitness for purpose.

Kind regards,
Attila


 

rma (X)
TOPIC STARTER
Best method to evaluate translations from others Apr 19, 2010

Thanks for your reply Atilla.

here are some details to my topic:
I want a system or some criteria to evaluate/measure translations, so that when I receive a translation from another translator I am able to decide if the translation is good or bad. now, I know it's impossibile to measure language. if there are two or three translators they all may translate the same source differently and all alternatives may be good, but this is not necessary the case.
there must be a way or some criteria to evaluate the quality of translations. this is what I'm looking for: a way or some criteria (ideas to develop a system) to measure translations that I receive from colleagues in the branch.

kind regards,
marco

[Edited at 2010-04-19 16:13 GMT]


 

Katalin Horváth McClure  Identity Verified
United States
Local time: 01:03
Member (2002)
English to Hungarian
+ ...
Do you understand the languages involved? Apr 19, 2010

Another important thing to understand what you need is whether you are looking for a way for yourself to assess somebody's translation, or you need to provide this assessment to somebody else (who may not understand the source and/or the target language)?
Or, alternatively, you have reviewers that would do the assessment, but you want to be able to make sense of those evaluations, even if you don't understand the source and/or the target language?
This may be an important point when setting up your QA system.

If you do a general search on "translation quality measurement" or "translation quality metrics", you will get lots of useful information.

It is also important what type of translations you are trying to assess, as some of the standards are more specific to, or originated from one particular industry, such as the SAE J2450 for automotive translations:
http://www.sae.org/technicalcommittees/j2450p1.htm

The LISA QA model is more specific to localization:
http://www.lisa.org/LISA-QA-Model-3-1.124.0.html

I hope it helps
Katalin

[Edited at 2010-04-19 18:27 GMT]


 

Adi Al-Ka'bi
Local time: 10:03
Arabic to English
+ ...
Check their correspondence as a first indication Apr 19, 2010

If you are a native of language A and you want to check the level of a translator whose native language is B, get the second translator to correspond with you in your language A.

If he/she is good in your language then you don't need to know what they translate from or to in their native language B. If the correspondence isn't good then the translation will be at the same levels of the correspondence.

This won't suffice if the other translator is translating between languages B and C unless you are proficient in language C or of bilingual level between your language A and language C.

Alternatively or additionally you can copy-paste their correspondence into a word processor with a spelling and sentence formulation checker. If that comes up with loads of colored lines, the translator will be making a similar percentage in errors in what they translate. Of course anyone can apply the checkers and get rid of the colored lines to make it look flawless, but you have to make sure what you get makes sense and sounds logical also.

The above was how I initially set up my associations with other translators whose services I use when clients require other than my pair of languages. All my associates translate between English and their native language. It gets difficult to assess what is being presented when you know neither language B or C.


 

rma (X)
TOPIC STARTER
@ Katalin Apr 20, 2010

First of all thank you for your reply.
I want to implement a system to check someone else's translations. I forgot to say, that I do not speak all the other languages.
Your second point/question would be something very interesting to me:
Or, alternatively, you have reviewers that would do the assessment, but you want to be able to make sense of those evaluations, even if you don't understand the source and/or the target language?


Some guidlines for the reviewer so that the reviewer can give me a precise feedback, and not only a general "it's good" or "it's bad". I need something so that I know what is bad, what went wrong, what is good, why is it good/bad. to retranslate means a lot of lost time. This is what I want to avoid.

Thanks again for your reply and your linksicon_smile.gif

All the best,
Marco


 

rma (X)
TOPIC STARTER
@ Adi Al-Ka'bi Apr 20, 2010

Thanks for your reply.

Actually your idea is not bad, I also thougt at that. But, corresponding with so many translators everytime I'm looking for a translator/reviewer it costs time. this is why I'm looking for some kind of an evaluating system/process, in order to save me time and also to have a precise feedback so that I can decide what to do further - continue with that translation or do a new one or parts only of it.

All the best,
Marco


 

rma (X)
TOPIC STARTER
something interesting Apr 20, 2010

here is a very interesting link for those of you who are interested:

[LINK]http://www.medicaldesignonline.com/article.mvc/Six-approaches-to-measuring-translation-quali-0001?VNETCOOKIE=NO[/LINK]

I realy want to hear your ideas about this subject: how can we measure and evaluate translations?

[Edited at 2010-04-20 08:54 GMT]


 

Tony M
France
Local time: 07:03
Member
French to English
+ ...
Nothing beats reliable assessors Apr 20, 2010

Some time ago, I worked briefly as an assessor for a company who sent me short sample translations from a number of translators and asked for an evaluation of the translation quality, based on scoring a number of criteria. The job was interesting, not least to see just how many different translators they had got samples from! However, the rate paid was abysmal, and it struck me as rather farcical that they were seeking to find low-cost outsourcers, and then paying a miserable rate to a whole team of assessors in order to try and form a meaningful opinion — it would probably have been more economically viable and certainly quicker to employ quality outsourcers in the first place!

What was, however, horrifying, was the truly appalling quality of the vast majority of the test translations submitted! I'd say about 90% were wholly unacceptable, and of the remainder, only about 2% were what I'd call 'quality' translations.

It was obvious that a very large number of these translators were in no way native-speaker level in the target language; and of those who perhaps were, it was equally clear that they did not have the required specialist technical knowledge in the specific field involved.

Translation is peculiar, isn't it, inasmuch as the customer is often ill-placed to judge the quality of what they're buying — unlike many other fields.


 

Tomás Cano Binder, BA, CT  Identity Verified
Spain
Local time: 07:03
Member (2005)
English to Spanish
+ ...
Indeed Apr 20, 2010

Tony M wrote:
Nothing beats reliable assessors

I entirely agree. The best you can get is an impartial evaluator who can give you a detailed report on why a translation does not show a professional level. I also think assessments like "It's not good" are not very helpful (or even fair, for that matter), so specific examples and solid references are a must.

I think you should probably think about some way of summarising and documenting any errors spotted by your evaluators, also with a categorisation of the different errors. Some of my customers have such documents in Excel format, and in them I classify the mistakes, explain the nature of the problem and score it in a scale based on severity. Maybe thinking about such an instrument is something you could consider.


 


To report site rules violations or get help, contact a site moderator:


You can also contact site staff by submitting a support request »

Best method to evaluate translations from others

Advanced search







BaccS – Business Accounting Software
Modern desktop project management for freelance translators

BaccS makes it easy for translators to manage their projects, schedule tasks, create invoices, and view highly customizable reports. User-friendly, ProZ.com integration, community-driven development – a few reasons BaccS is trusted by translators!

More info »
Déjà Vu X3
Try it, Love it

Find out why Déjà Vu is today the most flexible, customizable and user-friendly tool on the market. See the brand new features in action: *Completely redesigned user interface *Live Preview *Inline spell checking *Inline

More info »



Forums
  • All of ProZ.com
  • Term search
  • Jobs
  • Forums
  • Multiple search