Evaluating Tests
Thread poster: Ala Rabie

Ala Rabie  Identity Verified
Egypt
Local time: 10:12
Japanese to Arabic
+ ...
Sep 13, 2006

Help!

I was reached by an agency VM personnel to evaluate a test given to another freelancer. The PM was talking as a friend, with no official issues rising [that is exactly why I hate operating business on IM applications.]

I accepted the task in the same friendly manner, and only asked for official credit for my service---in short, I will do it for free.

Now comes the fun part: the VM guy sent me not only the test, but also an official internal Quality Assurance document for me to fill in, containing many strange fields like "source", "target", "correct translation", "comments", etc. It became clear to me that the task is not a simple evaluation, but a forced translation as well!

I had a previous experience of refusing doing the very same test myself for the agency never proved professionalism, and never approved my rates for they are "too high, and we get Japanese natives to do the job for 4 cents per word".

I sent the VM guy an official email stating that I will never be engaged in internal documents as it is "not an option for me". He replied offering explanation if any part of the document is not clear to me, to which I repeated myself. He finally gave in and asked me to make sure to include comments if possible.

I am surprised as how a favor-to-friend turned out to be a mere act of exploitation, and now I do not know what to do.

Now for the test itself. The translation is beyond awful! It sucks! And any attempt to include corrections will end up with a re-translation of more than 70% of the text, which I am not intending to do on whatsoever. What should I do? Mark the mistakes by category without explanations or corrections, while including a key-legend?

Advise, advise!

~Ala
alamnesis.com - Expect beyond perfection with the power of Semiology

[Edited at 2006-09-13 07:09]


Direct link Reply with quote
 
Sophia Hundt  Identity Verified
Local time: 03:12
Russian to English
+ ...
I wouldn't even be afraid to return it with the explanation you just provided... Sep 13, 2006

I think I would just return it with the exact same explanation you just provided. I find that it is in one's interest as a translator to be direct about not accomodating unreasonable requests on behalf of the outsourcer. After all, it seems to me that your time (which you will spend doing something better) is more valuable. In short, I wouldn't leave much room for exploitation for the fear of losing the client (you probably won't lose the client this way).

[Edited at 2006-09-13 04:16]


Direct link Reply with quote
 

Cristiana Coblis  Identity Verified
Romania
Local time: 11:12
Member (2004)
English to Romanian
+ ...
evaluating tests Sep 13, 2006

I have done my fair share of these evaluations and frankly most of them are not so good. However, there is an occasional one that makes your day. I was lucky enough to evaluate excellent tests every now and then, these make me feel much better

I would never do this for free because it is so time consuming. Depending on the size and quality of the test and the type of evaluation form you can spend up to 2 hours on the evaluation form alone.

In your case, since you committed to do this as a personal favor, I would give them a pass/no pass answer and let them know the reason.


Direct link Reply with quote
 

Özden Arıkan  Identity Verified
Germany
Local time: 10:12
Member
English to Turkish
Second Cristiana's suggestion: Sep 13, 2006

In your case, since you committed to do this as a personal favor, I would give them a pass/no pass answer and let them know the reason.

There's nothing else you can do, given that you are doing a professional service as a favor. Evaluation is more difficult, stressful, and time consuming than translation or even proofreading. In fact, it is better when the outsourcer supplies you with a form like that one, because it provides you a structure or a framework, so that you can stick to what is essential. However, I would never fill in the "correct translation" field, be it a favor or a job: it would amount to providing free translation service.


Direct link Reply with quote
 

Elisabete Cunha  Identity Verified
Portugal
Local time: 09:12
Member (2006)
English to Portuguese
+ ...
Evaluating tests Sep 13, 2006

I have already done some evaluations like these for a company and I also had to fill out a table and give points (in %), to: the overall content, punctuation, syntax, layout and final comments, etc.
What I have done was to proofread each one of the translations and used the MS Track Changes and Comments and inserted comments for each mistake.
I was paid to do this, of course. And I did found this kind of task to be very interesting. Of course I spent a lot of time with this. But the payment was not bad at all and I would do it again, if the opportunity arises.


Direct link Reply with quote
 

Claudia Krysztofiak  Identity Verified
Germany
Local time: 10:12
English to German
+ ...
Pass or did not pass and that's it Sep 13, 2006

As Cristiana said, let them have this information since you offered this as a personal favour.

You could of course explain to them that you could do an analysis of this for a rate of xy per word (if you wish to do such a thing).

And at the same time you should point out that you will not offer a correct translation, e.g. because there are many possible correct translations and this correction would be part of the original translators work, or for whatever reason you think suitable.

In general, I would make it a rule not to mix personal favours with professional work.

I do a favour to a friend sometimes, and I know that she is a friend, for she always is very reluctant about asking me the favour and always asks me not to put too much time into it. And if I then tell her, well that is more than I expected and I cannot do it, then she will apologize. This is what a personal favour is all about.

Everything else is professional and should be paid at a reasonable rate.

If you lose this client because of your behaviour you should not worry too much about it. If they do not respect your work they will only cause you more trouble in the future and try to exploit you even more.


Direct link Reply with quote
 

Magda Dziadosz  Identity Verified
Poland
Local time: 10:12
Member (2004)
English to Polish
+ ...
Misunderstanding, perhaps? Sep 13, 2006

enshrine wrote:

Now comes the fun part: the VM guy sent me not only the test, but also an official internal Quality Assurance document for me to fill in, containing many strange fields like "source", "target", "correct translation", "comments", etc. It became clear to me that the task is not a simple evaluation, but a forced translation as well!


Hi,

what you describe above seems to me a fairly standard way of evaluating tests - typically the evaluator needs to do just that: fill in a form as this, substantiate every comment he/she has. "Source", "target", "correct" etc. are typical fields of evaluation questionaire. It is a time consuming job, especially if the test is poor, but then it's paid job, as any other. Usually charged by the hour.

I wouldn't call it "forced" translation - it is just that evaluator must be accountable for his/her work and needs to be able to justify his/her opinion. Test evaluation is a very serious job.


I had a previous experience of refusing doing the very same test myself for the agency never proved professionalism, and never approved my rates for they are "too high, and we get Japanese natives to do the job for 4 cents per word".


Perhaps I misunderstood you here, but this question immediately comes to my mind - why on earth you want to make such a favour, a job for free, to an agency which you consider unprofessional and generally have bad experience with?

My advice is to contact the guy and tell him that you misunderstood his initial offer and you haven't realized what that job really means.

Magda


Direct link Reply with quote
 
Ford Prefect  Identity Verified
Burkina Faso
Local time: 08:12
German to English
+ ...
... Sep 13, 2006

Occasional quick freebies for established clients are one thing but for an agency you already refused to work for?

Also I have never seen any agency request the evaluator produces their own fresh translation - a score and comments are usually enough. If they are looking for good quality workers then you only need find one true screw-up in a test to reject it and spend no more time on it. It's the really good ones you need to be more careful with if you are to discriminate fairly between the potential candidates.

If the test is no good just say so and forget the bureaucracy. Especially if you aren't billing them.


Direct link Reply with quote
 

Ala Rabie  Identity Verified
Egypt
Local time: 10:12
Japanese to Arabic
+ ...
TOPIC STARTER
Did it! Sep 14, 2006

Here is what I did:
  • Turned tracking on.
  • Translated the first paragraph (240-ish Japanese characters) and included it below the other translation.
  • Edited formatting and simple mistakes.
  • Added a comment for each mistake indicating its category using Word's notes feature.
  • Completed my own Evaluation Report form.
  • Sent an email stating that the evaluation process had to be halted at x characters for too many mistakes returned. Attached the document and the Report.

My work was professional enough to get an 'excellent work - very kind of you'-y response.

After all, it seems to me that your time (which you will spend doing something better) is more valuable. In short, I wouldn't leave much room for exploitation for the fear of losing the client (you probably won't lose the client this way).

This may seem embarrassing, Sophia, but I was under no official obligation; the entire thing was put in a would-you-do-me-a-favor manner, and you know that no-one can reject helping someone in need.

As for the agency, well, I refused working for them twice before ^^; -- I told you it is getting embarrassing! -- and I do not care for changing my stand in the near future.

Thank you, Sophia

I was lucky enough to evaluate excellent tests every now and then, these make me feel much better

I get really shocked everytime I do an evaluation; if the quality of your work is THAT bad, why on earth would you submit it at all!? I hope I get one of those 'excellent test' too, Cristiana

In your case, since you committed to do this as a personal favor, I would give them a pass/no pass answer and let them know the reason.

I have sent them my very own Evaluation Report form. It is a score table with 15 negative types [or (mis)types] under 3 categories, and one positive type. Each major mistake results in a -2 pts, while each moderate one substracts 1 pt. The positive type adds 4 pts. When the translation remains above -10 pts, it is considered good and passes. If it goes below -50, it is hopeless!

This test got to the magical -50 pts after 230 characters! And the entire document is 1237 characters long!

God! God!

However, I would never fill in the "correct translation" field, be it a favor or a job: it would amount to providing free translation service.

Could not agree more, Özden

What I have done was to proofread each one of the translations and used the MS Track Changes and Comments and inserted comments for each mistake.

Exactly! Unless I develop another method, this one remains the most appropriate for .DOC files.

Thank you, Elisabete

In general, I would make it a rule not to mix personal favours with professional work.

I do a favour to a friend sometimes, and I know that she is a friend, for she always is very reluctant about asking me the favour and always asks me not to put too much time into it. And if I then tell her, well that is more than I expected and I cannot do it, then she will apologize. This is what a personal favour is all about.

Everything else is professional and should be paid at a reasonable rate.

If you lose this client because of your behaviour you should not worry too much about it. If they do not respect your work they will only cause you more trouble in the future and try to exploit you even more.

True, Claudia.. very true.
I learned my lesson and I should draw thicker boundaries.

I wouldn't call it "forced" translation - it is just that evaluator must be accountable for his/her work and needs to be able to justify his/her opinion. Test evaluation is a very serious job.

First of all, Magda, I agree with you that it is a very serious job. In fact, our entire profession is dead serious! What worried me here was the 'correct ranslation' part. I do not remember having any of my exam sheets returning with any this-is-the-correct-answer comments. (In a matter of fact, the Japanese 'ticking' style marks only the correct answers with red circles. This saves ink I guess)

'Correct translation' field means a free translation, and I would never do it again unless I am paid by the hour for a comprehensive editing job.

Thank you, Magda

Occasional quick freebies for established clients are one thing but for an agency you already refused to work for?

I know. In fact, I do not approve doing freebies to get clients as a policy.

If they are looking for good quality workers then you only need find one true screw-up in a test to reject it and spend no more time on it. It's the really good ones you need to be more careful with if you are to discriminate fairly between the potential candidates.

Exactly.

Thank you

~Ala
alamnesis.com - Expect beyond perfection with the power of Semiology

[Edited at 2006-09-14 11:22]


Direct link Reply with quote
 


To report site rules violations or get help, contact a site moderator:


You can also contact site staff by submitting a support request »

Evaluating Tests

Advanced search







SDL Trados Studio 2017 Freelance
The leading translation software used by over 250,000 translators.

SDL Trados Studio 2017 helps translators increase translation productivity whilst ensuring quality. Combining translation memory, terminology management and machine translation in one simple and easy-to-use environment.

More info »
WordFinder
The words you want Anywhere, Anytime

WordFinder is the market's fastest and easiest way of finding the right word, term, translation or synonym in one or more dictionaries. In our assortment you can choose among more than 120 dictionaries in 15 languages from leading publishers.

More info »



Forums
  • All of ProZ.com
  • Term search
  • Jobs
  • Forums
  • Multiple search