Pages in topic:   [1 2] >
Article about Fair Rate for Post-Editing
Thread poster: TRANSLATED SRL

TRANSLATED SRL
Italy
Local time: 10:22
Member (2002)
Italian to English
+ ...
Jan 24, 2013

Hello,

For the last few year we have been doing post-editing paying translators from ~70% to 110% of their requested standard rate. Unfortunately MT quality varies a lot based on the language pair and subject. In general most of the translators where happy with post-editing (see MyMemory SDL Plugin success) but some translators feel they have been overpaid (few ) and some underpaid ( many ).

We are working on models and tools to measure how MT is helping productivity and quality and when not.

If you can contribute with concrete ideas your comments are super welcome.

Here is the Article on TheBigWave
http://thebigwave.it/words-on-the-fly/post-editing-rate/

Thanks
Marco


Direct link Reply with quote
 

Phil Hand  Identity Verified
China
Local time: 17:22
Chinese to English
Good empirical approach Jan 24, 2013

It's great to be thinking about this empirically. I'm very much in the anti-MT camp - or rather, in the anti-post-editing camp. I think that if you do the experiments, and combine the time-and-motion studies with quality measures, you'll find that post-editing offers no benefits whatsoever. But it's good to do the measurements and confirm it.

One question: you write:
post-editing paying translators from ~70% to 110% of their requested standard rate


is that 70-110% of their standard post-editing rates, or standard translation rates? If post-editing rates, how do they compare with translation rates?


Direct link Reply with quote
 

TRANSLATED SRL
Italy
Local time: 10:22
Member (2002)
Italian to English
+ ...
TOPIC STARTER
Thanks for your comments Jan 24, 2013

Phil Hand wrote:
One question: you write:
post-editing paying translators from ~70% to 110% of their requested standard rate


is that 70-110% of their standard post-editing rates, or standard translation rates? If post-editing rates, how do they compare with translation rates?



That would be their standard translation rates - leveraging matches from TM or MT reduces the word count (equivalent wct).


Direct link Reply with quote
 

Philippe Etienne  Identity Verified
Spain
Local time: 10:22
Member
English to French
My experience too Jan 24, 2013

From your article:
"Our initial results show that post-editing data rich and morphologically simple languages, such as English, French, Italian, Spanish, etc., require an effort comparable to fixing a 75-99% TM fuzzy match (and by consequence would be paid ~60% of the full rate). "

I highlight this because some people tend to think that the "going" rate for post-editing is a proofreading rate (20-30% of full rate).
I assume here that 60% of full rate is applied to every single word and not "discounted" with more Trados discounts!

Using bilingual raw MTed files and a TM, I came to a similar conclusion, in that an MT segment was usually more useful compared to a 70% Trados fuzzy match, but usually less useful compared to an 80% fuzzy match.

BUT:

First restriction: In the few projects I have accepted from long-standing, reliable, trustworthy agency customers, amounting to a few hundred thousand words in total, MT wasn't about simply putting the source text into Google Translate or other out-of-the-box MT consumer solution, but a trained engine fed with man-made TMs, specific glossaries and upstream and downstream optimisation with each delivery.

Second restriction: the nature of texts. I used MT only on "user assistance" only, ie online help or software instruction manuals. Because I know that there, MT is helpful.

Third restriction: the objectives of post-editing are clear. In other words, don't expect a post-edited text to sound like a human translation.

Fourth restriction: this kind of work is sooo boring that it deserves to be paid more than straight translation (meaning you should earn more doing one hour of MT than one hour of standard translation).

Philippe


Direct link Reply with quote
 

Phil Hand  Identity Verified
China
Local time: 17:22
Chinese to English
So, is it worth it? Jan 24, 2013

If you're paying nearly the same for post-editing that you do for translation, what benefits are coming of it? Is post-edited MT better quality than human translation? Or does the ~20% saving in translation costs improve your business model?

Like I say, I'm anti-post-editing. In my pair, MT is so poor that it makes no sense. It's easier to retranslate from scratch. In other pairs, where MT is better, I can't see how post-editing adds value. Either you take the raw MT output for gist; or you should get it translated by a human for precision. I can't see what niche there is for post-editing.


Direct link Reply with quote
 

LegalTransform  Identity Verified
United States
Local time: 04:22
Member (2002)
Spanish to English
+ ...
Fair Rate for Post Editing Jan 24, 2013

I view MT as a tool that translators can use (in some cases) to increase productivity in a manner analogous to software programs used by accountants and architects and in no way does its current incarnation justify any discounts or reduced cost for professional translation despite the claims made by companies attempting to get discounts for post-editing machine translation "spaghetti".

For me, the fair rate for post-editing is the standard translation rate.

Related Article:
http://thelanguageconsultant.blogspot.com/2013/01/whats-wrong-with-machine-translation.html


Direct link Reply with quote
 

Anton Konashenok  Identity Verified
Czech Republic
Local time: 10:22
English to Russian
+ ...
A word from another anti-MT Jan 25, 2013

I think it's only fair that a linguist be paid the same amount for an hour of work, whether it be translation, proofreading, or post-editing, and that's how I try to assign my rates.
Several times in recent years, I have tried to assess the usefulness of several MT engines for professional work by putting a sample text through them and measuring how long it would take me to brush the result up (under my own quality criteria) and to translate another fragment of the same text without MT. As I work in the top segment of the market, I used fairly complex texts. In both pairs I tried, English-Russian and Russian-English (I am a native speaker of both languages), it took me about the same time to fix the MT results as to translate a similar text of that size from scratch. For the second stage of my test, I would put both translations away for several days and then proofread them. Despite the same amount of time spent, post-edited MT looked consistently worse in style than my own translation. Thus, the benefit of using MT in my case turned out to be clearly negative. Furthermore, if I were to make any adjustments to my rates, I'd adjust the post-editing rate upwards for the emotional damage I get from dealing with this kind of text.
It should be said, however, that the quality of MT varies very widely from one language pair to another. In particular, from my own observations, Google Translate produces fairly readable (although often incorrect) translations from German to English, uglier ones from Russian to English, and abysmally ugly ones from Hebrew to English. Just a couple of days ago, I used it to translate some text from Chinese. Unlike the previous three cases, I couldn't give even a very rough guess how accurate the translation was, but it surprised me with a fairly good word flow.


Direct link Reply with quote
 

Ambrose Li  Identity Verified
Canada
Local time: 04:22
Chinese to English
+ ...
Google Translate and Chinese Jan 25, 2013

Anton Konashenok wrote:

Just a couple of days ago, I used it to translate some text from Chinese. Unlike the previous three cases, I couldn't give even a very rough guess how accurate the translation was, but it surprised me with a fairly good word flow.


It depends on the text. Since it has been a while I tested GT for Chinese, I just gave it two little tests, one being a piece I submitted for the ProZ translation contest and the other one a very short blog post of mine. In neither case did GT give me anything that makes any sense.


Direct link Reply with quote
 

TRANSLATED SRL
Italy
Local time: 10:22
Member (2002)
Italian to English
+ ...
TOPIC STARTER
Thank you all for your comments. Jan 25, 2013

Anton Konashenok wrote:

I think it's only fair that a linguist be paid the same amount for an hour of work, whether it be translation, proofreading, or post-editing, and that's how I try to assign my rates.




This sentence from Anton aptly describes our approach: what matters to translation providers is how much they earn in a day's work. All attempts to defining a fair price should aim at increasing productivity without reducing the providers' income. This is what we are trying to achieve with our research and development in translation technology.


Direct link Reply with quote
 

Alex Lago  Identity Verified
Spain
Local time: 10:22
Member (2009)
English to Spanish
+ ...
More productivity should equal more income Jan 25, 2013

TRANSLATED SRL wrote:
All attempts to defining a fair price should aim at increasing productivity without reducing the providers' income. This is what we are trying to achieve with our research and development in translation technology.


And in my opinion this is the problem, you want to take an increase in productivity by the translator and only have you (the agency) and the client benefit from it, this is not fair.

If a translator achieves an increase in productivity they should also be entitled to receive an increase in income, if not what's the point of the increase in productivity.

We are not employees, we are freelancers, and should be treated as such.


Direct link Reply with quote
 

Samuel Murray  Identity Verified
Netherlands
Local time: 10:22
Member (2006)
English to Afrikaans
+ ...
Similar experience, but with GT Jan 25, 2013

Philippe Etienne wrote:
Using bilingual raw MTed files and a TM, I came to a similar conclusion, in that an MT segment was usually more useful compared to a 70% Trados fuzzy match, but usually less useful compared to an 80% fuzzy match.


My experience has been roughly the same, in fact. A very high fuzzy match may take less time to edit than an MT match, but at the same time even a medium high fuzzy match is easier to edit into something that sounds more idiomatic than the edited MT match. I can edit MT matches to sound very idiomatic, but that takes time (more time, in fact, than simply translating from scratch). However, I can get "adequate" translations using MT matches quicker than through translation from scratch, and some document types are more forgiving when it comes to mere adequacy.

My biggest problem with editing MT matches is that I make more editing mistakes (particularly certain types of mistakes) during the first round of MT editing than I would have made when editing a human translation or when translating from scratch.

An automated check that checks for words or lettergroups that occur more than once in the translation would have been a great help to me, for example, because neglecting to delete misplaced words after having retyped them elsewhere in the sentence is a typical editing mistake (for me) for MT matches. In fact, this accounts for 99% of my MT editing mistakes during the first round of MT editing.


Direct link Reply with quote
 

Stanislav Pokorny  Identity Verified
Czech Republic
Local time: 10:22
English to Czech
+ ...
Seconded Jan 25, 2013

Alex Lago wrote:
If a translator achieves an increase in productivity they should also be entitled to receive an increase in income, if not what's the point of the increase in productivity.


Indeed. Increased productivity means increased value which should be followed naturally by more benefits for all parties involved.

OTOH, I must appreciate that Marco discusses this with the translators, and not just with TAUS, peer agencies, and MT engine providers. The common practice is "here you have a post-editing job at 30% of your full rate; take it or leave it". Obviously, leave it is always the choice in cases like that.

So, thank you Marco, your initiative is appreciated, at least by some ProZians.

PS: I'm another member of the anti-MT camp...


Direct link Reply with quote
 

Samuel Murray  Identity Verified
Netherlands
Local time: 10:22
Member (2006)
English to Afrikaans
+ ...
One more comment Jan 25, 2013

Original article says:
Our initial results show that post-editing data rich and morphologically simple languages, such as English, French, Italian, Spanish, etc., require an effort comparable to fixing a 75-99% TM fuzzy match (and by consequence would be paid ~60% of the full rate).


I would not be happy to receive only 60% pay for MT editing because it is not certain that all MT matches will be useable. You might hit a bad patch with a source text that just happens to be incompatible with the MT engine, and then you're sunk.

Even if we suppose that in an MT-only job all of the matches are equivalent to 75% matches, then in my experience such a job would still be undesirable. Here's why:

Very few (if any) of my WWC translation jobs have high numbers of 75% matches in them. The number of 75% matches in my translation jobs tend to be rather low, as the majority of matches tend to be either 0% or 95%+). This means that if I do hit a bad patch of useless 75% matches, the time saving or rate paid for the other matches still make up for it.

But if the entire job were to consist of 75% matches, there would be no safety net for patches of useless matches.


Direct link Reply with quote
 

Grzegorz Gryc  Identity Verified
Local time: 10:22
French to Polish
+ ...
Post-editing, home made :) Jan 25, 2013

Phil Hand wrote:

If you're paying nearly the same for post-editing that you do for translation, what benefits are coming of it? Is post-edited MT better quality than human translation? Or does the ~20% saving in translation costs improve your business model?

I would rather say ~50% or ~75% savings.
The translation offices start at this kind of level and search for idiots...

Like I say, I'm anti-post-editing. In my pair, MT is so poor that it makes no sense. It's easier to retranslate from scratch.

I work mainly in the FR-PL pair and, at this stage and with the current assumptions, the machine translation simply makes no sense.
I.e. it's not a very frequent pair and almost nobody invests seriously in it.
AFAIK some big companies have well trained EN-PL machine translation engines which may give decent results (combined with some EBMT/CAT technology, not as standalone approach) but for the FR-PL pair nobody cares, the poor guys from translation offices simply load all their TMs in some Moses based stuff with no categorization at all, so the result is... huh... very Google like...
I.e. I don't blame Google here but I don't see the reason why I should give any discount to a translation office for an input I can receive easily for very few bucks from Google.
And the Google input is often better.

E.g., me and my wife, we pay 5-7 USD per month for Google Translate.
The price is 25 USD per 1 mln chars (let's say, some 150.000 words).
If a translation office tries to propose me 0.02 EUR per word for post-editing, they're simply kidding.
According to Google MT rates, I could propose a 0,00017 USD discount per word,
And basically this is the real value.
E.g. they should compare the size of their corpus and the Google's one.
These corpora are partially identical, e.g. Acquis Communautaire taken from free from DGT.
Me too, I processed DGT TMs and use 'em heavily.
I.e. I could give some discount for some added value but it's virtually no added value in this primitive approach.

In other pairs, where MT is better, I can't see how post-editing adds value.

As I work mainly in Déjà Vu (DVX2 now), a post-editing like approach is a reality for me.
From several years.
Despite of some evolution on the CAT market, it's probably still the only one CAT tool using machine translation algorithms (EBMT, intelligent external MT interfaces...), maybe Cafetran also does but (unfortunately) I don't follow its development.
So why I translated almost 30.000 words during the last 3 days in the EN-FR pair.
Of course, it was a draft quality (in fact, as expected), but I'm able to provide a decent quality for up to 7.000 words daily in some areas (e.g. IT help or user manuals).
It's basically the throughput the translation offices expect for post-editing.
But they're wrong.
With the garbage they provide this level can't be reached.

The difference is:
  • I must have total control over the project and over the translation memories (the translation offices don't grant it neither for online TMs nor for machine translation engines),

  • I have huge well tuned terminology data bases (the translation offices usually don't maintain terminology data bases at all 'cause the costs are big end imprecise).

But in this cas, it's me who provide the added value

It somebody does it for me in a sound way in the translation office, I could accept some discount but for the moment it's a pure theory.
All the post-editing propositions I ever received (i.e. their rates/technology ratio) simply insulted my intelligence.

Either you take the raw MT output for gist; or you should get it translated by a human for precision. I can't see what niche there is for post-editing.


With the current "zero investment" approach of most "tricky" translation offices, basically none.
E.g. I consider practically all sentences below 5 words are proposed incorrectly by not trained engines, below 10 words they're very dubious and often must be translated from scratch.
The general suggestions is not adapted for specialized areas.
So, functionally, it's a nonsense for most cases.

And even for the EU stuff (supposed to match well the DGT corpora) which produces a "flamboyant" legalese in machine translation. the problem is one must always check thoroughly related directives etc., so the real work can't be really much faster if a good quality is expected.
Unless the translation office expects I'll simply change the endings and don't care about the meaning...
In this case, 0.02 EUR per word would be OK

Cheers
GG


Direct link Reply with quote
 

Grzegorz Gryc  Identity Verified
Local time: 10:22
French to Polish
+ ...
MyMemory plugin settings... Google forwarding... Jan 25, 2013

TRANSLATED SRL wrote:

Phil Hand wrote:
One question: you write:
post-editing paying translators from ~70% to 110% of their requested standard rate


is that 70-110% of their standard post-editing rates, or standard translation rates? If post-editing rates, how do they compare with translation rates?


That would be their standard translation rates - leveraging matches from TM or MT reduces the word count (equivalent wct).


The thing I find completely aberrant in the MyMemory plugin is the fixed 85% match value for MT.
It should be configurable or at least set as the minimum match threshold defined in the project settings.

The 85% value is far too big assumed the nonsense thrown by this plugin e.g. in my main pair (FR-PL).
Just one rapid example.
FR "Depuis peu, nous avons recruté de nouvelles femmes de chambre"
GT-EN "Recently, we hired new maid" (almost good, sing. instead of plur.)
GT-PL "Niedawno zatrudniliśmy nową pokojówkę." (almost good, as above)
MM-PL "Niedawno, możemy mieć zatrudniony nowy pokój dla kobiet." (frankly aberrant)
MM-PL meaning "Recently, we can have employed a new room for women."
It's a very natural and rather simple sentence sentence, for a technical text it would be far worse.

I noticed also MyMemory sometimes seems to submit the source to Google and throws the Google output flagging it as a 85% match.... ehm...
I can't see the pattern but it works like that for some projects.

I.e. it's nice to have the MM plugin for free but its match (i.e. payment) related assumptions are crazy.

Cheers
GG


Direct link Reply with quote
 
Pages in topic:   [1 2] >


To report site rules violations or get help, contact a site moderator:


You can also contact site staff by submitting a support request »

Article about Fair Rate for Post-Editing

Advanced search







WordFinder
The words you want Anywhere, Anytime

WordFinder is the market's fastest and easiest way of finding the right word, term, translation or synonym in one or more dictionaries. In our assortment you can choose among more than 120 dictionaries in 15 languages from leading publishers.

More info »
Anycount & Translation Office 3000
Translation Office 3000

Translation Office 3000 is an advanced accounting tool for freelance translators and small agencies. TO3000 easily and seamlessly integrates with the business life of professional freelance translators.

More info »



Forums
  • All of ProZ.com
  • Term search
  • Jobs
  • Forums
  • Multiple search