Pages in topic:   [1 2 3 4 5] >
Translation Iteration Experiment English-Spanish: the results!
Thread poster: Tomás Cano Binder, BA, CT

Tomás Cano Binder, BA, CT  Identity Verified
Spain
Local time: 05:27
Member (2005)
English to Spanish
+ ...
Feb 23, 2010

Dear colleagues and nevertheless friends,

Many of you working in the English-Spanish/Spanish-English pairs might remember a little experiment I proposed about a month ago. The forum was here.

The goal of the experiment was to see what would happen to a short text if a team of 20 translators (10 English-Spanish, 10 Spanish-English) translated a text iteratively, i.e. if each of them translating the result of the translation of the previous team member. Expressed in the original forum's terms, it was "just to see what happens."

A very generous group of 20+ people offered to help, and we now already have the results. I thank all participants for their help, in most cases in the middle of a busy schedule, which makes your contributions even more valuable.

As those of you who participated will be eager to know what the original source text was and what the text has become after the process, here we go!

The original text was taken from a 1989 book about a DTP software called Ventura Publisher (yes, we are talking about the times of the GEM environment!). I keep this book for sentimental reasons, but of course it is so obsolete now that you won't find its contents anywhere in the Web. That was the main reason I chose this book: Internet would not corrupt the translation work.

Now, this was the original text:

Welcome to the revolution. A movement is underway that is revolutionizing the way businesspeople produce printed pages. This book will show you how to profit from that trend using a remarkable software program called Bold Tag Editor. To gain perspective, let's remember an earlier revolution and an event that dramatically illustrates the power of the printed word. In January 1776, Thomas Paine published Common Sense, a vivid call for the formation of a republic. His 50-page pamphlet led directly to the Declaration of Independence a few months later. Common Sense was an immediate bestseller (120,000 copies in three months, equivalent to 10 million today). Paine's problems lay not with the writing and selling, but with the printing and production, which delayed him and cost more than expected.

The text was not meant to be a translation test at all. Just a relatively simple text any professional translator can translate with a reduced effort.

And.... this is the same text after 10 translations and their respective backtranslations:

Welcome to the revolution! The process that will revolutionise the development of printed advertising is already under way. This book will show you how to use Bold Labels Editor so that you can benefit from this technology. But first, to put things in perspective, let us look back on an earlier revolution, an event that shows the immense power of the printed word. In January 1776, Thomas Paine published his pamphlet “Common Sense", in which he advocated a republic. The tract was a resounding success (selling 12,000 copies in three months, the equivalent of ten million copies in today’s terms). Although Paine had no problems writing or selling his work, there were a lot of printing and production errors that caused delays, making costs soar much higher than expected.

Now, our colleague Pablo Bouvier had the magnificent idea of making a survey on our perception of what would happen. This was suggested in this forum and the survey was made with SurveyMonkey.

Below you can see the results and some very interesting comments made by the participants, which we can comment on later on:
Question 1:
Tomas-TranslationExperiment-pic1.jpg

Question 2:
Tomas-TranslationExperiment-pic2.jpg

Question 3:
Tomas-TranslationExperiment-pic3.jpg

Question 4:
Tomas-TranslationExperiment-pic4.jpg

My main conclusions about the iterations in relation to the questions are:

Aspect 1. Overall quality:
I think that, despite the fact that the final text is less compact than the original, the meaning is still there --even if rather overtoned-- in most parts (some have been deleted or altered; please see below in Aspect 3).

Aspect 2. 10 human iterations better than 1 MT iteration:
Please see the results of machine translation in the next posting. My personal opinion is that we humans have made mistakes that MT would not do (in accuracy of data for instance), but still is objectively better than 1 iteration with machine translation because the human translation makes sense and is gramatically correct.

Aspect 3. What aspect has suffered more in the process:
Most of respondents to the survey thought that register/style would suffer more than accuracy of data. The surprising result here is that we did damage data. We also destroyed the part "His 50-page pamphlet led directly to the Declaration of Independence a few months later."

A blunt thought is that we might think we are more careful dealing with data than we actually are...icon_smile.gifAspect 4. Will the translation be longer or shorter after the process:
From what I can see in the list of translations made along the process, the text grew and shrunk at different moments. The final text is of course longer. There is only 2 words more (127 words in the original, 129 in the final text), but a chunk was lost in the translation ("His 50-page pamphlet led directly to the Declaration of Independence a few months later.")

What do you think about the results and about how it all compares with your expectations when you read about this experiment?


 

Tomás Cano Binder, BA, CT  Identity Verified
Spain
Local time: 05:27
Member (2005)
English to Spanish
+ ...
TOPIC STARTER
1 iteration with machine translation Feb 23, 2010

Now about the iteration with machine translation. I used Google to translate into Spanish and Babelfish to translate back into English. You might wonder why I used two separate tools. Well, it was the same reason why 20 different people did the human experiment: knowing about the previous translation contaminates the experiment, even in Google as you can quickly check by backtranslating a translation done with Google.

The result of the MT iteration is:

Welcomes to the revolution. The movement is in march that is revolutionizing the form of businesses of producing printed pages. This book will show how to remove to benefit from that tendency using a program to him of remarkable software called Bold Publishing Tag. In order to obtain a perspective, we before remember a revolution and an event that the power of the printed word illustrates of dramatic way. In January of 1776, Thomas Paine published Common Sense, a call lived for the formation of a republic. Its 50-pamphlet of a page lead directly to the Declaration of Independence of months later. Common sense was an immediate success of sales (120,000


 

Krzysztof Kajetanowicz (X)  Identity Verified
Poland
Local time: 05:27
English to Polish
+ ...
wow Feb 23, 2010

That was one interesting experiment.

I'm not surprised that the quality of 10 human iterations is better than MT. The MT version is appalling.

What did surprise me is that, save for the ommission and the deformed number, the meaning of the original was relatively well preserved.

[Edited at 2010-02-23 10:02 GMT]


 

John Rawlins  Identity Verified
Spain
Local time: 05:27
Spanish to English
+ ...
Good work Feb 23, 2010

Excellent idea Tomás - and very well implemented.

 

Samuel Murray  Identity Verified
Netherlands
Local time: 05:27
Member (2006)
English to Afrikaans
+ ...
Interviews with the translators Feb 23, 2010

Tomás Cano Binder, CT wrote:
Aspect 3. What aspect has suffered more in the process:
Most of respondents to the survey thought that register/style would suffer more than accuracy of data. The surprising result here is that we did damage data. We also destroyed the part "His 50-page pamphlet led directly to the Declaration of Independence a few months later."


It would be nice if you can ask ProZians if they have any questions for the translators, so that you can send the translators the question and let them answer it anonymously.

I for one would be interested to know from the translator who changed 120k to 12k why he did it. Perhaps he had a good reason (whether I agree with him or not). I would also like to ask the translator who decided to leave out the one sentence, why did he do it. Perhaps he had a good reason too.

Samuel


 

Tomás Cano Binder, BA, CT  Identity Verified
Spain
Local time: 05:27
Member (2005)
English to Spanish
+ ...
TOPIC STARTER
Plain confusions Feb 23, 2010

Samuel Murray wrote:
I for one would be interested to know from the translator who changed 120k to 12k why he did it. Perhaps he had a good reason (whether I agree with him or not). I would also like to ask the translator who decided to leave out the one sentence, why did he do it. Perhaps he had a good reason too.

I think I can answer this one: the figure 120,000 was unadvertedly changed to 112,000 at some stage, and then to 12,000 later on. In the Wikipedia page about Paine they mention 120,000, so these two changes and the omission simply look like plain mistakes to me...

[Edited at 2010-02-23 10:19 GMT]


 

Kate Chaffer
Italy
Local time: 05:27
Member (2009)
Italian to English
Really? Feb 23, 2010

Krzysztof Kajetanowicz wrote:

What did surprise me is that, save for the ommission and the deformed number, the meaning of the original was relatively well preserved.


Did that really surprise you? We're talking about 20 professional translators and a text which was not particularly difficult.

What surprised me was that there were actually three rather important errors (as highlighted by Tomás). It would be interesting to know whether it was one translator that made all three or three translators that made one each. Could you tell us that, Tomás?


 

Tomás Cano Binder, BA, CT  Identity Verified
Spain
Local time: 05:27
Member (2005)
English to Spanish
+ ...
TOPIC STARTER
Shared responsibility... Feb 23, 2010

Kate Chaffer wrote:
What surprised me was that there were actually three rather important errors (as highlighted by Tomás). It would be interesting to know whether it was one translator that made all three or three translators that made one each. Could you tell us that, Tomás?

Yes, I can answer that one: each of the omissions and errors was made by a different person along the process.


 

Noni Gilbert  Identity Verified
Spain
Local time: 05:27
Spanish to English
+ ...
Editing Feb 23, 2010

To keep things in perspective, we should remind ourselves that any real project should/would have been edited, but that this step has been omitted in this experiment. So, although we are writhing at the thought of those mistakes, we can be sure that in a real life situation, the mistakes would not have persisted.

Thanks Tomás for all your work in putting this together. Fascinating.

Noni


 

Frances Leggett  Identity Verified
United Kingdom
Local time: 04:27
Italian to English
+ ...
Teaching techique... Feb 23, 2010

I'm just reading about this now but it's very, very interesting. I still get people every day who are convinced that in just a few short years, machine translation will replace human translators totally and I always stand there aghast thinking 'could this be true' but of course it can't be, well not in a few years time anyway! We still have a few years left in us!

At the Liceo Classico in Italy, one of my friends told me that while studying Ancient Greek and Latin, they all had to translate texts from Italian - Ancient Greek and Italian - Latin and then pass the text along and have a classmate translate it back into Italian to find out where they were going wrong. This is not the sort of exercise that was included in my MA in Translation Studies but I think it would be an interesting skill-developing technique!


 

Tomás Cano Binder, BA, CT  Identity Verified
Spain
Local time: 05:27
Member (2005)
English to Spanish
+ ...
TOPIC STARTER
Just the target? Feb 23, 2010

Frances Leggett wrote:
At the Liceo Classico in Italy, one of my friends told me that while studying Ancient Greek and Latin, they all had to translate texts from Italian - Ancient Greek and Italian - Latin and then pass the text along and have a classmate translate it back into Italian to find out where they were going wrong. This is not the sort of exercise that was included in my MA in Translation Studies but I think it would be an interesting skill-developing technique!

Wow. Indeed I think it would be a really interesting thing to include in any translation study programme. Just to clarify this more: do you mean that the next person only had the Greek or Latin translation to work on?


 

Tomás Cano Binder, BA, CT  Identity Verified
Spain
Local time: 05:27
Member (2005)
English to Spanish
+ ...
TOPIC STARTER
I agree - An editor would have changed the picture Feb 23, 2010

aceavila - Noni wrote:
To keep things in perspective, we should remind ourselves that any real project should/would have been edited, but that this step has been omitted in this experiment.

Yes, I certainly agree. An editing step after each translation would have corrected the gross mistakes immediately and the result would have been all the more better.

I also think that, should this have been an actual paid job, the end result would have been a lot better. I wonder whether I should invest in running a similar experiment again as a series of paid jobs, and without saying that it was an experiment! Also, what would happen if we ran the experiment with students of translation and interpretation? Wouldn't it be interesting to see how they fare and what future awaits the profession?


 

Krzysztof Kajetanowicz (X)  Identity Verified
Poland
Local time: 05:27
English to Polish
+ ...
example Feb 23, 2010

Kate Chaffer wrote:

Krzysztof Kajetanowicz wrote:

What did surprise me is that, save for the ommission and the deformed number, the meaning of the original was relatively well preserved.


Did that really surprise you? We're talking about 20 professional translators and a text which was not particularly difficult.


It's not obvious to me. Let me use an example from the same text:


/SOURCE/ Paine's problems lay not with the writing and selling, but with the printing and production, which delayed him and cost more than expected.

where costs may or may not result from delays, is not the same as

/TARGET/ Although Paine had no problems writing or selling his work, there were a lot of printing and production errors that caused delays, making costs soar much higher than expected.

where it is not only implied that delays caused the costs but that they were the only reason.

This is the only substantial change of meaning I've noticed but it goes to show that professionals don't guarantee anything.

[Edited at 2010-02-23 11:51 GMT]


 

Kate Chaffer
Italy
Local time: 05:27
Member (2009)
Italian to English
My point exactly! Feb 23, 2010

Krzysztof Kajetanowicz wrote:

This is the only substantial change of meaning I've noticed but it goes to show that professionals don't guarantee anything


That was exactly my point. I would have expected more from professionals. That's why I said I was surprised. Out of 20 translators, 4 made serious mistakes (one translating the name of a software program, two making mistakes with numbers and one omitting an entire sentence).

We all make mistakes, but this was a short, not particularly complicated text where the participants knew what their translation was going to be used for. I don't think it would have been asking too much for them to proofread it before submitting it, whether they were being paid for it or not.

[Edited at 2010-02-23 12:00 GMT]


 

Tomás Cano Binder, BA, CT  Identity Verified
Spain
Local time: 05:27
Member (2005)
English to Spanish
+ ...
TOPIC STARTER
Not actual work Feb 23, 2010

Kate Chaffer wrote:
I don't think it would have been asking too much for them to proofread it before submitting it, whether they were being paid for it or not.

As I have expressed above, I think the picture would have changed completely in case this was paid work instead of a quick favour to a Proz friend in the midst of actual jobs and a busy schedule. I don't think it is fair to extrapolate and judge the results of this experiment as the result of an actual work situation.


 
Pages in topic:   [1 2 3 4 5] >


To report site rules violations or get help, contact a site moderator:


You can also contact site staff by submitting a support request »

Translation Iteration Experiment English-Spanish: the results!

Advanced search







SDL Trados Studio 2019 Freelance
The leading translation software used by over 250,000 translators.

SDL Trados Studio 2019 has evolved to bring translators a brand new experience. Designed with user experience at its core, Studio 2019 transforms how new users get up and running and helps experienced users make the most of the powerful features.

More info »
BaccS – Business Accounting Software
Modern desktop project management for freelance translators

BaccS makes it easy for translators to manage their projects, schedule tasks, create invoices, and view highly customizable reports. User-friendly, ProZ.com integration, community-driven development – a few reasons BaccS is trusted by translators!

More info »



Forums
  • All of ProZ.com
  • Term search
  • Jobs
  • Forums
  • Multiple search