https://www.proz.com/forum/post_editing_machine_translation/306068-the_basics_of_quality_estimation.html
This site uses cookies. Some of these cookies are essential to the operation of the site, while others help to improve your experience by providing insights into how the site is being used. For more information, please see the ProZ.com privacy policy.
Cookie settings

Translation - art & business » Post-editing & Machine Translation »
The Basics of Quality Estimation
Thread poster: Juan Martín Fernández Rowda
Juan Martín Fernández Rowda
Juan Martín Fernández Rowda  Identity Verified
United States
Local time: 18:29
English to Spanish
+ ...
Sep 1, 2016

Sharing one of my posts recently published on the GALA blog for those interested in quality estimation:

https://www.gala-global.org/blog/basics-quality-estimation#sthash.Lzyi78eK.Dj7Ydxt5.dpbs
 
Samuel Murray
Samuel Murray  Identity Verified
Netherlands
Local time: 03:29
Member (2006)
English to Afrikaans
+ ...
@Juan Sep 1, 2016

Juan Martín Fernández Rowda wrote:
Sharing one of my posts recently published on the GALA blog for those interested in quality estimation:
https://www.gala-global.org/blog/basics-quality-estimation#sthash.Lzyi78eK.Dj7Ydxt5.dpbs


So, if I understand correctly, the QE designer would figure out which things make a sentence less likely to be translated well, based on the language combination. Right?

So for example, if it is known that sentences in Upper Slobovian that contain many nouns will likely produce a poor translation in Mid Slobovian but likely produce a reasonably accurate translation in Lower Slobovian, then such sentences would be graded with a low QE score if the target language is Mid Slobovian (which means that they'll go to the top of the pile of segments to be checked by a human) but a high QE score if the target language is Lower Slobovian (which means they'll move to the bottom of that pile). Do I understand this correctly?

So QE is not an evaluation of an existing translation by comparing it with a human translation, but a guess about the quality of the translation based on the elements of sentences in a given language combination. Yes?
 
Juan Martín Fernández Rowda
Juan Martín Fernández Rowda  Identity Verified
United States
Local time: 18:29
English to Spanish
+ ...
TOPIC STARTER
@ Samuel Sep 1, 2016

Samuel Murray wrote:

So, if I understand correctly, the QE designer would figure out which things make a sentence less likely to be translated well, based on the language combination. Right?

So for example, if it is known that sentences in Upper Slobovian that contain many nouns will likely produce a poor translation in Mid Slobovian but likely produce a reasonably accurate translation in Lower Slobovian, then such sentences would be graded with a low QE score if the target language is Mid Slobovian (which means that they'll go to the top of the pile of segments to be checked by a human) but a high QE score if the target language is Lower Slobovian (which means they'll move to the bottom of that pile). Do I understand this correctly?

So QE is not an evaluation of an existing translation by comparing it with a human translation, but a guess about the quality of the translation based on the elements of sentences in a given language combination. Yes?



Thanks for your interest, Samuel. Here's a very simplistic explanation: QE is based on language combination, yes. Now, the QE designer, so to call it, is usually a scientist - not a lot of linguists can deal with this. The main reason is that it's a complex process. There are standard set of features (the features are what indicate the potential presence of an issue in a sentence) and there are more specific ones. Knowing which features you want to use, a QE model is created using supervised machine learning on parallel data (source, target). So these features are "learned" from already translated text. That model can then be create to estimate the quality of other texts for which you don't have a reference translation.

What you say is correct - QE is only a prediction of the quality, not an evaluation of it. However, we always need to keep in mind that with language, 1+1 is not always 2. I did some work on quality estimation based only on linguistic features, with the sole purpose of making QE more accessible to translators and linguists. I think I posted links in this forum.

If you want to learn more, let me recommend some papers:
Machine Translation Quality Estimation Across Domains - http://www.aclweb.org/anthology/C14-1040
Confidence estimation for Machine Translation - http://web.eecs.umich.edu/~kulesza/pubs/confest_coling04.pdf
Estimating the Sentence-Level Quality of Machine Translation Systems - http://www.mt-archive.info/EAMT-2009-Specia.pdf
 


There is no moderator assigned specifically to this forum.
To report site rules violations or get help, please contact site staff »


The Basics of Quality Estimation


Translation news
» Translation trends at the London Book Fair: Japanese are overtaking the UK market, translations from Ukrainian are on the rise
(0 comments)
» Survey finds generative AI proving major threat to the work of translators
(0 comments)
» New ISO Standard 5060 Focuses on Human Evaluation to Ensure Translation Quality
(0 comments)


Forum rules
Help and orientation




Protemos translation business management system
Create your account in minutes, and start working! 3-month trial for agencies, and free for freelancers!

The system lets you keep client/vendor database, with contacts and rates, manage projects and assign jobs to vendors, issue invoices, track payments, store and manage project files, generate business reports on turnover profit per client/manager etc.

More info »
Trados Business Manager Lite
Create customer quotes and invoices from within Trados Studio

Trados Business Manager Lite helps to simplify and speed up some of the daily tasks, such as invoicing and reporting, associated with running your freelance translation business.

More info »