Need help for Machine Translation Questionnaire

mediamatrix wrote:

I don't wish to be discourteous, or discourage worthwhile research, but I do think you would be well-advised to:

A Define your interpretation of 'machine translation' which means all things to all men (and women, of course...). Your questionnaire seems to be mixing many topics, some of which have little to do with MT as I understand it.

Ian Winick wrote:

Secondly, I think you'll find that language professionals such as ourselves are not the target group that use machine translation, for the same reason as taxi drivers tend not to drive Ladas.

Jeff Allen wrote:

I have read through the MT questionnaire carefully and have found all points to be fully relevant to the topic of Machine Translation (MT) and translation assessment measures of MT output.

I don't see what points seem to you to be inconsistent with the topic of MT.

mediamatrix wrote:
Thanks for your comments, Jeff. I hesitate to differ with an acknowleged expert in MT but, for what it's worth, here are just a few of the 'inconsistencies' I found on a first reading of the questionnaire, and now confirmed on a second and third reading, and having looked (again) at www.eamt.org:[/quote]





I recommend reading the following posts that I have previously made on this topic:

Definition of CAT
http://www.proz.com/post/184880#184880

&

MT is part of CAT & MT postediting is worthwhile
http://www.proz.com/post/189873#189873






This question does not seem to ask if these are MT tools or CAT tools, but rather if they are used as aids to MT.
On-line dictionaries are used for building MT dictionaries. Same with term banks and glossaries. Word processors are typically used for MT, and I specifically stated in my book review (http://www.multilingual.com/allen46.htm) concerning a book on the topic of MT postediting published in 2001 that it referred to MT processing and postediting techniques based on paper evaluation processes that were used in the late 1980s and early 1990s.
Even in the mid-1990s at Caterpillar, many of the translators using the MT systems did their assessment on print-out copies of the documents.
Translation Workstations have been part of the discussion of MT usage for many years. See:
http://cslu.cse.ogi.edu/HLTsurvey/ch8node6.html
http://ourworld.compuserve.com/homepages/WJHutchins/Conchist.htm
http://www.aymara.org/biblio/mtranslation.pdf
http://acl.ldc.upenn.edu/X/X96/X96-1024.pdf
http://ethesis.helsinki.fi/julkaisut/hum/yleis/vk/hoge/towardsa.pdf






I specifically described the difference between Content gisting (inbound translation, also referred to as assimilation in MT circles) and translation for publication (outbound Translation, also referred to as dissemination in MT circles) in the following:

Inbound vs Outbound MT systems and software
http://www.proz.com/post/190126#190126

And a description of each level of working with the MT output (known as MT postediting) is in my book chapter on the topic:
Post-editing. In Computers and Translation: A Translators Guide. Benjamins Translation Library, Number 35.
http://www.geocities.com/mtpostediting/PE-book-chapter-jeff-allen.pdf






On this point, your definition is closer to the one normally used. Since 2001, I have also started to refer to MT dictionary building as a pre-editing (or rather an MT pre-processing) step which can be done before MT processing or in parallel with it.




On this point, the answers show that she means "what are the key diffenciators, or selling points, for using MT in your context?"



Yes, objectives, targets, and measurements need to be set. In many cases, as I have written in a couple of articles, these are not set. Two of my current articles in progress and nearly ready to be submitted will deal with these issues in more detail.



It appears that she was taking the assessment criteria from standards like J2450 or Blackjack. I agree that additional criteria, like the one you mentioned, could be added. However, the criteria she listed are objective, measurable items.



"Flagrant errors of comprehension" is in itself a subjective evaluation-based statement, which can be determined based on assessing a combination of the measurable points she listed in that question. I understand that what you are trying to say is "this is junk output that I don't understand". But how do we evaluate "excellent translation" versus "very good translation" versus "good translation" versus "acceptable attempt" versus "half-way understandable" versus "total junk"? It is difficult to establish a value level for subjective criteria and use it consistently, unless such an item can be calculated from other measurable sub-criteria.

Thanks for your feedback and discussion on this topic. It is encouraging to see that you have taken the time to look at the questionnaire, and carefully consider it compared with other information available elsewhere on this general topic.

This also shows that surveys need to be carefully created and presented. My experience in creating and administering about 10 different language technology surveys confirms that they more time and effort invested up front to making the survey understandable and easy-to-fill-in will make the survey/questionnaire more valuable for gathering accurate and useful responses.

I hope that Agata can benefit from your comments.

Jeff


[Edited at 2006-04-16 22:49]

mediamatrix wrote:
What is more - as if to deny the validity of your own argument - one of the references you provided ( http://ourworld.compuserve.com/homepages/WJHutchins/Conchist.htm ) states, on the very first page:

"The term 'machine translation' (MT) refers to computerized systems responsible for the production of translations with or without human assistance. It EXCLUDES computer-based translation tools which support translators by providing access to on-line dictionaries, remote terminology databanks, transmission and reception of texts, etc. The boundaries between machine-aided human translation (MAHT) and human-aided machine translation (HAMT) are often uncertain and the term computer-aided translation (CAT) can cover both, but the central core of MT itself is the automation of the full translation process."

Another of your references ( http://cslu.cse.ogi.edu/HLTsurvey/ch8node6.html ) is specifically concerned with Machine-Assisted Human Translation (MAHT), which it equates with CAT, so it is again not very relevant with regard to the point you seem to be trying to reinforce which is that a word-processor is an MT aid. No! - at best a word-processor can be used as an MAHT aid.
....

The fact remains that a questionnaire about MT must include a definition of the scope of the survey (MT, MAHT, HAMT, CAT, etc.), it must stay within the bounds of the declared scope, and above it must all avoid confusing the issue with things like dumb word-processors.

mediamatrix wrote:
Another of your references ( http://cslu.cse.ogi.edu/HLTsurvey/ch8node6.html ) is specifically concerned with Machine-Assisted Human Translation (MAHT), which it equates with CAT, so it is again not very relevant with regard to the point you seem to be trying to reinforce which is that a word-processor is an MT aid. No! - at best a word-processor can be used as an MAHT aid.

Jeff Allen wrote:
It is difficult to establish a value level for subjective criteria and use it consistently, unless such an item can be calculated from other measurable sub-criteria.

mediamatrix wrote:

PS - Somewhat off topic, but far more interesting IMHO: You write:

There is an interesting comparison to be made here, I believe, between the subjective/objective assessment of the quality of translations (human or MT) and the quality of other forms of communication. In my specialist field, one of the main themes of study for many decades has been the search for objective measures for the subjective quality of TV sound and pictures. The result of this research has been the establishment and progressive refinement of some remarkably reliable, consistent and repeatable test methods that are recognised internationally in the broadcasting field, and indeed have been standardised for use world-wide. It seems to me that there might be some scope for the transposition of such methods into MT...