Pages in topic:   [1 2] >
Feedback on the 9th Translation Contest: ideas for improvement
Thread poster: Silvia Prendin

Silvia Prendin  Identity Verified
Italy
Local time: 16:59
Member (2009)
English to Italian
+ ...
Oct 16, 2009

I would like to use this topic to gather your comments about the contest process, so that it can be improved. Yes, I think there's room for improvement.

So here's something to start with.

Suppose you have 60 entries for a given language pair (e.g. English to French), they will appear on your screen in random order. I'm not sure we all have the time to go through 60 entries... Anyway, the voters will pick some of them and vote. Now, this method won't guarantee that these 60 entries will have the same *chances* to pass the first voting round. Some of them might get to be read by 3 people only and consequently be turned down for lack of votes. The risk is: even if quality is good and the entry has what it takes to pass the round, it won't pass. On the other hand other entries getting more votes but questionable quality will pass.

In such case, the first thing I can think of would be making the number of votes visible for each entry, so as a voter, if I see that an entry got 50 votes and another one got 1 vote, I would read the latter and then vote.

What do you think? Other ideas/comments?


[Subject edited by staff or moderator 2009-10-16 11:23 GMT]


Direct link Reply with quote
 

Juan González Pérez
Spain
Local time: 16:59
Member (2008)
English to Spanish
+ ...
Translations to Spanish Oct 16, 2009

Hi Silvia,

Thanks for starting this thread, I love this contest and every idea that could improve it must be welcome

Here I go with my two cents: I think the contest could have some different categories for Spanish (i.e., European Spanish, Latin American Spanish...). It has been already done for other languages, and I consider it could be appropriate doing the same in this case.

Regards!


Direct link Reply with quote
 

IwonaASzymaniak  Identity Verified
United Kingdom
Local time: 15:59
English to Polish
+ ...
Number of reviewers Oct 16, 2009

Well, Silvia is right we do not have time to review all the translations.

Sometimes we do not have time to review more than one and decide not to vote at all (I have to admit I was so busy this time I didn't cast my vote).

Perhaps it would be good to keep track of both reviewers and voters, and then devise a sort of proportional formula to get the voting in the fair perspective and numbers.

I am not sure, however, if it is a good idea to make the number of voters visible to the voting public during the evaluation stage. Why? Simply becasue we are likely to "follow the flock" in our voting decisions.


Iwona

[Zmieniono 2009-10-16 10:47 GMT]


Direct link Reply with quote
 

Lianne Wilson
United Kingdom
Local time: 15:59
Japanese to English
+ ...
Translations to English Oct 16, 2009

At least in the French->English pair (I don't know about others) we had a lot of comments about texts sounding 'unamerican' or something like that. However, unless I am wrong, the to English translation contests do not specify whether to use American or British English, making such comments irrelevant.

Since no AE/BE preference is stated, I believe entries should not be criticised for using AE or BE terms or style in their translations.
So, I would suggest either:

A) making it more obvious that either AE or BE is fine and that comments against one or the other are not appropriate.
B) splitting the English competitions (at least those with enough entries) into AE and BE categories.
C) some other solution I'm too stoopid to think of right now


Direct link Reply with quote
 

Silvia Prendin  Identity Verified
Italy
Local time: 16:59
Member (2009)
English to Italian
+ ...
TOPIC STARTER
Thank you both Oct 16, 2009

for your comments and ideas. Yes, I think that two different categories for Spanish would be appropriate, and this would be advisable for other languages too.

About the number of voters, I think it has nothing to do with likes and dislikes, rather the chances to be read. An entry might have 30 votes and an average rating of 1.2!


Direct link Reply with quote
 

Silvia Prendin  Identity Verified
Italy
Local time: 16:59
Member (2009)
English to Italian
+ ...
TOPIC STARTER
Thank you Lianne! Oct 16, 2009

Yes, the same goes for UK/US.

Direct link Reply with quote
 

Gyula Jerkovich  Identity Verified
Hungary
Local time: 16:59
English to Hungarian
Screening? Oct 16, 2009

To address the question of too many entries and few votes to qualify them: in this case, maybe some kind of screening could work, e.g. if tagging would be required in the first round and items with above a certain no. of spelling mistakes or with 'mistranslation' tags, etc. would automatically not be qualified for the next round (i.e. a 'negative' qualification in the first round). This could help resolve the 'underrating' problem.

Direct link Reply with quote
 

Henry Dotterer
Local time: 10:59
SITE FOUNDER
Clarification Oct 16, 2009

Thanks for the ideas. Coming up with the best possible structure for contests is a fun challenge, and the job is definitely still underway. There are suggestions each time around that result in improvements to the contest structure, and I am sure this will continue for some time.

To clear up any possible misunderstandings:

* You don't have to rate all entries in a given language pair. Few people do.

* No entry has ever been eliminated due to lack of votes.

* Showing the number of raters (in the qualification round) is something that was suggested and implemented in the current contest.

Thanks!


Direct link Reply with quote
 

Henry Dotterer
Local time: 10:59
SITE FOUNDER
We have "ruling out" for that, Gyula Oct 16, 2009

Gyula Jerkovich wrote:

To address the question of too many entries and few votes to qualify them: in this case, maybe some kind of screening could work, e.g. if tagging would be required in the first round and items with above a certain no. of spelling mistakes or with 'mistranslation' tags, etc. would automatically not be qualified for the next round (i.e. a 'negative' qualification in the first round). This could help resolve the 'underrating' problem.

What we have along these lines is "ruling out". When it becomes highly improbable, mathematically, that a given entry will wind up among the top 7 entries in terms of average rating, it gets removed from contention. This works.


Direct link Reply with quote
 

Geraldine Oudin  Identity Verified
United Kingdom
Japanese to French
+ ...
Random order Oct 18, 2009

I think that the entries should appear in a random order, and that the order should change each time the section is accessed. Since most users (including myself) give up after rating the first 10 or 20, it would give a chance for all texts to be reviewed.

*****
http://geraldineoudin.com


Direct link Reply with quote
 

Silvia Prendin  Identity Verified
Italy
Local time: 16:59
Member (2009)
English to Italian
+ ...
TOPIC STARTER
Thank you Gyula, Henry and Geraldine Oct 18, 2009

I think the rules - as they are - are not clear enough. The steps taken are unknown.

1. How do you deal with an entry that was rated 4 just by one reader in the first round?(provided it was rated only once)

2. How do you deal with an entry that was rated 1 just by one reader in the first round?
(provided it was rated only once)

3. What if an entry is rated three times only?

4. Is there a minimum of ratings to pass the first round? Or can an entry pass with just two ratings for example?

5. Would it be possible to make the number of raters public in the first round? The number of ratings is now visible, but it is a sum of all ratings (not just the first round of course). If we have an idea about how many entries were rated and how many times, we could focus on the entries that received few ratings. I don't think such a step would take anything away from the other entries.

In short, I am thinking of a way to balance the distribution of ratings. This should not affect the rating in any way, because the quality is judged by the reader.

Geraldine, I know that entries are shown at random, but this is no guarantee that they will have equal chances to be read, especially if you consider that there might have been 30 raters in all for the first round. Also, as far as I am concerned, I rated the first entries on the list and the last ones, skipping the entries in the middle. So as you see, we cannot take anything for granted.


Direct link Reply with quote
 

Silvia Prendin  Identity Verified
Italy
Local time: 16:59
Member (2009)
English to Italian
+ ...
TOPIC STARTER
9th Contest Oct 18, 2009

Henry, could you please tell us how many readers rated entries in the first round for each language pair?
Also, what do you think about separating language variants?


Direct link Reply with quote
 

James_xia  Identity Verified
China
Member
English to Chinese
+ ...
To review part of the entries should be fine Oct 19, 2009

Restricting the time to consume, also to ensure the prudity and quality of the reviewing results, part of the entries can be reviewed instead of showing a comment on most or even the whole of the entries in a rush.

For a contest involving dozens of languages that covers 140 pairs, to organize an highly effective process is not as easy as it looks. Nonetheless, one thing is not for sure: I'm wonder why the participants were invited to give a comment on others.

Personally, when given the chance to make a comment by ticking 'like' or 'dislike', I simply did not know how to start but make the careful identification on a couple of obvious mistakes (if apprpriate), and dare not to provide a relatively comprehensive review on each entry.

Later I thought maybe this way is in the hope of not neglecting the true comments, not only for their participation, but also to ensure a fair contest to the maxium level.

Thanks!


Direct link Reply with quote
 

Silvia Prendin  Identity Verified
Italy
Local time: 16:59
Member (2009)
English to Italian
+ ...
TOPIC STARTER
Comments Oct 19, 2009

Thank you, James.

I think comments on mistakes are very important, but that's just my point of view. Some comments I saw were unsubstantiated, and there was nothing we could do about them: something else to think of too!


Direct link Reply with quote
 

Geraldine Oudin  Identity Verified
United Kingdom
Japanese to French
+ ...
Are you sure? Oct 19, 2009

Silvia P. wrote:

I know that entries are shown at random, but this is no guarantee that they will have equal chances to be read, especially if you consider that there might have been 30 raters in all for the first round. Also, as far as I am concerned, I rated the first entries on the list and the last ones, skipping the entries in the middle. So as you see, we cannot take anything for granted.


Are you sure about that? During the second round, there were still many entries in the pairs I was allowed to rate, and I rated them in several times. However, the order did not seem to change...


Direct link Reply with quote
 
Pages in topic:   [1 2] >


To report site rules violations or get help, contact a site moderator:

Moderator(s) of this forum
Jared Tabor[Call to this topic]

You can also contact site staff by submitting a support request »

Feedback on the 9th Translation Contest: ideas for improvement

Advanced search






memoQ translator pro
Kilgray's memoQ is the world's fastest developing integrated localization & translation environment rendering you more productive and efficient.

With our advanced file filters, unlimited language and advanced file support, memoQ translator pro has been designed for translators and reviewers who work on their own, with other translators or in team-based translation projects.

More info »
LSP.expert
You’re a freelance translator? LSP.expert helps you manage your daily translation jobs. It’s easy, fast and secure.

How about you start tracking translation jobs and sending invoices in minutes? You can also manage your clients and generate reports about your business activities. So you always keep a clear view on your planning, AND you get a free 30 day trial period!

More info »



Forums
  • All of ProZ.com
  • Term search
  • Jobs
  • Forums
  • Multiple search