Pages in topic:   [1 2 3] >
KudoZ averages
Thread poster: James Calder

James Calder  Identity Verified
United Kingdom
Local time: 07:21
Member (2006)
Spanish to English
+ ...
Feb 9, 2005

Perhaps this subject has already been discussed but like other users of KudoZ I'm not entirely convinced that the total number of KudoZ points amassed by a translator is necessarily an accurate indicator of how good they are.

I think a fairer reflection of our ability would be to calculate a "KudoZ Average" by dividing the number of points won by the number of questions answered. This average could be displayed on our profile pages making it easier for askers and clients alike to assess our abilities.

I feel this system would make answering more of an exact science with people concerned more about maintaining their averages than firing off answers left, right and centre in the hope of getting as many points as possible.

My apologies in advance to responsible point hunters who choose their answers carefully and stick to their areas of expertise.

James


Direct link Reply with quote
 

Elvira Stoianov  Identity Verified
Luxembourg
Local time: 08:21
German to Romanian
+ ...
I think the subject was discussed in the past, but here's what I can think of first Feb 9, 2005

If you answer a question and it is not chosen by the asker, it does not necessarily mean your answer is incorrect (i.e. similar answers within a short time, several correct answers, out of which the asker can only choose one). If you don't answer, the help might be missed, if you answer, your average might go down, even if your answer was helpful.
So I am not in favour of the proposal.


Direct link Reply with quote
 

Nesrin  Identity Verified
United Kingdom
Local time: 07:21
English to Arabic
+ ...
Similar discussion Feb 9, 2005

HI James -

I raised a similar point and made a similar suggestion not so long ago - please take a look at the ensuing discussion http://www.proz.com/topic/28519

Regards,

Nesrin


Direct link Reply with quote
 

Angela Arnone  Identity Verified
Local time: 08:21
Member (2004)
Italian to English
+ ...
We have discussed this Feb 9, 2005

and I don't recall the outcome, but on the whole I tend to disagree, James.
I have often given a correct answer and the points have gone to someone else, lots of kudoz answers I have offered have never been graded. So I would object to my average dropping due to this mathematical application.
Moreover, the fact that people can acquire thousands of points (with all due respect) is no guarantee that they are good translators - it probably means they are good at research and that they are extremely generous in wanting to help others, as well as enjoying the challenge offered by the KudoZ part of the site.
Look at those language pairs where few questions get asked - does the fact that Jane Smith in the Tagalog-Urdu pair is not as good as me because she only has 6 points and I have many more? Not really. Just means I had more opportunity to answer.
KudoZ can be an aid to catching a potential customer's eye - but how many actually understand the mechanism behind it and how that part of the site works for us?
Thanks for starting an interesting topic, though!
Angela


Direct link Reply with quote
 
xxxCMJ_Trans
Local time: 08:21
French to English
+ ...
I launched a similar thread also not long ago...... Feb 9, 2005

http://www.proz.com/topic/27427


My problem with answering questions is that I am an addict!


Direct link Reply with quote
 

Nesrin  Identity Verified
United Kingdom
Local time: 07:21
English to Arabic
+ ...
Good argument for an average Feb 9, 2005

Angela Arnone wrote:

I have often given a correct answer and the points have gone to someone else, lots of kudoz answers I have offered have never been graded. So I would object to my average dropping due to this mathematical application.
Angela


Actually, when "wrong" answers are chosen over yours this also affects your total points (current system) - so even under the current system this is a frustrating phenomenon.


Look at those language pairs where few questions get asked - does the fact that Jane Smith in the Tagalog-Urdu pair is not as good as me because she only has 6 points and I have many more? Not really. Just means I had more opportunity to answer.


Precisely - that is a good argument FOR having an average rather than a "total points system" - that way, even someone with an unusual language combination can figure highly in the translators search, as that gives them the opportunity to have the same average as an English>German translator for example - something which is unthinkable under the current points system.
Personally, I am all for a change.


Direct link Reply with quote
 

Kirill Semenov  Identity Verified
Ukraine
Local time: 09:21
Member (2004)
English to Russian
+ ...
A long history of discussion of this topic Feb 9, 2005

Dear James,

As other have already said, the idea arises very often (and the fact itself shows it's worth something). I think that for two recent years this "reliability ratio" was proposed at least ten times. Being among those who also proposed it in the past, I support it wholeheartedly.

On my opinion, the most heated (but also rewarding!) discussion was this one:

http://www.proz.com/post/170063

You may a lot of pro-and-con arguments at the thread above -- not only that "I like/dislike it", but really well-based opinions of those who are interested in making the site better, not only to pursue their own ends. I highly recommend to read this old thread carefully.

NB. The thread contains some thoughts on how statistics works. For me as a former mathematician statistical laws nullify random affects of "wrong askers' choices" in the long run.

[Edited at 2005-02-09 14:54]


Direct link Reply with quote
 

writeaway  Identity Verified
French to English
+ ...
Answers chosen are just a personal decision by Asker Feb 9, 2005

I agree with Angela and Elvira.
Imho, there is no point at all in showing averages. The answer chosen is purely a personal decision by Asker, right, wrong or somewhere in between. Many times there is no single 'right' answer, it's just a matter of opinion. And when there is a specifically correct answer (in legal, technical, etc. question) and Asker goes for something completely wrong or closes question ignoring a correct answer with peer agrees by other experts in the field, then this reflects on Askers lack of skills in the particular field and as a translator in general. People seem to forget the transparency of the www. Numbers (points, averages etc.) simply don't or shouldn't matter.

[Edited at 2005-02-09 15:14]


Direct link Reply with quote
 

Jabberwock  Identity Verified
Poland
Local time: 08:21
Member (2004)
English to Polish
KudoZ is not a measure of translation skills! Feb 9, 2005

James Calder wrote:

...I'm not entirely convinced that the total number of KudoZ points amassed by a translator is necessarily an accurate indicator of how good they are.

...I think a fairer reflection of our ability...
James


KudoZ is not, and cannot be, a tool to measure a professional ability. No point system can change that. What KudoZ points do reflect well, I think, is the effort that many participants took in helping each other. If I see a person who has amassed 10000 points, it means he or she is a very active participant of ProZ, even if he or she receives points only for every second question.

Every statistic system may be abused. If the average system is adopted, then the easiest way to have the highest score is to select one rare question (preferably when the most answerers in the given pair are asleep), answer it and enjoy the one hundred percentage. Of course, providing answers in other questions would only jeopardize the average, so the person would be unwilling to risk it.

This would effectively eliminate answers from people who want to provide an alternative, or simply an explanation that take more than just the "agree/disagree" box. Literary translations would suffer most, as there are often no "good" answers, only interesting proposals. But also in other areas brainstorming terms on the KudoZ page is sometimes more beneficial for the Asker than just a dry valid answer. Remember also, that the Asker awards the most _helpful_answer, not necessarily the "correct" one.

As I see it now, having more answers, not necessarily all correct ones, is better than to have no answers at all. That is, introducing the averaging system would be in the long run detrimental for the askers.

KudoZ is a way of helping. Points are only a nice bonus...


Direct link Reply with quote
 

Nesrin  Identity Verified
United Kingdom
Local time: 07:21
English to Arabic
+ ...
Interesting thread Feb 9, 2005

Kirill Semenov wrote:

On my opinion, the most heated (but also rewarding!) discussion was this one:

http://www.proz.com/post/170063



Somebody should have pointed me to that thread when I made my suggestion last month - it really covers all aspects of the discussion.
Thanks
Nesrin


Direct link Reply with quote
 

Kirill Semenov  Identity Verified
Ukraine
Local time: 09:21
Member (2004)
English to Russian
+ ...
The ratio is already in the system Feb 9, 2005

Dear colleagues,

I just want to emphasize the point: the information on the reliability already IS in the system, and anyone who wants to see it has a simple way to retrieve it. To get the info one should: check the total number of kudoZ points accumulated by any proZians, then find any question asked by this proZian and see how many answers (s)he gave.

Thus, the only question is to make it directly visible or not.
Otherwise, _it's already here_ and easily calculated.


Direct link Reply with quote
 
Subhamay Ray  Identity Verified
Local time: 11:51
English to Bengali
+ ...
No indication of translation skills Feb 9, 2005

In fact my experience of closely monitoring the activities of several Kudoz pairs over the last four years makes me think that it should not be a criterion of ranking translators. In fact I fail to see why translators will have to be ranked like tennis players. It seems to me funny. Translation is a subjective exercise and if you really need to assess the skills of a translator, you'll have to go to his translations! And even that judgement will be subjective.

Direct link Reply with quote
 

James Calder  Identity Verified
United Kingdom
Local time: 07:21
Member (2006)
Spanish to English
+ ...
TOPIC STARTER
Make this information more visible/accessible Feb 9, 2005

Kirill Semenov wrote:

Dear colleagues,

I just want to emphasize the point: the information on the reliability already IS in the system, and anyone who wants to see it has a simple way to retrieve it. To get the info one should: check the total number of kudoZ points accumulated by any proZians, then find any question asked by this proZian and see how many answers (s)he gave.

Thus, the only question is to make it directly visible or not.
Otherwise, _it's already here_ and easily calculated.


What I'm arguing for is that this information should be made even more accessible for other users. Surely it wouldn't take much to provide an average/rating based on information that's already available.

I really can't see why people object to the proposal I and others before me have put forward. If you're going to have a points system and publish points tables in each language pair as well as tables of KudoZ leaders for the last 3 months etc. why not go the whole hog and publish a points/per question average?

The system of awarding points is undoubtedly flawed so what I suggest is either do away with it completely and not award points at all (a bit draconian, admittedly) or make the system as transparent and quality-oriented as possible. Obviously it all depends on how you view KudoZ: as a little bit of fun designed as a resource to help fellow translators in need (in which case, why award points?) or as a means of gauging expertise in a specific field/language pair, which is why our KudoZ points are broken down into specific categories, I presume.

Personally, I think it can and should combine both these functions. I also think it's a fantastic resource that we should try to make as professional, informative, relevant and as useful as possible. If that means making people think twice before they answer a question, then so be it. I, for one, won't feel discouraged from answering a question in my fields or language pair just because my average might go down if it's not selected.


Direct link Reply with quote
 

Kirill Semenov  Identity Verified
Ukraine
Local time: 09:21
Member (2004)
English to Russian
+ ...
Don't blame askers Feb 9, 2005

writeaway wrote:
The answer chosen is purely a personal decision by Asker, right, wrong or somewhere in between. Many times there is no single 'right' answer, it's just a matter of opinion.


Yes, sometimes askers _do_ mistake. But sometimes they do their mistakes in your favour. Don;t blame askers who are often confused by kudoZ grading interface (which is far from perfect), by their own biases or by their subjective opinions. Remember: askers are encouraged to choose not _the correct_ answer (which may be many), but _the most helpful answer_ (for them). Again, in the long run, fluctuations are nullified. If you read the link provided by me carefully you will see how specifically an individual answer/grade affects your "reliability ratio" depending on the general number of answers you gave. Statistics is not a empirical science, it's a real rule.

[Edited at 2005-02-09 18:16]


Direct link Reply with quote
 

writeaway  Identity Verified
French to English
+ ...
Not 'blaming' anyone; points are given to answer chosen as most helpful but how can that be ranked? Feb 9, 2005

Kirill Semenov wrote:

writeaway wrote:
The answer chosen is purely a personal decision by Asker, right, wrong or somewhere in between. Many times there is no single 'right' answer, it's just a matter of opinion.


Yes, sometimes askers _do_ mistake. But sometimes they do their mistakes in your favour. Don;t blame askers who are often confused by kudoZ grading interface (which is far from perfect), by their own biases or by their subjective opinions. Remember: askers are encouraged to choose not _the correct_ answer (which may be many), but _the most helpful answer_ (for them). Again, in the long run, fluctuations are nullified. If you read the link provided by me carefully you will see who specifically an individual answer/grade affects your "reliability ratio" depending on the general number of answers you gave. Statistics is not a empirical science, it's a real rule.


I think you may have misunderstood my incoherent English. Asker selects the answer he/she finds most useful, so it may even be based on additional references provided or some other criterion. My point is, that I don't see anyone can be 'ranked', unless it's according to their 'helpfulness'. There are often many excellent answers provided and the fact they were not chosen does not detract from the translation skills of those who answered. So why detract by some numerical system? Seems pointless (no pun (really) intended )

[Edited at 2005-02-09 17:58]


Direct link Reply with quote
 
Pages in topic:   [1 2 3] >


To report site rules violations or get help, contact a site moderator:


You can also contact site staff by submitting a support request »

KudoZ averages

Advanced search






CafeTran Espresso
You've never met a CAT tool this clever!

Translate faster & easier, using a sophisticated CAT tool built by a translator / developer. Accept jobs from clients who use SDL Trados, MemoQ, Wordfast & major CAT tools. Download and start using CafeTran Espresso -- for free

More info »
BaccS – Business Accounting Software
Modern desktop project management for freelance translators

BaccS makes it easy for translators to manage their projects, schedule tasks, create invoices, and view highly customizable reports. User-friendly, ProZ.com integration, community-driven development – a few reasons BaccS is trusted by translators!

More info »



Forums
  • All of ProZ.com
  • Term search
  • Jobs
  • Forums
  • Multiple search