Mobile menu

Pages in topic:   [1 2] >
Proposal for change of ranking calculation system
Thread poster: xxx------
xxx------
German to Spanish
+ ...
Oct 8, 2005

I don't know if this has been discussed before but I would like to suggest a new system for calculation of the ranking of kudoz answerers which might be fairer and more convincing. The number of kudoz points should be multiplied with the average of points obtained in relation with answered questions.
A peer with 100 questions answered and 200 points would multiply with 2 =400 points.
A peer with 200 questions anwered and 200 points would multiply with 1 = 200 Points.
A peer with 400 questions answered and 200 points would multiply with 0,5= 100 points
A peer with 50 questions answered and 200 points would multiply with 4 = 800 points
and so on...
That would increase quality a lot and would help peers who give less but better answers. Actually it is possible to be among the "top ten" of the month with 900 questions answered and only 600 points obtained. I think that is not the best publicity...
Best regards
Janfri

[Edited at 2005-10-08 19:59]


Direct link Reply with quote
 

Andy Watkinson
Spain
Local time: 22:35
Member
Catalan to English
+ ...
A necessary modification Oct 8, 2005

Hi Janfri,

I think you’ll find this has been discussed at length at various times in the past. A search for “reliability ratio” should return most of the discussions on the subject.

You’re obviously quite right.

Under the current system, answerers are rewarded for having provided the most helpful answer for the asker and, although the rationale of this has been questioned, that is what is in place at present.

On the other hand, there is no “official“ disincentive to firing off answers haphazardly at anything that moves.

I say “official” because the system itself does nothing to dissuade these people from answering questions for which they are clearly linguistically- and informationally-challenged. (Sorry about that…got carried away).

The only major disincentive (and if they had any sense they would realize this) is that their reputation as reliable answerers amongst their peers/possible clients must be close to zero.

Their kudoz ranking is as reliable as a fortune-tellers’.

“If I make enough predictions, I’m bound to get it right X percent of the time.”
“If I answer enough questions, ……..”

Quality would undoubtedly be improved as people would think twice about “damaging” their reputation (in the form of points) before firing off aother answer which is badly expressed, ignores/misunderstands the question, and is of more than dubitable reliability.

The reason why this has never been put into practice is apparently because it would be difficult to justify in certain language pairs or would in itself be misleading.

I don't see how this prevents it from being introduced in the "major" language pairs, though.

Cheers,
Andy


Direct link Reply with quote
 

Pablo Grosschmid  Identity Verified
Spain
Local time: 22:35
English to Spanish
+ ...
yes, and it could and should be be applied for all language pairs Oct 8, 2005

so it would not be discriminatory

However, since this would represent a major change of the rules, it should be introduced only after a certain date, and new points added to the previous ones, even if this may favour "older" members


Direct link Reply with quote
 
xxx------
German to Spanish
+ ...
TOPIC STARTER
Thank you. Oct 8, 2005

Thanks for the information Andy, I see that the “reliability ratio” has been broadly discussed in different threads before I joined proz. The number of kudoz points definitely says nothing if not compared to the number of questions answered. Until there is a better solution we will have to search the glossaries, practice mental arithmetic and execute calc.ex to find out more about the skills of the answerers.
Cheers+saludos desde Las Palmas
Janfri/Stefan


Direct link Reply with quote
 

Walter Landesman
Uruguay
Local time: 17:35
Member (2005)
English to Spanish
+ ...
I don`t know.... Oct 8, 2005

I am not sure about this proposal. Not that I disagree, but I`m not sure. Kudoz points are not always related to the right answers.
Many times, "the" right answer is given by two or more colleagues almost at the same time, but the Kudoz go to only one of them.
Many other times, answers are different but not wrong; it`s up to the asker to choose the most appropiate answer and hence the Kudoz points.
Sometimes - and we can see this in the glossaries - a wrong answer is accepted and given Kudoz- due to the rush or because the asker considers it to be more apppropiate than the right one or whatsoever.
I mention all this to explain why the ratio Questions-Answered/Points-Obtained wouldn`t be fair either. You can give many right answers and be selected just a few times.
I don`t know if I made myself understood. I don´t know......
Walter

[Edited at 2005-10-08 18:33]


Direct link Reply with quote
 
xxx------
German to Spanish
+ ...
TOPIC STARTER
That's not the normal situation, Walter. Oct 8, 2005

Walter Landesman wrote:
Questions-Answered/Points-Obtained wouldn`t be fair either. You can give many right answers and be selected just a few times.


[Edited at 2005-10-08 18:33]



There is often more than one good solution, so the propability exists that your correct answer might not be selected. But that is not the normal situation and among 1000 answers it is statistically not important. Anyway, with the multiplication system I propose, you still have no negative impact on your kudoz points if you "win" at least 25% of all your answers. I think it's a challenging risk. If there is already a good solution, why not agree and add just a peer comment with a new suggestion or better reference instead of making a new proposal which probably is not better.If answering is "expensive", we will think twice if it's worth to spend our kudoz money. If you are sure that your answer will be better than the given answer, good luck. For repeated and identical answers there should be the possibility of hiding the answer without counting it or maybe it could automatically be awarded the same points. There should also exist the possibility of "not for points" answers which should not count for the ranking. Point splitting or accepting several answers as correct would also be nice features.
Regards
Janfri


Direct link Reply with quote
 

Otilia Acosta  Identity Verified
Local time: 16:35
English to Spanish
+ ...
¡It sounds logical! Oct 8, 2005

I think that every idea to improve KudoZ is a great idea, but what is difficult is to put every suggestion into practice immediately. I strongly believe that KudoZ is undergoing a natural process of improvement, and in the future we will have a more perfect KudoZ system. I am sure ProZ’ staff pays to much attention to every suggestion to improve KudoZ, because KudoZ is one of the most attractive features of the Site.
Your idea sounds logical: to take into account not only KudoZ points, but also the number of answers given to get those points. By sure, it will affect the answers quality and professionalism in a positive way. I agree with you.
I hope your suggestion will be heard and taken into account.


Direct link Reply with quote
 
xxx------
German to Spanish
+ ...
TOPIC STARTER
It would favour old and new members Oct 8, 2005

Pablo Grosschmid wrote:


so it would not be discriminatory

However, since this would represent a major change of the rules, it should be introduced only after a certain date, and new points added to the previous ones, even if this may favour "older" members




I don't even think this would favour older members more than the new ones. IMHO kudoz should be a real open glossary and it should be possible that we also loose the points we have earned, if there are better answers after a question is graded. All glossary terms should be permanently revised and if necessary regraded. That could be a very interesting part of this game. The number of kudoz questions is down after the newest improvements. Why not reopen old questions and search the glossaries for errors? I am sure that would be a good way to clean up the database.
Cordiales saludos
Janfri


Direct link Reply with quote
 

Walter Landesman
Uruguay
Local time: 17:35
Member (2005)
English to Spanish
+ ...
Maybe Oct 8, 2005

Janfri wrote:

[
There is often more than one good solution, so the propability exists that your correct answer might not be selected. But that is not the normal situation and among 1000 answers it is statistically not important. Anyway, with the multiplication system I propose, you still have no negative impact on your kudoz points if you "win" at least 25% of all your answers. I think it's a challenging risk. If there is already a good solution, why not agree and add just a peer comment with a new suggestion or better reference instead of making a new proposal which probably is not better.If answering is "expensive", we will think twice if it's worth to spend our kudoz money. If you are sure that your answer will be better than the given answer, good luck. For repeated and identical answers there should be the possibility of hiding the answer without counting it or maybe it could automatically be awarded the same points. There should also exist the possibility of "not for points" answers which should not count for the ranking. Point splitting or accepting several answers as correct would also be nice features.
Regards
Janfri

Yes, it sounds more reasonable now. May be. Let`s work this idea further.

[Edited at 2005-10-08 20:50]


Direct link Reply with quote
 

Kim Metzger  Identity Verified
Mexico
Local time: 15:35
German to English
KudoZ ranking system Oct 8, 2005

Janfri wrote:

There should also exist the possibility of "not for points" answers which should not count for the ranking.


That's a good idea, Janfri. ASKERS can now specify that their QUESTIONS are not for points. The idea behind this change, I believe, was to see what happens in a no-points environment, i.e. whether the quality of answers would improve. But few askers select the not-for-points option, and it's my impression that those who do select this option often don't know what they're doing, i.e. they are askers who are new to KudoZ, haven't read the instructions and don't even know what KudoZ points are all about.

One of the reasons I proposed a "no-points" option long ago was to counter the mad rush to enter just any old answer before anyone else had a chance, to do something about the "race" for points instead of thoughtful, carefully considered proposals that sometimes take more than just two or three minutes to come up with.

There are many occasions, for instance, when what is really needed to start off with is a good definition of a difficult source term and/or target term rather than a quick translation proposal. I've seen many translation proposals that are carelessly and even dishonestly "winging it" because the answerer isn't even sure he knows what the source term means and may well know absolutely nothing about the subject.

At this point it is helpful to the KudoZ process to submit not a translation but a monolingual definition so that others will more likely go down the right path with their proposals. If some sort of reliability feature is ever introduced to KudoZ, we wouldn't want to punish people who submit definitions instead of translations because these are the people who have the outcome for the asker and the KOG in mind rather than the points. We would need a way for answerers to opt out of the points system.


Direct link Reply with quote
 
xxx------
German to Spanish
+ ...
TOPIC STARTER
Space for out of the points system, but with points Oct 8, 2005

Kim Metzger wrote:

ASKERS can now specify that their QUESTIONS are not for points. The idea behind this change, I believe, was to see what happens in a no-points environment, i.e. whether the quality of answers would improve. ....
we wouldn't want to punish people who submit definitions instead of translations because these are the people who have the outcome for the asker and the KOG in mind rather than the points. We would need a way for answerers to opt out of the points system.


Thank's for your contribution, Kim! I referred to a "not for points" option for answerers. I often make proposals "not for points" because of the space limits for texts of peer comments.Sometimes you want to add a reference or a larger comment and there is no space for that. This should not be negative for the overall kudoz-average.
I totally agree with the point "we would need a way for answerers to opt out of the points system". I was thinking about a kind of a simultaneous chat function for all questions or a space for general opinions, where peers could discuss directly about a question. Like the actual "ask the asker" feature, but with more space for any type of aditional information. As I said to Pablo, I think there could and should be a second and third stage of glossary revision. Slow answerers but good investigators could then find better solutions for tough questions and earn kudoz points when the question is regraded.So they could stay outside the point system but earn points at the same time.
Best regards
Janfri


Direct link Reply with quote
 
xxxMihai Badea  Identity Verified
Luxembourg
Member (2004)
English to Romanian
+ ...
Ranking system for the Freelancers Directory Oct 9, 2005

Janfri wrote:

I don't know if this has been discussed before but I would like to suggest a new system for calculation of the ranking of kudoz answerers which might be fairer and more convincing. The number of kudoz points should be multiplied with the average of points obtained in relation with answered questions.
A peer with 100 questions answered and 200 points would multiply with 2 =400 points.
A peer with 200 questions anwered and 200 points would multiply with 1 = 200 Points.
A peer with 400 questions answered and 200 points would multiply with 0,5= 100 points
A peer with 50 questions answered and 200 points would multiply with 4 = 800 points
and so on...


Dear Janfri,

I am inclined not to agree with this system as it would increase the gap between the old members and the new ones. Let’s take one hypothetical example:

In the actual system:
Old member: 200 points (100 questions answered)
New member: 20 points (10 questions answered)
Gap: 180 points

In your proposed system:
Old member: 400 points (100 questions answered)
New member: 40 points (10 questions answered)
Gap: 360 points

But I agree with you that the system we have now can be improved. I like the idea of the "reliability ratio", but I understood that, for some reasons, it cannot be implemented.

Another problem with the current system is that the gap between old members and new members is naturally increasing. I think the new members may find this system unfair and, in some sort, it is actually unfair. It is nobody's fault, but, in my opinion, something should be changed.

Let’s think about what happened with the ATP ranking system. Pete Sampras (a great player!) could still be number one even if he took a 1-year vacation, but this was not fair for the other players and the system was changed.

Resetting the ranking system to zero yearly would probably not be a good idea in our case, but I think that ranking the members according to the points won in the previous year may be feasible.

[Edited at 2005-10-09 13:37]


Direct link Reply with quote
 

Walter Landesman
Uruguay
Local time: 17:35
Member (2005)
English to Spanish
+ ...
ranking last 12 months Oct 9, 2005

Mihai Badea wrote:
I am inclined not to agree with this system as it would increase the gap between the old members and the new ones.

But I agree with you that the system we have now can be improved. I like the idea of the "reliability ratio", but I understood that, for some reasons, it cannot be implemented.

Another problem with the current system is that the gap between old members and new members is naturally increasing. I think the new members may find this system unfair and, in some sort, it is actually unfair. It’s nobody’s fault, but, in my opinion, something should be changed.

Let’s think about what happened with the ATP ranking system. Pete Sampras (a great player!) could still be number one even if he took a 1-year vacation, but this was not fair for the other players and the system was changed.

Resetting the ranking system to zero yearly would probably not be a good idea in our case, but I think that ranking the members according to the points won in the previous year may be feasible.

[Edited at 2005-10-09 12:54]


Hi Mihai,

The previous year. Last twelve monts. Yes, I suggested this a few months ago too. Maybe keeping a percentage of kudoz earned up to then, for the sake of old members, as a compensation, just in case somebody took a vacation or was sick or something. Special reward. But ranking based in the last twelve months is a good choice.
When I suggested it I didn´t get any feedback, though. Wish you better luck.

Walter


Direct link Reply with quote
 
xxx------
German to Spanish
+ ...
TOPIC STARTER
It really depends Oct 9, 2005

Hi Mihai,
thank's for commenting.

[/quote]

I am inclined not to agree with this system as it would increase the gap between the old members, with a lot of points, and the new ones. [/quote]

It depends, this varies a lot from member to member. Some old members would benefit and many others would lose their position.

[/quote]I think the new members may find this system unfair and, in some sort, it is actually unfair. It’s nobody’s fault, but, in my opinion, something should be changed. [/quote]

I agree with that point. I also found it a bit unfair when I started participating.In the beginning I lost points which I deserved because nobody knew me and the hierarchy was already established. Nowadays sometimes I am awarded points which I do not deserve, just because I am one of the leaders in my combination. I have written in another thread that I would prefer anonymous answering, but this has been previously discarded for different reasons. We actually have three different rankings and in this sense old and new members have equal conditions.
Best regards
Janfri


Direct link Reply with quote
 
xxxMihai Badea  Identity Verified
Luxembourg
Member (2004)
English to Romanian
+ ...
Leaders rankings vs. Freelancers/Agencies Directory Oct 9, 2005

Walter Landesman wrote:

Hi Mihai,

The previous year. Last twelve monts. Yes, I suggested this a few months ago too. Maybe keeping a percentage of kudoz earned up to then, for the sake of old members, as a compensation, just in case somebody took a vacation or was sick or something. Special reward. But ranking based in the last twelve months is a good choice.
When I suggested it I didn´t get any feedback, though. Wish you better luck.

Walter


Dear Walter,

I agree with you that the old members should be rewarded for their continued support of this site. Without them, this site would not have been what it is now: a wonderful workplace for the translators worldwide. I think keeping a percentage of KudoZ earned up to then is an excellent idea.


Janfri wrote:

We actually have three different rankings and in this sense old and new members have equal conditions.
Best regards
Janfri


Dear Janfri,

I guess you are referring to the Leaders rankings (Last 3 Months - Last 12 Months - All Time ). Well, I think the clients are supposed to find us mainly through the Freelancers/Agencies Directory and that Directory is based on the overall points.


Direct link Reply with quote
 
Pages in topic:   [1 2] >


To report site rules violations or get help, contact a site moderator:


You can also contact site staff by submitting a support request »

Proposal for change of ranking calculation system

Advanced search






TM-Town
Manage your TMs and Terms ... and boost your translation business

Are you ready for something fresh in the industry? TM-Town is a unique new site for you -- the freelance translator -- to store, manage and share translation memories (TMs) and glossaries...and potentially meet new clients on the basis of your prior work.

More info »
Across v6.3
Translation Toolkit and Sales Potential under One Roof

Apart from features that enable you to translate more efficiently, the new Across Translator Edition v6.3 comprises your crossMarket membership. The new online network for Across users assists you in exploring new sales potential and generating revenue.

More info »



All of ProZ.com
  • All of ProZ.com
  • Term search
  • Jobs