| Pages in topic: [1 2] > |
Proposal for change of ranking calculation system Thread poster: ------ (X)
|
|---|
------ (X)
German to Spanish
+ ...
I don't know if this has been discussed before but I would like to suggest a new system for calculation of the ranking of kudoz answerers which might be fairer and more convincing. The number of kudoz points should be multiplied with the average of points obtained in relation with answered questions.
A peer with 100 questions answered and 200 points would multiply with 2 =400 points.
A peer with 200 questions anwered and 200 points would multiply with 1 = 200 Points.
A peer wit... See more I don't know if this has been discussed before but I would like to suggest a new system for calculation of the ranking of kudoz answerers which might be fairer and more convincing. The number of kudoz points should be multiplied with the average of points obtained in relation with answered questions.
A peer with 100 questions answered and 200 points would multiply with 2 =400 points.
A peer with 200 questions anwered and 200 points would multiply with 1 = 200 Points.
A peer with 400 questions answered and 200 points would multiply with 0,5= 100 points
A peer with 50 questions answered and 200 points would multiply with 4 = 800 points
and so on...
That would increase quality a lot and would help peers who give less but better answers. Actually it is possible to be among the "top ten" of the month with 900 questions answered and only 600 points obtained. I think that is not the best publicity...
Best regards
Janfri
[Edited at 2005-10-08 19:59] ▲ Collapse
| | | |
Andy Watkinson 
Spain
Local time: 14:42
Member
Catalan to English
+ ...
| A necessary modification | Oct 8, 2005 |
Hi Janfri,
I think you’ll find this has been discussed at length at various times in the past. A search for “reliability ratio” should return most of the discussions on the subject.
You’re obviously quite right.
Under the current system, answerers are rewarded for having provided the most helpful answer for the asker and, although the rationale of this has been questioned, that is what is in place at present.
On the other hand, there is... See more Hi Janfri,
I think you’ll find this has been discussed at length at various times in the past. A search for “reliability ratio” should return most of the discussions on the subject.
You’re obviously quite right.
Under the current system, answerers are rewarded for having provided the most helpful answer for the asker and, although the rationale of this has been questioned, that is what is in place at present.
On the other hand, there is no “official“ disincentive to firing off answers haphazardly at anything that moves.
I say “official” because the system itself does nothing to dissuade these people from answering questions for which they are clearly linguistically- and informationally-challenged. (Sorry about that…got carried away).
The only major disincentive (and if they had any sense they would realize this) is that their reputation as reliable answerers amongst their peers/possible clients must be close to zero.
Their kudoz ranking is as reliable as a fortune-tellers’.
“If I make enough predictions, I’m bound to get it right X percent of the time.”
“If I answer enough questions, ……..”
Quality would undoubtedly be improved as people would think twice about “damaging” their reputation (in the form of points) before firing off aother answer which is badly expressed, ignores/misunderstands the question, and is of more than dubitable reliability.
The reason why this has never been put into practice is apparently because it would be difficult to justify in certain language pairs or would in itself be misleading.
I don't see how this prevents it from being introduced in the "major" language pairs, though.
Cheers,
Andy ▲ Collapse
| | | |
| yes, and it could and should be be applied for all language pairs | Oct 8, 2005 |
so it would not be discriminatory
However, since this would represent a major change of the rules, it should be introduced only after a certain date, and new points added to the previous ones, even if this may favour "older" members
| | | |
------ (X)
German to Spanish
+ ...
TOPIC STARTER
Thanks for the information Andy, I see that the “reliability ratio” has been broadly discussed in different threads before I joined proz. The number of kudoz points definitely says nothing if not compared to the number of questions answered. Until there is a better solution we will have to search the glossaries, practice mental arithmetic and execute calc.ex to find out more about the skills of the answerers.
Cheers+saludos desde Las Palmas
Janfri/Stefan
| | |
|
|
|
| I don`t know.... | Oct 8, 2005 |
I am not sure about this proposal. Not that I disagree, but I`m not sure. Kudoz points are not always related to the right answers.
Many times, "the" right answer is given by two or more colleagues almost at the same time, but the Kudoz go to only one of them.
Many other times, answers are different but not wrong; it`s up to the asker to choose the most appropiate answer and hence the Kudoz points.
Sometimes - and we can see this in the glossaries - a wrong answer is accept... See more I am not sure about this proposal. Not that I disagree, but I`m not sure. Kudoz points are not always related to the right answers.
Many times, "the" right answer is given by two or more colleagues almost at the same time, but the Kudoz go to only one of them.
Many other times, answers are different but not wrong; it`s up to the asker to choose the most appropiate answer and hence the Kudoz points.
Sometimes - and we can see this in the glossaries - a wrong answer is accepted and given Kudoz- due to the rush or because the asker considers it to be more apppropiate than the right one or whatsoever.
I mention all this to explain why the ratio Questions-Answered/Points-Obtained wouldn`t be fair either. You can give many right answers and be selected just a few times.
I don`t know if I made myself understood. I don´t know......
Walter
[Edited at 2005-10-08 18:33] ▲ Collapse
| | | |
------ (X)
German to Spanish
+ ...
TOPIC STARTER | That's not the normal situation, Walter. | Oct 8, 2005 |
Walter Landesman wrote:
Questions-Answered/Points-Obtained wouldn`t be fair either. You can give many right answers and be selected just a few times.
[Edited at 2005-10-08 18:33]
There is often more than one good solution, so the propability exists that your correct answer might not be selected. But that is not the normal situation and among 1000 answers it is statistically not important. Anyway, with the multiplication system I propose, you still have no negative impact on your kudoz points if you "win" at least 25% of all your answers. I think it's a challenging risk. If there is already a good solution, why not agree and add just a peer comment with a new suggestion or better reference instead of making a new proposal which probably is not better.If answering is "expensive", we will think twice if it's worth to spend our kudoz money. If you are sure that your answer will be better than the given answer, good luck. For repeated and identical answers there should be the possibility of hiding the answer without counting it or maybe it could automatically be awarded the same points. There should also exist the possibility of "not for points" answers which should not count for the ranking. Point splitting or accepting several answers as correct would also be nice features.
Regards
Janfri
| | | |
| ¡It sounds logical! | Oct 8, 2005 |
I think that every idea to improve KudoZ is a great idea, but what is difficult is to put every suggestion into practice immediately. I strongly believe that KudoZ is undergoing a natural process of improvement, and in the future we will have a more perfect KudoZ system. I am sure ProZ’ staff pays to much attention to every suggestion to improve KudoZ, because KudoZ is one of the most attractive features of the Site.
Your idea sounds logical: to take into account not only KudoZ points, b... See more I think that every idea to improve KudoZ is a great idea, but what is difficult is to put every suggestion into practice immediately. I strongly believe that KudoZ is undergoing a natural process of improvement, and in the future we will have a more perfect KudoZ system. I am sure ProZ’ staff pays to much attention to every suggestion to improve KudoZ, because KudoZ is one of the most attractive features of the Site.
Your idea sounds logical: to take into account not only KudoZ points, but also the number of answers given to get those points. By sure, it will affect the answers quality and professionalism in a positive way. I agree with you.
I hope your suggestion will be heard and taken into account. ▲ Collapse
| | | |
------ (X)
German to Spanish
+ ...
TOPIC STARTER | It would favour old and new members | Oct 8, 2005 |
Pablo Grosschmid wrote:
so it would not be discriminatory
However, since this would represent a major change of the rules, it should be introduced only after a certain date, and new points added to the previous ones, even if this may favour "older" members
I don't even think this would favour older members more than the new ones. IMHO kudoz should be a real open glossary and it should be possible that we also loose the points we have earned, if there are better answers after a question is graded. All glossary terms should be permanently revised and if necessary regraded. That could be a very interesting part of this game. The number of kudoz questions is down after the newest improvements. Why not reopen old questions and search the glossaries for errors? I am sure that would be a good way to clean up the database.
Cordiales saludos
Janfri
| | |
|
|
|
Janfri wrote:
[
There is often more than one good solution, so the propability exists that your correct answer might not be selected. But that is not the normal situation and among 1000 answers it is statistically not important. Anyway, with the multiplication system I propose, you still have no negative impact on your kudoz points if you "win" at least 25% of all your answers. I think it's a challenging risk. If there is already a good solution, why not agree and add just a peer comment with a new suggestion or better reference instead of making a new proposal which probably is not better.If answering is "expensive", we will think twice if it's worth to spend our kudoz money. If you are sure that your answer will be better than the given answer, good luck. For repeated and identical answers there should be the possibility of hiding the answer without counting it or maybe it could automatically be awarded the same points. There should also exist the possibility of "not for points" answers which should not count for the ranking. Point splitting or accepting several answers as correct would also be nice features.
Regards
Janfri
Yes, it sounds more reasonable now. May be. Let`s work this idea further.
[Edited at 2005-10-08 20:50]
| | | |
| KudoZ ranking system | Oct 8, 2005 |
Janfri wrote:
There should also exist the possibility of "not for points" answers which should not count for the ranking.
That's a good idea, Janfri. ASKERS can now specify that their QUESTIONS are not for points. The idea behind this change, I believe, was to see what happens in a no-points environment, i.e. whether the quality of answers would improve. But few askers select the not-for-points option, and it's my impression that those who do select this option often don't know what they're doing, i.e. they are askers who are new to KudoZ, haven't read the instructions and don't even know what KudoZ points are all about.
One of the reasons I proposed a "no-points" option long ago was to counter the mad rush to enter just any old answer before anyone else had a chance, to do something about the "race" for points instead of thoughtful, carefully considered proposals that sometimes take more than just two or three minutes to come up with.
There are many occasions, for instance, when what is really needed to start off with is a good definition of a difficult source term and/or target term rather than a quick translation proposal. I've seen many translation proposals that are carelessly and even dishonestly "winging it" because the answerer isn't even sure he knows what the source term means and may well know absolutely nothing about the subject.
At this point it is helpful to the KudoZ process to submit not a translation but a monolingual definition so that others will more likely go down the right path with their proposals. If some sort of reliability feature is ever introduced to KudoZ, we wouldn't want to punish people who submit definitions instead of translations because these are the people who have the outcome for the asker and the KOG in mind rather than the points. We would need a way for answerers to opt out of the points system.
| | | |
------ (X)
German to Spanish
+ ...
TOPIC STARTER | Space for out of the points system, but with points | Oct 8, 2005 |
Kim Metzger wrote:
ASKERS can now specify that their QUESTIONS are not for points. The idea behind this change, I believe, was to see what happens in a no-points environment, i.e. whether the quality of answers would improve. ....
we wouldn't want to punish people who submit definitions instead of translations because these are the people who have the outcome for the asker and the KOG in mind rather than the points. We would need a way for answerers to opt out of the points system.
Thank's for your contribution, Kim! I referred to a "not for points" option for answerers. I often make proposals "not for points" because of the space limits for texts of peer comments.Sometimes you want to add a reference or a larger comment and there is no space for that. This should not be negative for the overall kudoz-average.
I totally agree with the point "we would need a way for answerers to opt out of the points system". I was thinking about a kind of a simultaneous chat function for all questions or a space for general opinions, where peers could discuss directly about a question. Like the actual "ask the asker" feature, but with more space for any type of aditional information. As I said to Pablo, I think there could and should be a second and third stage of glossary revision. Slow answerers but good investigators could then find better solutions for tough questions and earn kudoz points when the question is regraded.So they could stay outside the point system but earn points at the same time.
Best regards
Janfri
| | | |
Mihai Badea (X)
Luxembourg
English to Romanian
+ ...
| Ranking system for the Freelancers Directory | Oct 9, 2005 |
Janfri wrote:
I don't know if this has been discussed before but I would like to suggest a new system for calculation of the ranking of kudoz answerers which might be fairer and more convincing. The number of kudoz points should be multiplied with the average of points obtained in relation with answered questions.
A peer with 100 questions answered and 200 points would multiply with 2 =400 points.
A peer with 200 questions anwered and 200 points would multiply with 1 = 200 Points.
A peer with 400 questions answered and 200 points would multiply with 0,5= 100 points
A peer with 50 questions answered and 200 points would multiply with 4 = 800 points
and so on...
Dear Janfri,
I am inclined not to agree with this system as it would increase the gap between the old members and the new ones. Let’s take one hypothetical example:
In the actual system:
Old member: 200 points (100 questions answered)
New member: 20 points (10 questions answered)
Gap: 180 points
In your proposed system:
Old member: 400 points (100 questions answered)
New member: 40 points (10 questions answered)
Gap: 360 points
But I agree with you that the system we have now can be improved. I like the idea of the "reliability ratio", but I understood that, for some reasons, it cannot be implemented.
Another problem with the current system is that the gap between old members and new members is naturally increasing. I think the new members may find this system unfair and, in some sort, it is actually unfair. It is nobody's fault, but, in my opinion, something should be changed.
Let’s think about what happened with the ATP ranking system. Pete Sampras (a great player!) could still be number one even if he took a 1-year vacation, but this was not fair for the other players and the system was changed.
Resetting the ranking system to zero yearly would probably not be a good idea in our case, but I think that ranking the members according to the points won in the previous year may be feasible.
[Edited at 2005-10-09 13:37]
| | |
|
|
|
| ranking last 12 months | Oct 9, 2005 |
Mihai Badea wrote:
I am inclined not to agree with this system as it would increase the gap between the old members and the new ones.
But I agree with you that the system we have now can be improved. I like the idea of the "reliability ratio", but I understood that, for some reasons, it cannot be implemented.
Another problem with the current system is that the gap between old members and new members is naturally increasing. I think the new members may find this system unfair and, in some sort, it is actually unfair. It’s nobody’s fault, but, in my opinion, something should be changed.
Let’s think about what happened with the ATP ranking system. Pete Sampras (a great player!) could still be number one even if he took a 1-year vacation, but this was not fair for the other players and the system was changed.
Resetting the ranking system to zero yearly would probably not be a good idea in our case, but I think that ranking the members according to the points won in the previous year may be feasible.
[Edited at 2005-10-09 12:54]
Hi Mihai,
The previous year. Last twelve monts. Yes, I suggested this a few months ago too. Maybe keeping a percentage of kudoz earned up to then, for the sake of old members, as a compensation, just in case somebody took a vacation or was sick or something. Special reward. But ranking based in the last twelve months is a good choice.
When I suggested it I didn´t get any feedback, though. Wish you better luck.
Walter
| | | |
------ (X)
German to Spanish
+ ...
TOPIC STARTER | It really depends | Oct 9, 2005 |
Hi Mihai,
thank's for commenting.
[/quote]
I am inclined not to agree with this system as it would increase the gap between the old members, with a lot of points, and the new ones. [/quote]
It depends, this varies a lot from member to member. Some old members would benefit and many others would lose their position.
[/quote]I think the new members may find this system unfair and, in some sort, it is actually unfair. It’s nobody’s fault... See more Hi Mihai,
thank's for commenting.
[/quote]
I am inclined not to agree with this system as it would increase the gap between the old members, with a lot of points, and the new ones. [/quote]
It depends, this varies a lot from member to member. Some old members would benefit and many others would lose their position.
[/quote]I think the new members may find this system unfair and, in some sort, it is actually unfair. It’s nobody’s fault, but, in my opinion, something should be changed. [/quote]
I agree with that point. I also found it a bit unfair when I started participating.In the beginning I lost points which I deserved because nobody knew me and the hierarchy was already established. Nowadays sometimes I am awarded points which I do not deserve, just because I am one of the leaders in my combination. I have written in another thread that I would prefer anonymous answering, but this has been previously discarded for different reasons. We actually have three different rankings and in this sense old and new members have equal conditions.
Best regards
Janfri ▲ Collapse
| | | |
Mihai Badea (X)
Luxembourg
English to Romanian
+ ...
| Leaders rankings vs. Freelancers/Agencies Directory | Oct 9, 2005 |
Walter Landesman wrote:
Hi Mihai,
The previous year. Last twelve monts. Yes, I suggested this a few months ago too. Maybe keeping a percentage of kudoz earned up to then, for the sake of old members, as a compensation, just in case somebody took a vacation or was sick or something. Special reward. But ranking based in the last twelve months is a good choice.
When I suggested it I didn´t get any feedback, though. Wish you better luck.
Walter
Dear Walter,
I agree with you that the old members should be rewarded for their continued support of this site. Without them, this site would not have been what it is now: a wonderful workplace for the translators worldwide. I think keeping a percentage of KudoZ earned up to then is an excellent idea.
Janfri wrote:
We actually have three different rankings and in this sense old and new members have equal conditions.
Best regards
Janfri
Dear Janfri,
I guess you are referring to the Leaders rankings (Last 3 Months - Last 12 Months - All Time ). Well, I think the clients are supposed to find us mainly through the Freelancers/Agencies Directory and that Directory is based on the overall points.
| | | |
| Pages in topic: [1 2] > |
Login to reply/comment 
