| Pages in topic: [1 2] > | Method of sorting used in ProZ.com directory to be updated Thread poster: Jared Tabor
|
|---|
Dear members,
After about ten years without a change, the way that translators are sorted in the ProZ.com translator directory is being evaluated for an update.
Several adjustments may ultimately be made, and searchers will be given the option of defining their preferred sort order. For now, the plan is to change the default sorting order to use the number of KudoZ PRO points earned over the past twelve months (total PRO points earned ever, which is used ... See more Dear members,
After about ten years without a change, the way that translators are sorted in the ProZ.com translator directory is being evaluated for an update.
Several adjustments may ultimately be made, and searchers will be given the option of defining their preferred sort order. For now, the plan is to change the default sorting order to use the number of KudoZ PRO points earned over the past twelve months (total PRO points earned ever, which is used now, will be an optional secondary sorting choice).
Recommendations for further improvements are also being evaluated. If you would like to enter a suggestion for an improvement to the directory, please do so using a support request.
The translator directory is located under the "Jobs and directories" menu tab.
Regards,
Jared
Member services
Why change the way the directory works at all?
To keep it as relevant and useful as possible over time.
Why change from sorting by KudoZ points all time, to sorting by KudoZ points earned over the past year?
As things stand, to move toward the top of the directory in a given combination (ie. pair and field), a person who has been at ProZ.com for just one or two years would have to gain as many points in that time frame as colleagues who have been at ProZ.com for as many as 10 years. Obviously this is not fair, and the adjustment will level the playing field somewhat.
Does this mean that the points I earned previously are now worthless?
No. Secondary sort is still carried out according to points earned over all time.
What about the advantage in exposure that members have had over non-members?
That will not change. As always, when someone searches the directory, they will get two sets of results: paying members first, then non-members. What is being updated is the sort order used within each set. ▲ Collapse
| | | | Damian Harrison (X)
Germany
Local time: 14:16
German to English
Jared wrote:
Several adjustments may ultimately be made, and searchers will be given the option of defining their preferred sort order. For now, the default sorting order will be changed to use the number of KudoZ PRO points earned over the past twelve months (total PRO points earned ever, which is used now, will be an optional secondary sorting choice).
Aside from the fact that the current system has now been in place for ten years, why do you plan to make this change?
| | | | | Providing more relevant default search results | Jun 2, 2010 |
Hi Damian,
A goal of this change is to provide default search results that are more relevant to the searcher (since those who have been active in the field more recently are shown), and more fair to translators who are active in KudoZ but who have joined more recently than others.
Regards,
Jared
| | | |
Jared wrote:
Hi Damian,
A goal of this change is to provide default search results that are more relevant to the searcher (since those who have been active in the field more recently are shown), and more fair to translators who are active in KudoZ but who have joined more recently than others.
Regards,
Jared
...you will be unfair to all the people that have joined this website a long time ago, and have contributed in much more significant ways than KudoZ, have a full-time job as freelancers and not much time to dedicate answering the KudoZ (btw answers are often wrong and often the wrong ones are awarded with points).
Anyway I wrote a ticket about this.
| | |
|
|
|
gianfranco 
Brazil
Local time: 09:16
Member (2001)
English to Italian
+ ...
| I suggest a weighted method based on various criteria | Jun 2, 2010 |
Jason,
there is an alternative and fair method has been suggested many times and always ignored.
KudoZ points are quite unfair as they are a "cumulative" system. It means that any mediocre answered, provided with enough time and nothing better to do, can cumulate points and raise to the top of the directory, because even if he/she provides n-thousands incorrect answers, the score only shows and cumulates the correct ones.
The directory should be sorted b... See more Jason,
there is an alternative and fair method has been suggested many times and always ignored.
KudoZ points are quite unfair as they are a "cumulative" system. It means that any mediocre answered, provided with enough time and nothing better to do, can cumulate points and raise to the top of the directory, because even if he/she provides n-thousands incorrect answers, the score only shows and cumulates the correct ones.
The directory should be sorted by quality, not quantity, adding to the equation a factor that takes into account the ratio (chosen answers / total answers ).
Example:
800 chosen / 1000 total is an excellent ratio, 80%
considering that there are often other good answers, or the asker may even occasionally discard a perfectly good answer for any reason.
On the other hand
800 correct answers / 4000 total answers (a 20% ratio)
currently gives the same score and position, but it is actually obtained only by persistence and not much real skills.
I suggest to sort the directory, at least starting from a certain treshold (i.e. above 250-500 points) only showing the ratio and not the actual score, and get the most reliable answerer raise to the top.
Alternatively, possibly more balanced than the criterion illustrated above, could be to introduce a weighted system that may include one or more of the following:
1. total score
2. recent score (latest 1-2 years)
3. reliability ratio (as defined above)
4. other relevant contributions to the site
Factor 1. should not be removed as it would be unfair to long time members
Factor 2. can be introduced to give a higher visibility to the current active members, but not entirely unfair to the old timers
Factor 3. illustrated above, very important and identified a long time ago by many members
Factor 4. makes sense as activities beneficial to the site can be rewarded, in some measure, with some increased visibility in the directory.
None of these factors alone is perfect, but a good combination and a weighted system may be the least unfair overall. I would exclude a system consisting in alternative sorting methods because what happens is that everybody would stop to the default method, which may be heavily biased in one way or another.
I suggest a combined method, possibly assigning to each factor a fair weight, and not necessarily the same weight as the other factors.
Gianfranco
[Edited at 2010-06-02 21:37 GMT] ▲ Collapse
| | | | | Disagree with suggested change | Jun 2, 2010 |
Hi Jared,
I agree with Giuseppina. This would be unfair to those who abided by certain rules and fought our way up in the Kudoz system several years ago.
We also had to deal with old members with such a high number of kudoz we thought we could never reach. But we did, we invested time and efforts and research to answer questions and get those kudoz.
It is unfair that rules may change now when we are up on the scale. Should we start all over again? We might... See more Hi Jared,
I agree with Giuseppina. This would be unfair to those who abided by certain rules and fought our way up in the Kudoz system several years ago.
We also had to deal with old members with such a high number of kudoz we thought we could never reach. But we did, we invested time and efforts and research to answer questions and get those kudoz.
It is unfair that rules may change now when we are up on the scale. Should we start all over again? We might not have the time to contribute now, the time we had as new members and new freelancers and following your suggestions (Proz`s) we built a strong profile including getting many kudoz because you said that was the way to be among the firsts in a directory search.
I see it as a way of giving an incentive to new members as part of the current Membership Campaign and also to encourage people to participate more in the kudoz system.
We believed in Proz.com, we also invested our membership money to support the site all these years and abide to the rules valid at that time.
I really disagree with this proposal. I think new members or newcomers to Kudoz should fight their way up as we did several years ago. ▲ Collapse
| | | | | Phase-in required to ensure fairness | Jun 2, 2010 |
I think most of us have seen this coming. Clearly KudoZ participation has been suffering from a decline for several years now and the intent of this change is obvious: to encourage active involvement.
At the same time, I think it's important to amend the system in a way that meets this objective while also ensuring that past KudoZ contributions are recognised in some way and that all of us are treated fairly.
It's also essential to understand the kind of impact this may... See more I think most of us have seen this coming. Clearly KudoZ participation has been suffering from a decline for several years now and the intent of this change is obvious: to encourage active involvement.
At the same time, I think it's important to amend the system in a way that meets this objective while also ensuring that past KudoZ contributions are recognised in some way and that all of us are treated fairly.
It's also essential to understand the kind of impact this may have on the basis of specific examples. A brief search of the Directory indicates just what a massive rebalancing of the value of 1 KudoZ point we are talking about here:
With this new system in place, answering 3-4 questions can take you from zero to hero in no time. While all of us, including those at the top of the rankings, would probably recognise that KudoZ points were never a great way of distinguishing between good translators and bad translators, it was some sort of indicator nonetheless.
If you were a terrible translator it used to be difficult (although not impossible as has been shown in my language pair by a number of "hard-working" individuals) to get to the top of the directory. Now, all you need to do is grab a few easy points and you're there. The outcome of this is that activity is rewarded, however I don't think that outsourcers searching the Directory are looking for people who have been active in the last 12 months - they're looking for people who can do the job they need done.
In addition, the fact that we (or at least I) had no warning of this impending change means that those of us who were, quite reasonably, acting in accordance with the old system are now at a sizeable disadvantage. Someone who has been active in the last 12 months has had a head start of... 12 months.
In view of these two things, I would suggest that a more balanced system should be based on the following ideas:
1. Historical contributions should be recognised in some (albeit significantly less influential) way.
2. Regardless of whether 1. is implemented, notifications of this change should be sent to all current and past KudoZ participants informing them that it will be made effective in 12 months. This gives everyone an opportunity to start from a level playing field, while the extra time allows the system to be refined and perfected. ▲ Collapse
| | | | Mats Wiman
Sweden
Local time: 14:16
Member (2000)
German to Swedish
+ ...
In memoriam | A totally irrlevant change IMHO | Jun 2, 2010 |
Hi Jared,
This seems to be a change for change's sake.
1. Why would it be more relevant with the last 12 months' contributions than all contributions?
2. Is it fair to strip someone of his/her achieved KudoZ points only because they were gained 2000-2009? (Yes - I am partial)
No outsourcer will agree on that unless they are only looking for hungry and active newbies.
A seasoned outsourcer would rather look for an experienced translator wi... See more Hi Jared,
This seems to be a change for change's sake.
1. Why would it be more relevant with the last 12 months' contributions than all contributions?
2. Is it fair to strip someone of his/her achieved KudoZ points only because they were gained 2000-2009? (Yes - I am partial)
No outsourcer will agree on that unless they are only looking for hungry and active newbies.
A seasoned outsourcer would rather look for an experienced translator with lots of proven
competence (the older the better).
The option for the outsourcer to choose sorting order should IMHO depart from a default sorting order, i.e. the original one - by total KudoZ points in the pair.
As we have extensively discussed two years ago, a sorting by rate is IMHO a relevant and accceptable sorting method. (I maintained that being cheap is no advantage and many an outsourcer shares this view).
Sorting by years of experience is also a possibility.
Sorting by credentials is also a possibility.
Sorting by target language native speech is also relevant to outsourcers.
For interpreters, location is also a elevant sorting option
I sincerely hope your suggestion won't be implemented wihout dicussion among members.
It's too important to be a rush discussion withou consultation.
PS addition "intended to level the playing field".
Jared, This is not a playing field but the result of heavy investment, very relevant to the outsourser and very important to the freelancer.
All KudoZ points could have a note (like 2000-2007) indicating the time period when point were earned, thus putting a newbie at less disadvantage.
OTOH, having old points should not be rewarded with a disadvantage (see above)
Mats
[Edited at 2010-06-02 22:03 GMT] ▲ Collapse
| | |
|
|
|
Natalie
Poland
Local time: 14:16
Member (2002)
English to Russian
+ ...
MODERATOR SITE LOCALIZER
Some addicted KudoZers who provide tons of answers using machine (or Google) translator also gather points rapidly - so what will happen now? Will they be listed the top of the directory? I doubt that outsourcers will be really happy when the directory would suggest them such candidates...
Directory sorting by simple sum of points gathered rapidly within a short period of time (and 12 months is a very short period of time, believe me!) can be hardly considered as a serious parameter... See more Some addicted KudoZers who provide tons of answers using machine (or Google) translator also gather points rapidly - so what will happen now? Will they be listed the top of the directory? I doubt that outsourcers will be really happy when the directory would suggest them such candidates...
Directory sorting by simple sum of points gathered rapidly within a short period of time (and 12 months is a very short period of time, believe me!) can be hardly considered as a serious parameter allowing high rankings in the directory. As it has been suggested many times in the past, rankings should be based on more complicated indices that take into account not only the sum of the points, but also the reliability of answerers. Lots of discussions, suggestions and propositions can be found in the archives - please just open those old threads and take a look at them. There were excellent suggestions of weighted scoring that would take into account reliability, period of time and many other factors.
Also, before introducing drastic changes it would be fair to announce them beforehand. You should give people a chance to become more active and improve their rankings.
Natalia ▲ Collapse
| | | | | reliability instead of quantity | Jun 2, 2010 |
Natalie wrote:
As it has been suggested many times in the past, rankings should be based on more complicated indices that take into account not only the sum of the points, but also the reliability of answerers.
Precisely. Thanks, Natalia.
johanna
| | | | Jaroslaw Michalak
Poland
Local time: 14:16
Member (2004)
English to Polish
SITE LOCALIZER
A change which makes KudoZ a place for a more cutthroat competition? Where KudoZ chasers will be given even more prominence? Where the "winning" answer will have to be given within seconds? Where users who actually care about their ranking will claw and grasp at every conceivable question not to get ahead, but just to keep afloat?
You can't seriously think that the change will not affect the quality of KudoZ as such... Considering that in the current system you are never "pun... See more A change which makes KudoZ a place for a more cutthroat competition? Where KudoZ chasers will be given even more prominence? Where the "winning" answer will have to be given within seconds? Where users who actually care about their ranking will claw and grasp at every conceivable question not to get ahead, but just to keep afloat?
You can't seriously think that the change will not affect the quality of KudoZ as such... Considering that in the current system you are never "punished" for giving wrong answers, it might be enough to answer as many questions as possible - there are enough clueless askers around. ▲ Collapse
| | | | | A Suggestion: Depreciating KudoZ Points | Jun 2, 2010 |
Every community aims to encourage - and reward - active participation, and ours is no exception. I see this change as a means of achieving that objective. Moreover, now could just be the time to try something new and fresh.
Having said that, I can envision a simple mechanism that could take into account one's older accomplishments together with newer ones. And it would still serve the ultimate purpose, i.e. motivate people to stay active and keep contributing.
Ma... See more Every community aims to encourage - and reward - active participation, and ours is no exception. I see this change as a means of achieving that objective. Moreover, now could just be the time to try something new and fresh.
Having said that, I can envision a simple mechanism that could take into account one's older accomplishments together with newer ones. And it would still serve the ultimate purpose, i.e. motivate people to stay active and keep contributing.
Many things in life depreciate with time, so why not KudoZ points? I propose a simple weighting method: use multipliers for all points based on when they were earned. For instance, 100 points from 5 years ago could be worth as much as 20 earned in the last thee months.
The exact math for this can be discussed, played with and tested to maximize the utility and fairness of the new ranking system. The key idea is to recognize past accomplishments and equate them with newer ones in a fair and balanced way. The specifics will follow. ▲ Collapse
| | |
|
|
|
Erik Freitag
Germany
Local time: 14:16
Member (2006)
Dutch to German
+ ...
| reliability ratio | Jun 2, 2010 |
Natalie wrote:
As it has been suggested many times in the past, rankings should be based on more complicated indices that take into account not only the sum of the points, but also the reliability of answerers. Lots of discussions, suggestions and propositions can be found in the archives - please just open those old threads and take a look at them. There were excellent suggestions of weighted scoring that would take into account reliability, period of time and many other factors.
I agree 100%! One example where this (and the reasons proz.com might have to not implement something like that) has been discussed at large can be found here.
| | | | | Previous, repeated discussions and proposals | Jun 2, 2010 |
Jared,
I am glad to hear that you are considering adding more options to the directory sorting. However, I have to agree with those that disagree with your proposed change. It does not accomplish the stated intended purpose: to provide more reliable information for outsourcers. Instead, please focus on the word: reliability.
That is what's important.
There were several discussions in the past about this, here is one, with... See more Jared,
I am glad to hear that you are considering adding more options to the directory sorting. However, I have to agree with those that disagree with your proposed change. It does not accomplish the stated intended purpose: to provide more reliable information for outsourcers. Instead, please focus on the word: reliability.
That is what's important.
There were several discussions in the past about this, here is one, with what I wrote there:
http://www.proz.com/forum/kudoz/119297-hit_rate_or_points_to_closed_answers_ratio_pta.html#980033
As it was pointed out before, everybody can see his/her own KudoZ statistics in his/her profile, the reliability ratio is there, it is called "acceptance rate" and it is a percentage figure (so 65.14 means 65.14%, or 0.6514).
If for directory ranking purposes, we want to use not only the number of KudoZ points but also the reliability ratio (how many answers were given in relation to the points gained), the easiest way is to MULTIPLY the two figures and use the result for ranking.
I think this is the best of both worlds.
Using only the KudoZ point total omits the information about reliability. (It also gives unfair advantage to the aggressive, machine-gun style point-hunters, and disadvantageous for newcomers.)
Using only the reliability ratio omits the information about level of activity. (It would give unfair advantage to people that answer very few questions, and it may discourage people from answering questions.)
This combination has no problem for scaling (it can be used for any high number of points), and it works in a fair way, at least I found it fair. I calculated a few scenarios, and I liked what I saw.
If people want it, I can type up a few examples here, but I guess anyone can do it.
Katalin ▲ Collapse
| | | | | Not fair in my opinion | Jun 2, 2010 |
Hello Jared
I disagree with this change, Natalie has just provided a good point and I also agree with Giuseppina, a change like this will be unfair for old members who have little time to participate actively in KudoZ, not to mention the very wrong answers that are often selected.
Please reconsider
Regards
Angio
| | | | | Pages in topic: [1 2] > | To report site rules violations or get help, contact a site moderator: You can also contact site staff by submitting a support request » Method of sorting used in ProZ.com directory to be updated | Trados Studio 2022 Freelance | The leading translation software used by over 270,000 translators.
Designed with your feedback in mind, Trados Studio 2022 delivers an unrivalled, powerful desktop
and cloud solution, empowering you to work in the most efficient and cost-effective way.
More info » |
| | Anycount & Translation Office 3000 | Translation Office 3000
Translation Office 3000 is an advanced accounting tool for freelance translators and small agencies. TO3000 easily and seamlessly integrates with the business life of professional freelance translators.
More info » |
|
| | | | X Sign in to your ProZ.com account... | | | | | |
Login to reply/comment 
