Pages in topic:   [1 2 3] >
Feature suggestion: Individual KudoZ efficiency metric
Thread poster: Boris Shapiro

Boris Shapiro
Russian Federation
Local time: 14:46
English to Russian
+ ...
Apr 18

Hi!

I wonder if it is planned to introduce some kind of efficiency metric regarding the way individual members answer terminology questions. I mean, instead of showing the total number of point, show the points to answers given ratio. 'Cause clearly some colleagues are just spamming the system with their answers on the off-chance that they actually hit it. And while that in and of itself is just a nuisance, nothing more, with the way total KudoZ points are used to rank people here this turns out to be very much misleading!


IT>EN Legal
 

Jared Tabor
Local time: 08:46
SITE STAFF
Answer acceptance rate? Apr 18

HI Boris,

If you go to a person's KudoZ activity in their profile, there is a tab called "Stats" where you can see, among other things, their answer "acceptance rate", which shows the percentage of answers selected as most helpful against the number of questions answered. Is that what you mean?

Jared


 

Boris Shapiro
Russian Federation
Local time: 14:46
English to Russian
+ ...
TOPIC STARTER
Can you post a screenshot? Apr 18

Hi Jared,

I'm confused. Whenever I click on someone else's KudoZ activity, I only see Point totals, Asked, Opened, and Answered tabs. No 'Stats' tab and no acceptance rate.

And even if there is such a value, aren't people still rated by their point totals?


 

Thomas T. Frost  Identity Verified
Member (2014)
Danish to English
+ ...
Stats only in my own profile Apr 18

The Stats information is good, but I'm only shown this tab in my own profile, not others'. Is that how it should work?

Hedwig Spitzer Cáceres
Teresa Borges
 

Jared Tabor
Local time: 08:46
SITE STAFF
My mistake, on two counts Apr 18

Sorry about that, I thought the Stats were shown, but 1) they are only available to paying members and 2) they are only shown to the profile owner, not publicly.

What do others think about Boris' suggestion?



Added: ranking is done by point totals, yes. There could also potentially be an option to rank by point total and acceptance rate, for example.

Jared


 

Thomas T. Frost  Identity Verified
Member (2014)
Danish to English
+ ...
What I think Apr 18

Jared Tabor wrote:

What do others think about Boris' suggestion?

Added: ranking is done by point totals, yes. There could also potentially be an option to rank by point total and acceptance rate, for example.


I agree with the overall idea, but a simple ratio would be misleading. Let's say I just answer one question right as my only Kudoz activity ever, and I'd have a 100% success rate.

It's a complicated matter, which requires careful analysis.

There should probably be some 'decay' of old data, so that a question answered 20 years ago doesn't count as much as a question answered recently (and similar for questions asked). For each year, each parameter could lose x% of its weight.

Then there is Boris' suggested answered-to-asked ratio, which is a good factor.

There is also the success rate out of answered questions.

And something else that sometimes bothers me about Kudoz: some members quickly get unpleasant if you don't agree with them. They may take a 'disagree' vote as a personal insult and attack the voter. There is too much uncivilised behaviour. I would suggest that if a moderator has to step in against a member for such a reason, it would result in a Kudoz-point penalty that can be felt, so as to deter that type of behaviour. Of course, abusive ‘disagree’ votes could also be penalised, i.e. if someone votes a suggestion down without being able to provide any linguistic justification.


Yoana Ivanova
Hedwig Spitzer Cáceres
Andy Watkinson
Elizabeth Tamblin
mk_lab
Teresa Borges
 

Yoana Ivanova  Identity Verified
Estonia
Local time: 14:46
Member
English to Bulgarian
+ ...
Before that Apr 18

Such a feature could indeed be useful. However, I think that first some kind of better system should be implemented to ensure that Kudoz questions don't stay open indefinitely without an answer ever being chosen.

I know that the asker gets reminded via email and then questions can be closed automatically based on agrees, but when neither of those work out, they can remain open forever. That in turn automatically brings down our answer acceptance rate.


Thomas T. Frost
Platon Danilov
Michele Fauble
 

Katalin Horváth McClure  Identity Verified
United States
Local time: 07:46
Member (2002)
English to Hungarian
+ ...
@Jared - take a stroll down Memory Lane. :-) Apr 18

This topic has been discussed numerous times, and there were multiple proposals on how to incorporate the acceptance ratio into the directory.
Here is one link, but you can do a search for +acceptance +rate and you will get more.
https://www.proz.com/forum/kudoz/119297-hit_rate_or_points_to_closed_answers_ratio_pta.html#980033


Sheila Wilson
Yolanda Broad
Cilian O'Tuama
Andy Watkinson
Erik Freitag
mk_lab
Mirko Mainardi
 

Thomas T. Frost  Identity Verified
Member (2014)
Danish to English
+ ...
And … Apr 18

The detailed stats should probably remain private, but a compound score based on weighted parameters would probably be more useful than just the total score.

In order for each member to evaluate their own relative score, it would be helpful to include the global average for ‘acceptance rate’, ‘average agrees per question’ and ‘average disagrees per question’ in each category in the private stats.


 

Manuel Bas y Mansilla  Identity Verified
Netherlands
Member (Mar 2019)
English to Spanish
Peer review Apr 19

As someone mentioned, it's been discussed before, and one suggestion was to have it based not on chosen answers (it'd be misleading and might discourage participation) but on peer agree/disagree.
No system is perfect, and certain areas get less participation from colleagues, but there should be some 'penalty metric' for people who constantly get downvoted by various members (as to avoid personal issues).


mk_lab
Platon Danilov
 

Andy Watkinson
Spain
Local time: 13:46
Member
Catalan to English
+ ...
Decay? Yikes! Apr 19

Thomas T. Frost wrote:


There should probably be some 'decay' of old data, so that a question answered 20 years ago doesn't count as much as a question answered recently (and similar for questions asked). For each year, each parameter could lose x% of its weight.



I entirely understand your point about proposing just one answer, getting it right and thus being hailed as a 100% genius. There are so many factors involving language combinations, ratio asked/answered, ratio answers right/rejected, accumulated agrees/disagrees, etc..that my CSE maths-level brain hurts.

But what I don't understand is this idea of attaching more importance to recent answers than those from xx years ago.

I joined ProZ about 18 years ago, discovered KudoZ, thought "Eh up, lad! This looks like fun" and immediately started answering anything that moved. I acquired a couple of thousand points fairly quickly, continued for a bit and over recent years probably average a couple of answers a month, if that.

So, my recent contributions might not add up to much but I don't see my sixty-something rate of chosen answers in Catalan or fifty odd in Law* being any less valid or useful now than they were when asked. Someone might well have used one of them yesterday for all I know, the same way I obtain valuable information from answers posted by former colleagues who even left the site years ago.

i.e., their knowledge, experience, reliability are no less valid now than then.
In fact, all three are probably greater now than then, given the time elapsed. Afer all, you don't say "Hendrix hasn't played anything for 50 years. Francis Rossi is now a better guitarist."

And I'm decaying quite nicely all on my sweet own, thank you.
I don't need my reputation following suit.

------o------

* Thanks to whoever pointed out that these Stats existed - I had no idea until now.

No te acostarás sin saber una cosa más.

[Edited at 2019-04-19 04:00 GMT]

[Edited at 2019-04-19 14:27 GMT]


Kay Denney
Matthias Eng
Tina Vonhof
Michele Fauble
Sandra& Kenneth
Recep Kurt
 

Lincoln Hui  Identity Verified
Hong Kong
Local time: 19:46
Member
Chinese to English
+ ...
Throw out the baby, keep the bathwater! Apr 19

Great idea, it will discourage people from participating and helping others, the primary goal of KudoZ, in order to maintain the integrity of the points system, a secondary goal.

[Edited at 2019-04-19 04:21 GMT]


mk_lab
Michele Fauble
 

Sandra& Kenneth  Identity Verified
Israel
Local time: 14:46
Member (2009)
French to English
+ ...
No. Apr 19

Jared Tabor wrote:

What do others think about Boris' suggestion?

There could also potentially be an option to rank by point total and acceptance rate, for example.

Jared


If it ain't broke don't fix it.

Each language and direction has its own participants; some in the thousands, and some hardly any. In popular language pairs, there is lively participation and competition for points, in others, it's not even worth posting questions...

This means that the chance of acceptance in popular languages is much lower, for the same number of answers. Even for a correct answer, somebody rushing in to give an earlier answer may be superseded by someone else chiming in with a more appropriate synonym.

I enjoy the game of Kudoz and the confrontation of ideas it generates. Leave it as it is. It's fun.

I also think that posting the ratio of accepted/given answers would be a form of public shaming and should not be acceptable, even if some individuals give irrelevant answers.

Sandra

[Edited at 2019-04-19 07:34 GMT]


Elizabeth Tamblin
mk_lab
ahartje
Teresa Borges
Michele Fauble
Baran Keki
 

Daniel Frisano
Switzerland
Local time: 13:46
Member (2008)
English to Italian
+ ...
KudoZ doesn't make sense anymore Apr 19

When KudoZ started, answers were mostly based on actual skills: knowledge, experience, deciphering obscure sentences, intuition.

Now it's just a race to whoever gets the most Google hits.

Agrees are awarded and exchanged (and thanked, as a personal favour) between buddies to boost each other's egos.

To make matters worse, the staff won't do a thing against multiple answers by the same user and other aggressive practices.

It has become a joke. But hey, welcome to 2019, when being liked is everything that matters.

(By the way, no "likes" on this post please. It is all SO childish).


writeaway
Platon Danilov
Beatriz Ramírez de Haro
 

Elizabeth Tamblin  Identity Verified
United Kingdom
Local time: 12:46
Member (2012)
French to English
Bad idea Apr 19

Jared Tabor wrote:

Sorry about that, I thought the Stats were shown, but 1) they are only available to paying members and 2) they are only shown to the profile owner, not publicly.

What do others think about Boris' suggestion?



Added: ranking is done by point totals, yes. There could also potentially be an option to rank by point total and acceptance rate, for example.

Jared


When I use the Term Search feature, I like to see a range of suggestions. I don't always choose the term that was awarded points, because one of the other suggested terms might be more suitable in the context.

Just because a term is not "the winner" does not mean it is an invalid or a bad suggestion. Often there are several valid suggestions, and it would be unfair to penalise people who are being very helpful to their colleagues be devaluing their contributions.


Sandra& Kenneth
Platon Danilov
Teresa Borges
IT>EN Legal
Thomas T. Frost
mk_lab
Michele Fauble
 
Pages in topic:   [1 2 3] >


To report site rules violations or get help, contact a site moderator:


You can also contact site staff by submitting a support request »

Feature suggestion: Individual KudoZ efficiency metric

Advanced search






CafeTran Espresso
You've never met a CAT tool this clever!

Translate faster & easier, using a sophisticated CAT tool built by a translator / developer. Accept jobs from clients who use SDL Trados, MemoQ, Wordfast & major CAT tools. Download and start using CafeTran Espresso -- for free

More info »
PerfectIt consistency checker
Faster Checking, Greater Accuracy

PerfectIt helps deliver error-free documents. It improves consistency, ensures quality and helps to enforce style guides. It’s a powerful tool for pro users, and comes with the assurance of a 30-day money back guarantee.

More info »



Forums
  • All of ProZ.com
  • Term search
  • Jobs
  • Forums
  • Multiple search