| Pages in topic: [1 2] > | How to make the KudoZ system fairer and more professional? Thread poster: Paul Cohen
|
|---|
Paul Cohen 
Greenland
Local time: 21:52
German to English
+ ...
We're all familiar with the many problems plaguing KudoZ.
Answerers who have been around for ages have an insurmountable advantage over newcomers in the directory simply because they have had time to amass an astronomical number of points. Is that fair? No.
Other people -- we could call them shoot-from-the-hip artists -- have garnered an impressive number of points by merely randomly firing off answers. These individuals seem to be online at all times of the d... See more We're all familiar with the many problems plaguing KudoZ.
Answerers who have been around for ages have an insurmountable advantage over newcomers in the directory simply because they have had time to amass an astronomical number of points. Is that fair? No.
Other people -- we could call them shoot-from-the-hip artists -- have garnered an impressive number of points by merely randomly firing off answers. These individuals seem to be online at all times of the day (don't they ever have work to do?!), with their eyes glued to their screens for email notifications. Is it fair that such point-chasers, who consistently produce mediocre and even appalling answers, should enjoy an excellent ranking in the directory? No.
Mediocre translators who pad their profiles and directory listings by chasing points are bad enough, but what makes matters worse is that such people, who seem to live and breathe KudoZ, often exert a disproportionate influence on the outcome of questions with their 'agrees' and 'disagrees'. Is it right that individuals with a horrendous track record should have as much influence on the outcome of a question as people with an excellent track record?
How to make the KudoZ system fairer and more professional?
How to discourage answerers who are dragging down the overall quality of KudoZ?
It's a problem that I've been pondering for some time now, especially following a recent thread on the "political correctness" of the KudoZ system, the declining quality of KudoZ questions, and our powerlessness to change the situation (see: http://www.proz.com/topic/153244).
I've come up with a solution that might raise some eyebrows.
There have been many calls in the past to introduce KudoZ answer acceptance rates as an alternative measure of people's KudoZ proficiency. These proposals have not been implemented (yet), but there's nothing stopping us from discussing an even more radical solution.
In order to alleviate the problems listed above, I would suggest the following:
- Introduce KudoZ acceptance rates to the site directory as a search option. This would put more of an emphasis on quality and give top-notch newcomers a chance to shine in the eyes of potential customers -- and this information would be visible on profiles and next to people's names on KudoZ.
- Weigh people's 'agrees' and 'disagrees' according to a combination of their acceptance rates and total points.
This is how this second point might work:
Someone who has an acceptance rate of 2.4 and a total of 1,500 KudoZ points would carry nearly four times as much weight as someone with an acceptance rate of 0.6 and 3,000 points (BTW, I always calculate the acceptance rate by dividing the number of points by the number of answers given).
In other words, if you give good answers, then you have the privilege of exerting more influence on the outcome of KudoZ questions. If however you are a translator who has been polluting KudoZ with ridiculous answers, then you would enjoy less influence, despite your ability to collect thousands of points by virtue of the sheer awe-inspiring numbers of questions that you answer.
According to this system, it might take four 'agrees' from the worst of the point-chasers to outweigh one 'disagree' from an individual with an outstanding acceptance rate.
This would put an end to what I would call the "dumbing down" of the site and the tyranny of twitchy-fingered answerers. It would also add a measure of sanity and professionalism to those questions that are decided by the robot.
I think it's also fair to say that this would spell the end of a certain "political correctness" on the site, which is something that many members might actually welcome. No, it's not one translator one vote! It's a dictatorship by those who produce acceptable answers!
[Edited at 2009-12-18 00:06 GMT] ▲ Collapse
| | | | Jack Doughty
United Kingdom
Local time: 23:52
Russian to English
+ ...
In memoriam | A dictatorship by those who produce acceptable answers? | Dec 18, 2009 |
I would prefer that (if it existed, which I don't think it does) to a democracy which produces unacceptable answers.
| | | | Heinrich Pesch
Finland
Local time: 01:52
Member (2003)
Finnish to German
+ ...
| Don't forget this is a business | Dec 18, 2009 |
This site is for making money, not for generating trustworthy glossaries. So only changes are implemented that generate more traffic and income for the site.
The problems you mention affect only big-traffic language pairs. Finnish-German is very quiet, a question I put there yesterday is still without answer, and I have already sent away the translation. Welcome to the Finnish-German-Finnish Kudoz!
Regard... See more This site is for making money, not for generating trustworthy glossaries. So only changes are implemented that generate more traffic and income for the site.
The problems you mention affect only big-traffic language pairs. Finnish-German is very quiet, a question I put there yesterday is still without answer, and I have already sent away the translation. Welcome to the Finnish-German-Finnish Kudoz!
Regards
Heinrich ▲ Collapse
| | | | Laurent KRAULAND (X)
France
Local time: 00:52
French to German
+ ...
| Agree, but... | Dec 18, 2009 |
Heinrich Pesch wrote:
This site is for making money, not for generating trustworthy glossaries. So only changes are implemented that generate more traffic and income for the site.
Regards
Heinrich
how about the scenario in which site traffic will decrease due to the appalling quality of KudoZ answers and other problems, such as those described in the OP? As with the so-called "crowdsourcing", bad answers chase good answers away. If this site and other ones can live on that, fine...
[Edited at 2009-12-18 10:05 GMT]
| | |
|
|
|
| One minor consideration | Dec 18, 2009 |
While I am broadly in favour of incorporating it somehow into the rankings, the trouble with proportions/acceptance rates is that they too are open to manipulation. Get your first ever answer accepted for 4 points and bingo - 100% acceptance rate (or 4.0 by your system) and you are (joint equal) top of the pile for ever.
That said, I agree with the point about the "weight of history", as it were, skewing the results for curren... See more While I am broadly in favour of incorporating it somehow into the rankings, the trouble with proportions/acceptance rates is that they too are open to manipulation. Get your first ever answer accepted for 4 points and bingo - 100% acceptance rate (or 4.0 by your system) and you are (joint equal) top of the pile for ever.
That said, I agree with the point about the "weight of history", as it were, skewing the results for current times. I think a 12-month rolling system would be a fair method for ratings and rankings. Or possibly a method where the last 12 months are weighted 1.0, 12-24 months are weighted 0.5, 24-48 months ago weighted 0.25 etc.
Somewhere deep in all these suggestions made over the years lies the idea points/ranking system.
Trouble is, frankly, I think that even if the perfect points system were implemented, a large number of the best answerers have left, never to return. For one thing, I get the feeling a lot of them like to feel free to at least gently chide, if not openly lambast, askers who seem unable to open a dictionary or run a decent search, and you would need to review the system in its entirety to tempt people back. And I'm not sure the perceived ROI makes it worthwhile for the site to pursue this option. ▲ Collapse
| | | | Erik Freitag
Germany
Local time: 00:52
Member (2006)
Dutch to German
+ ...
| Woodstock (X)
Germany
Local time: 00:52
German to English
+ ...
| Tilting at windmills? | Dec 18, 2009 |
Hi, Paul,
While your suggestion is very interesting, I'm not sure how much it would change the situation. The same complaints have been aired regularly since I've been a member, and it will probably be the same 5 years from now. At some point you shrug your shoulders and either stop using or contributing to KudoZ or simply stop caring about it.
In my main language pair I feel that poor or just wrong KudoZ answers are pretty well regulated by "neutral/disagree" comments... See more Hi, Paul,
While your suggestion is very interesting, I'm not sure how much it would change the situation. The same complaints have been aired regularly since I've been a member, and it will probably be the same 5 years from now. At some point you shrug your shoulders and either stop using or contributing to KudoZ or simply stop caring about it.
In my main language pair I feel that poor or just wrong KudoZ answers are pretty well regulated by "neutral/disagree" comments. Not only that, the serious translators get to know who these twitchy-fingered answerers are, and they won't be taken seriously for long. I rarely used to give "neutral/disagree" comments, and I still don't for questions I have also given answers to (I leave it to others), but I've lost patience with non-native speakers or those claiming to be native speakers giving crappy answers in a language they so obviously do not have a command of. Now I have absolutely no qualms about calling them out for suggesting answers that no native speaker would ever use. You are right, many seem to have no work to do, but we probably know the reason ... Or those people are not active translators who depend on translating for a living, but doing it for fun - at our expense, in a way.
What I find even more annoying are the askers who are too lazy to do their own research, especially for fairly simple, straightforward terms, yet the same people (those who are glued to their screens, as you mention) give them answers over and over again, even though they MUST know by a certain point that they are being used and even abused. All for the mighty points? I don't understand it, but have stopped wasting energy thinking about it. I just don't bother clicking on the term when I see who the Asker is: usually someone who has asked a zillion questions, many not closed, and no answers... ever. It just surprises me that they still get any translation jobs ... if they are translators at all. Some are not and just like reading English novels/stories while letting us do their research for them.
So as much as I would welcome changes that could improve the quality of KudoZ, I've pretty much worked out my own way of dealing with the flaws and just given up on anything changing anytime soon. This "laissez-faire" way of thinking has definitely helped my blood pressure.
Edit for addition
[Edited at 2009-12-18 11:32 GMT] ▲ Collapse
| | | | Robert Forstag
United States
Local time: 18:52
Spanish to English
+ ...
| Interesting suggestions / not sure how much they would improve matters though | Dec 18, 2009 |
The "weighting" suggestion is certainly an improvement over the call to blacklist offending Askers and Answerers which was tabled last week. Charlie's concern about inflated acceptance rates based on a limited number of responses could be addressed by setting a minimum number of responses to be eligible to be ranked (and therefore to have one's responses weighted) in the first place.
One problem with the suggestion is that every translator has a sphere of competence within which sa... See more The "weighting" suggestion is certainly an improvement over the call to blacklist offending Askers and Answerers which was tabled last week. Charlie's concern about inflated acceptance rates based on a limited number of responses could be addressed by setting a minimum number of responses to be eligible to be ranked (and therefore to have one's responses weighted) in the first place.
One problem with the suggestion is that every translator has a sphere of competence within which said rating can be thought of as applying. So if, for example, a translator has a 40% acceptance rating in Spanish-English garnered mostly from answers related to Law and Business, it would hardly seem fair for him to carry his "Big Kahuna" status into a Medical or Technology question.
Yet another problem is the fact that a translator having a high acceptance rate does not guarantee that he will act in a fair and equitable manner. Thus, he may choose to use his weighty "disagrees" to punish poor answers from those who tick him off for one reason or another (e.g., by having rejected or not supporting his own answers) while giving a free pass to equally poor answers proposed by those who have not caused such offense, or who have even supported him in the past.
It is naive to think that this kind of thing does not happen all the time............
I still think that, in general terms, the fundamental reality is this:
1.
The system is fated to be flawed in some fundamental ways because of its openness.
2.
The system as currently constituted provides for appropriately punishing the kinds of serial offenders Paul is referring to. Thus, while I know that there is something of a group norm in most language pairs that shuns the posting of "disagrees," if it is noted that there is someone who is consistently garnering points for poor answers, then it is perhaps appropriate for "the gloves to come off" and to go ahead and post "disagrees" that are backed by solid reasoning and free of personal attack.
I think that, in general terms, the higher one unjustifiably perches himself, the greater the justification there is for exposing his or her unmerited exalted status. There is a big difference between someone posting inadequate responses accompanied by tentative comments and a CL of 1 or 2 who has not accumulated a high number of points and someone posting the same kinds of responses with a CL of 5 and who has accumullated many thousands of points.
[Edited at 2009-12-18 15:33 GMT] ▲ Collapse
| | |
|
|
|
LEXpert
United States
Local time: 17:52
Member (2008)
Croatian to English
+ ...
| Remember who the deciders of acceptance are | Dec 18, 2009 |
I'm not sure that acceptance rate is a workable proxy for quality. Besides the potential for manipulation, the "judge" who does the accepting (i.e., the asker) is very often either uninformed as to what the "correct" answer might be (rather by definition, otherwise they wouldn't need to ask in the first place), or too lazy to look it up themselves, or both. It would be like having your grade on an exam determined by your fellow students rather than an instructor. While one could argue that th... See more I'm not sure that acceptance rate is a workable proxy for quality. Besides the potential for manipulation, the "judge" who does the accepting (i.e., the asker) is very often either uninformed as to what the "correct" answer might be (rather by definition, otherwise they wouldn't need to ask in the first place), or too lazy to look it up themselves, or both. It would be like having your grade on an exam determined by your fellow students rather than an instructor. While one could argue that the asker will, in theory, be able to choose the best answer based on how well it is supported, such "support' more often than not turns out to be indicative of one's ability to do a google search rather than translation ability.
[Edited at 2009-12-18 16:57 GMT] ▲ Collapse
| | | | Yaotl Altan
Mexico
Local time: 16:52
Member (2006)
English to Spanish
+ ...
To me, the acceptance rate is the solution. That proposal has been planned several times in the site.
| | | | | Post removed: This post was hidden by a moderator or staff member for the following reason: Not in line with forum rule http://www.proz.com/siterules/forum/14#14 | Erik Freitag
Germany
Local time: 00:52
Member (2006)
Dutch to German
+ ...
Yaotl Altan wrote:
To me, the acceptance rate is the solution. That proposal has been planned several times in the site.
Well, planned? It has been proposed several times, but never acknowledged by site staff. Regrettably.
| | |
|
|
|
Brian Young
United States
Local time: 15:52
Danish to English
| how about NO points? | Dec 18, 2009 |
This whole system of chasing what I would call "Brownie points" turns me off. Why should anyone get "points" for answering a question? Aren't we supposed to be just helping each other out, as fellow translators?
Maybe, if there were no points associated with this, then only serious askers and answerers would use it. I can't even stand to look at that part of this website anymore. The people asking many of these questions obviously do not own any dictionaries, do not go on-line, use Google,... See more This whole system of chasing what I would call "Brownie points" turns me off. Why should anyone get "points" for answering a question? Aren't we supposed to be just helping each other out, as fellow translators?
Maybe, if there were no points associated with this, then only serious askers and answerers would use it. I can't even stand to look at that part of this website anymore. The people asking many of these questions obviously do not own any dictionaries, do not go on-line, use Google, or any of the almost endless resources available on the internet.
I think too many people, after a vacation abroad, or a semester in college, think they can set up shop as translators.
That is the only explanation I can think of for why some of these simple questions are posted. ▲ Collapse
| | | | Robert Forstag
United States
Local time: 18:52
Spanish to English
+ ...
| Posting easy strings of words | Dec 20, 2009 |
Olaf Knechten wrote:
In my language pairs there aren't only lots of very easy questions (too lazy to do research, lack of understanding of source language), there are also many queries for suggestions about how to phrase something in the target language. I stopped answering these because I think writing skills are a prerequisite for becoming a translator.
I see this in my pair as well, and I could not agree more. To me, a pattern of postings that does not involve any kind of specialized terminology at all, but that merely deals with how to phrase a string of simple words, is indicative of one of the following: 1.) a complete novice (which is acceptable); 2.) a translator who is capable but lazy (unacceptable); 3.) an individual who has chosen the wrong profession (which merits compassion, but which should prompt sober reflection on the part of the poster); or 4.) a person suffering from some form of aphasia (which requires healing powers beyond anything an online community of translators is capable of offering).
| | | | Paul Cohen
Greenland
Local time: 21:52
German to English
+ ...
TOPIC STARTER | Chasing away good answers -- and good answerers | Dec 20, 2009 |
Laurent KRAULAND wrote: Heinrich Pesch wrote:
This site is for making money, not for generating trustworthy glossaries. So only changes are implemented that generate more traffic and income for the site.
Regards
Heinrich how about the scenario in which site traffic will decrease due to the appalling quality of KudoZ answers and other problems, such as those described in the OP? As with the so-called "crowdsourcing", bad answers chase good answers away. If this site and other ones can live on that, fine... [Edited at 2009-12-18 10:05 GMT]
If the site and others can live on that, indeed!
I’ve heard this site referred to by Henry D. (the owner) as a kind of telephone book. In other words, the site has no control over who uses ProZ as a service. Everyone who wants to be listed is included. It’s open to everyone. According to this line of reasoning, ProZ cannot be blamed for the flagging quality of KudoZ, or for rampant dumping rates for that matter.
I don’t believe such arguments. I think this site could do a great deal more to empower high-quality translators and discourage the wannabes from directing the show.
It’s also interesting that Larent brought up crowdsourcing. KudoZ has many things in common with crowdsourcing. It’s done for free and the resulting product often leaves much to be desired.
I agree with Heinrich that the site is profit driven, as any good business should be. However, the Internet is a rapidly-growing, dense jungle, and it’s becoming increasingly difficult to locate top-quality translators using this site’s directory because many outstanding language specialists are fed up with all the shenanigans and unwilling to keep “playing the game”.
As a result, many of us are not getting much bang for our buck on this site. That is the bottom line for me when it comes to questions like: “How can we make the KudoZ system fairer and more professional?”
With all its faults, KudoZ is virtually the only way that we can stand out from the crowd in the directory, short of going after the “P” PRO Certificate which, in my opinion, is a highly controversial aspect of this site -- and that, ultimately, is the reason why I won’t be renewing my paid membership when it runs out in January, 2011.
Eventually, ProZ may become a victim of its own success and the leading translators in the industry will probably migrate to greener pastures, if they haven’t already started to leave. There is nothing really new about my proposals. By consistently ignoring the ongoing quality concerns of translators, the site is alienating many of its very best members and actually creating a niche on the market for websites dedicated solely to true professionals.
If there were another high-profile website out there where we could stand out from the crowd and make truly interesting contacts, I’m sure many of us would have no qualms about paying substantial membership fees. That’s also a potential business proposition that I’m sure could put ProZ in an unpleasant position. After all, doesn’t the name “ProZ” imply that everyone on this site is a professional and that this is not just a venue for “adequate” translators?
ProZ must be keenly aware of that problem, but since the bottom line for the site is the same as that of its members -- namely “what gives me the biggest payoff in the end?” -- I don’t expect a major shift in policy anytime in the near future.
And so it goes.
We’ve heard from a number of members here that they have more or less turned a blind eye to KudoZ and given up on changing the system. As Woodstock so aptly put it, it helps the “blood pressure” not to get too worked up about it.
I couldn’t agree more.
[Edited at 2009-12-20 13:57 GMT]
| | | | | Pages in topic: [1 2] > | To report site rules violations or get help, contact a site moderator: You can also contact site staff by submitting a support request » How to make the KudoZ system fairer and more professional? | Anycount & Translation Office 3000 | Translation Office 3000
Translation Office 3000 is an advanced accounting tool for freelance translators and small agencies. TO3000 easily and seamlessly integrates with the business life of professional freelance translators.
More info » |
| | CafeTran Espresso | You've never met a CAT tool this clever!
Translate faster & easier, using a sophisticated CAT tool built by a translator / developer.
Accept jobs from clients who use Trados, MemoQ, Wordfast & major CAT tools.
Download and start using CafeTran Espresso -- for free
Buy now! » |
|
| | | | X Sign in to your ProZ.com account... | | | | | |
Login to reply/comment 
