| Pages in topic: < [1 2 3] > | KudoZ averages Thread poster: James Calder
|
|---|
James Calder 
United Kingdom
Local time: 12:51
Spanish to English
+ ...
TOPIC STARTER | It all evens out in the end | Feb 9, 2005 |
[quote]Kirill Semenov wrote:
[quote] in the long run, fluctuations are nullified. quote]
Absolutely, Kirill. If you have a good command of your language pair and your fields, then your answers will be chosen on a fairly regular basis.
As a user of the system I would rather have three "good" answers from people with a knowledge of the area than ten "bad" answers from people who know nothing about it, haven't even read the question properly and think, "Hey, what the hell, it's only a bit of fun, isn't it?", before submitting their confidence-level 5 answer with not a reference or explanation in sight. That approach doesn't help me as an asker and it doesn't do much for KudoZ as a translation resource. Introduce a points per question average and those people might think twice before submitting an answer. After all, surely the most fundamental point about Kudoz is that it's supposed to be helpful.
Regards
James
| | | | Kirill Semenov
Ukraine
Local time: 14:51
Member (2004)
English to Russian
+ ...
| Law of large numbers | Feb 9, 2005 |
writeaway wrote:
I think you may have misunderstood my incoherent English.
It's perefectly OK, I'm not a English-speaker, too.
The key is the statistical law of large numbers. The "reliability ratio" proposed (points or question won divided on the number of answers given) shows the general ratio of "helpfulness". I know people whose ratio is more than 2.8, and I know persons whose one is less than 0.5. If almost 3 answers of a person out of four are choosen as "the most helpful" it matters! It shows the person/translator is a very reliable! On the other hand, if a person gives a good (even subjectively) answers once on 15-20 answers, (s)he's answers are more probably just a "lucky shots". That's the meaning.
Of course, any statistics starts from large numbers, so I always noted that this "reliability ratio" should be calculated for proZians only after they give, say, a hundred answers or so.
| | | | James Calder
United Kingdom
Local time: 12:51
Spanish to English
+ ...
TOPIC STARTER | That's what I've been getting at | Feb 9, 2005 |
[quote]Kirill Semenov wrote:
I know people whose ratio is more than 2.8, and I know persons whose one is less than 0.5. If almost 3 answers of a person out of four are choosen as "the most helpful" it matters! It shows the person/translator is a very reliable! On the other hand, if a person gives a good (even subjectively) answers once on 15-20 answers, (s)he's answers are more probably just a "lucky shots". That's the meaning.
quote]
Spot on again, Kirill. Of course it matters. We've already got the information, it's lurking away somewhere on our profile pages. Why not get it out in the open? What's the harm in that?
James
| | | | | Clarification please from those who want the statistics | Feb 9, 2005 |
I haven't been able to determine why those in favour of showing statistics related to KudoZ want these statistics. What are you going to use the information for? What will the statistics show? In other words, what conclusions will you derive from the information. I am asking translators in particular.
Henry mentioned an interesting fact during the Oxford conference. He said that customers (agencies) wanted "statistics" to make it easier to decide who should be given jobs.
| | |
|
|
|
Andy Watkinson
Spain
Local time: 13:51
Member
Catalan to English
+ ...
| Purpose of statistics | Feb 10, 2005 |
I confess I'm unable to understand why anyone should summarily dismiss the idea of having an indicator of reliability.
The advantages for all involved are obvious.
- For both "outside" askers and regular translators on the site who ask questions, it is obviously to their advantage to be able to see that translator xxx is reliable yyy percent of the time compared to translator zzz, irrespective of the number of points they may each have accumulated, which we all know is... See more I confess I'm unable to understand why anyone should summarily dismiss the idea of having an indicator of reliability.
The advantages for all involved are obvious.
- For both "outside" askers and regular translators on the site who ask questions, it is obviously to their advantage to be able to see that translator xxx is reliable yyy percent of the time compared to translator zzz, irrespective of the number of points they may each have accumulated, which we all know is irrelevant.
When booking a flight, you don't want to know how many passengers they've carried that year.
You want to know how many of those passengers arrived on time (preferably in one piece and with their luggage:-).
- For clients, see above.
The "objections" I've seen so far, both in this thread and that posted above by Stuart Allsop, are far from convincing.
They broadly fall into two categories:
a) "This turns the site into some kind of Big Brother". "I don't want to be spied on." "What's behind all of this?"
Answer: Ridiculous. The raw information is already there. James is merely suggesting making the information more readily available and comprehensible. An eminently sensible idea, I would have thought. Whether it is practical or not is another matter, but the aim/idea I find perfectly valid.
b) "This will discourage people from answering as they will not want their ratio to be reduced for simply wanting to help someone out."
Answer: An admirable sentiment and one I share.
Those in my language pair who know me will also know that I regularly post "quasi-answers" with a confidence level of 1 just to provide additional info or anything which I think might prove useful. I also, rather unsuccessfully, simply post the link to the glossary page where there are several dozen possible translations of "stakeholder", for example, in order to remind people that there is a very useful tool there if only they'd make use of it.
And I take the opportunity to thank those colleagues who also, instead of making "easy" points, prefer to "point" (sorry) the asker in the general direction of the glossaries.
If someone sees b) as a problem, the site can simply introduce a feature whereby those who wish to offer help without being graded can choose a confidence level of "zero" which means that statistically they haven't answered. Any trouble with that?
If some of those who have participated had taken the time to read (or in my case, re-read) the thread posted by Kirill, they would have found that Henry, the site founder thought the idea useful, desirable and his only objection was on technical/statistical grounds, not in principle.
Henry: "Two steps I will take (assuming no objections) Sep 21, 2004
Thanks, Stuart, for raising again this interesting topic. Your suggestion is welcome and good. KudoZ can be improved, and I believe we should continue to look for ways. I also agree with Kirill's feeling that showing more about an answerer's history might diminish the amount of guesswork."
How, exactly, this can be undertaken is a matter of opinion. It may not be possible due to practical reasons as Henry setsout in the thread, but to dismiss the idea out of hand I find rather disappointing, honestly.
"Disappointed in Barcelona" ▲ Collapse
| | | | Andy Watkinson
Spain
Local time: 13:51
Member
Catalan to English
+ ...
Below is your post.
I've substituted "Kudoz Leaders" for statistics.
"I haven't been able to determine why those in favour of showing "KudoZ Leaders" want these statistics. What are you going to use the information for? What will the statistics show? In other words, what conclusions will you derive from the information. I am asking translators in particular."
----------------------------------------------------------------
Marijke, are you agai... See more Below is your post.
I've substituted "Kudoz Leaders" for statistics.
"I haven't been able to determine why those in favour of showing "KudoZ Leaders" want these statistics. What are you going to use the information for? What will the statistics show? In other words, what conclusions will you derive from the information. I am asking translators in particular."
----------------------------------------------------------------
Marijke, are you against showing the Kudoz Leaders too?
If not, there is no possible reason why you should be against a more refined version of these statistics.
As I mentioned in my earlier post, which do you prefer to know?
How many computers company XXX sells? or,
What percentage/ratio of "reliable" computers does company XX sell?
Obvious. ▲ Collapse
| | | | danya
Local time: 14:51
English to Russian
+ ...
| I beg to disagree | Feb 10, 2005 |
Kirill, I remember you have long been an active supporter of this modus putandi, but look: sometimes it is matter of minutes, who answers first; sometimes the wrong opinion prevails; some questions have unlimited range of fitting answers (esp. when literary translation is concerned); answerer's habits also differ - some answer only if it's a sure win, some answer whatever rot they may think of, some give there comments in the answer field and it counts as an answer...
to recapitulate: whil... See more Kirill, I remember you have long been an active supporter of this modus putandi, but look: sometimes it is matter of minutes, who answers first; sometimes the wrong opinion prevails; some questions have unlimited range of fitting answers (esp. when literary translation is concerned); answerer's habits also differ - some answer only if it's a sure win, some answer whatever rot they may think of, some give there comments in the answer field and it counts as an answer...
to recapitulate: while there is no clear set of grading criteria, no ratio would reflect PRECISELY peer's expertise and skills, both kudoZ number and the ratio you propose can give one only a rough idea. IMHHHHHHHHO 8))) ▲ Collapse
| | | | James Calder
United Kingdom
Local time: 12:51
Spanish to English
+ ...
TOPIC STARTER | Points do not reflect accuracy/reliability/helpfulness | Feb 10, 2005 |
andycw wrote:
the site can simply introduce a feature whereby those who wish to offer help without being graded can choose a confidence level of "zero" which means that statistically they haven't answered. Any trouble with that?
An excellent idea, Andy. I don't see why it couldn't be put into practice.
In answer to Marijke, I find the current points system flawed. My KudoZ points tally is just a number. It is probably the most irrelevant and misleading piece of information on my profile page and doesn't tell anyone (prospective clients, fellow translators, KudoZ askers) anything about me as a translator. Either scrap it altogether or make it more relevant (but not foolproof, I admit) by adding a points per question average.
Secondly, I'm not going to use the information for anything. I'm not an agency or a statistics freak. I'm looking at this issue as an occasional asker of KudoZ questions. A KudoZ average/rating/ratio or whatever you want to call it will tell me an answerer's reliability/helpfulness in that area. As I say above, the current points system is no reflection of reliability/helpfulness whatsoever. You can make your way into the KudoZ top ten for the last 3 months simply by answering lots and lots of questions. Sooner or later you'll start amassing a few points and, hey presto, people think you're a reliable answerer and an authority on your language pair, and you start getting even more points (don't tell me askers, "agreerers" and "disagreers" aren't influenced by answerers' points totals). That's where the system is flawed. Show me how many questions that person has answered to break their way into the top ten and I'll tell you whether they're any good or not. I've said it before and I'll say it again and again until I'm blue in the face - the stats are already there, why not make them more accessible?
Just one more point before I sign off. It seems to me, having used this site for a couple of years now, that quite a few respected translators with a lot to offer in their fields have given up answering KudoZ questions because it's sometimes little more than a points free-for-all. By introducing a KudoZ average, which ought to discourage indiscriminate point-hunting, we might just attract some of these people back and thereby increase the quality of the resource. It would be interesting to know what former users of KudoZ feel about this.
[Edited at 2005-02-10 08:52]
| | |
|
|
|
Kirill Semenov
Ukraine
Local time: 14:51
Member (2004)
English to Russian
+ ...
| Typical Questions and Answers | Feb 10, 2005 |
Having been at proZ a bit more than two years, and being a mathematician/programmer in my "former life" I was always curious how things tick in kudoZ. I think, that by now I do feel how kudoZ area works and what are it's strong and weak sides as it is now. Not pretending to be a wise seer or anything -- mind you! - I just want to propose my own (and, naturally, very biased by my own views) FAQ on how KudoZ may look. I use a typical structure of arguments and answers/comments:
Argume... See more Having been at proZ a bit more than two years, and being a mathematician/programmer in my "former life" I was always curious how things tick in kudoZ. I think, that by now I do feel how kudoZ area works and what are it's strong and weak sides as it is now. Not pretending to be a wise seer or anything -- mind you! - I just want to propose my own (and, naturally, very biased by my own views) FAQ on how KudoZ may look. I use a typical structure of arguments and answers/comments:
Argument: The kudoZ (or proZ) is OK as it is. Don't change anything!
Comment: There is no perfection, unless in the world of Platonic ideal forms. Any complex system is never perfect and allows improvements. The goal is not to reach the perfection but to perfect ad infinitum.
A: We don't need kudoZ points at all!
C: We do. KudoZ is one of the major reasons why proZ is so popular. Leaving aside the great and very important aspect of socializing with your colleagues, learning new things and getting more language insights, the idea of KudoZ points is great because of a simple thing: while helping your fellows/competitors you became more competitive, skillful and visible yourself. It's a more advanced marketing (through collaboration) than an straightforward "everyone for himself" approach.
A: I just want to help others, and I don't need any points.
C: A "non-point area" should be implemented in kudoZ zone.
A: KudoZ points do not really reflect persons' skills and professionalism in translation art.
C: No, they don't. That's why we need a wider, more encompassing and more visible range of statistics related to kudoZ area/points. The current ranking system is not sophisticated.
A: The reliability ratio (points per answers) is a subject to random events (first of all, to the subjective opinion of askers).
C: Sure it is, but so are kudoZ points themselves. The statistics IS a discipline which analyze random events with a strict mathematical precision. Psychologically, we tend to treat our own asnwers as great and askers who don't choose them as "stupid". This is not true. Let's face it: in many cases our answers are not that perfect, and still askers grade them. Individual cases of "mistakes" (which often only look like mistakes), however frustrating feelings they may cause, are totally nullified in the long run. Moreover, in reality the effects of such fluctuations on our reliability ratio is tiny (supposing you've already given hundreds of answers). And please, don't forget that askers' and peers' subjectivity affects not only you, personally, but all other answerers. We are all in the same situation, so everyone of us is a subject to the same rules.
A: Showing reliability ratio does not reflect particular fields of translator's expertise.
C: Yes, so this ratio should be shown for each major field (like Technical, Business, Law, Art/Literary, etc.) in each language pair -- just a second number next to the current raw kudoZ points.
A: Reliability ratio will make people to give less answers.
C: Even if so, then why not? What's wrong if we start to think twice before answering or stop to write the first wild guesses which comes to our mind? If the goal of KudoZ system is to help each others, then thoughtful answers are the best way to provide such help. Also, reliability ratio is a great way to stop kudoZ hunters who give dozens of answers in an hour just as "lucky shots" to grab more points (empirical observations show that kudoZ hunters usually have a low reliability ratio).
A: There are people who answer just the very easy questions, while others prefer difficult. At the same time, the points are the same.
C: The reason is the current obscure definition of "Easy" vs. "Pro". In fact, "Easy" questions should go to "non-point area", with only "Pro" questions awarded with points.
A: Some people use kudoZ as "an answering machine" and abuse it with an incredible number of questions, without helping others at all.
C: Very true, and it will be only just if kudoZ system is based on "give-and-take" basis. In other words, there must be a connection between the points won and questions asked. Assuming that there are "Easy" (non-points) area and "Pro" (point-awarding) area, we may allow an asker to ask two or three questions in "Pro" area for each 1 KudoZ point the akser has; otherwise he may ask as many questions as (s)he wants in the "Easy" area (which is non-points and thus less popular).
Such approach would provide some "give-and-take" balance into the kudoZ system. ▲ Collapse
| | | | Ruxi
German to Romanian
+ ...
And I have said it before, is:
1. KudoZ would work fair and good without points.We are not dogs to receive a candy for a good job.(I don't mean to offend anybody).
2. KudoZ points do not reflect anything and above all the quality of a translator.
3. Jobs should not be granted on such a basis (KudoZ statistics).It is totally wrong.
4. A good alternative would be to enable askers to grant points to all correct answers to a question, even if they chose only one. It could be ... See more And I have said it before, is:
1. KudoZ would work fair and good without points.We are not dogs to receive a candy for a good job.(I don't mean to offend anybody).
2. KudoZ points do not reflect anything and above all the quality of a translator.
3. Jobs should not be granted on such a basis (KudoZ statistics).It is totally wrong.
4. A good alternative would be to enable askers to grant points to all correct answers to a question, even if they chose only one. It could be more correct opinions and then all should be rewarded.
What about the questions in one language? There is no translation there, but grammar or spelling aspect.How are they granting points on such questions?
5. Someone who answers rarely or not at all in KudoZ has different reasons: rare used language pair, lack of time, or even he/she gave up because he/she was offended several times and considers the system unfair.
And I keep asking myself and you the same question: why do anyone need such a statistics? To show what?
Jobs should be granted exclusively on profesional skills and translation samples.
Personally I would prefer to help people with answers if there are are not points given, because it would be more fair and no one would be offended.It is not fair if anyone receives points for a good (or even bad answer, depending on the asker too) and is full of medals, while another one also giving good answers does not receive points and his/her effort is not recognised.Either we are all treated equally, or no one is treated in no way.
Ruxi ▲ Collapse
| | | | | Stats do not really provide any added value and may be dangerous | Feb 10, 2005 |
James Calder wrote:
...the stats are already there, why not make them more accessible?
Yes the "pseudo" stats are there and everybody has come to their own conclusions but the stats do not seem to actually provide any insights.
My argument against having any kind of point system or KudoZ- related statistical information available is that nothing can really be concluded. We cannot say that XXX is a good translator nor, for that matter, reliable since nobody is answering questions day-in-day-out for years on end. We would only be leading translators and customers down the garden path because we as humans love to come to conclusions which, in the case of KudoZ, would inevitably be the wrong ones.
In my book, the only conclusion that we can arrive at based on the stats would be:
The person who got the points is good at terminology research
My suggestion would be to get rid of the whole point system and have done with it.
The examples others have quoted assume that the lack of points also infers something: the translator is unreliable.
This is dangerous in my viewpoint since we are encouraging other translators and customers to arrive at wrong conclusions. Do we really want this?
| | | | | Improving your statistics by hiding answers | Feb 10, 2005 |
Is there any point in introducing these "statistics" if they can be improved artificially simply by hiding answers which have not been awarded any points?
An example:
Someone answers two questions and is awarded 4 points for question #1 and nothing for question #2.
His average is then 2 points per question.
He then hides the answer that was not selected and, hey presto, his average becomes 4.
Ciao,
C
| | |
|
|
|
Kirill Semenov
Ukraine
Local time: 14:51
Member (2004)
English to Russian
+ ...
| Hiding doesn't help | Feb 10, 2005 |
Cilian O'Tuama wrote:
Is there any point in introducing these "statistics" if they can be improved artificially simply by hiding answers which have not been awarded any points?
Dear Cilian, there are two places where the number of answers is shown -- and the numbers are different!
The first one is "KudoZ Questions Answered" which you see when you click on KudoZ in any profile. This is not a real number of answers (hidden ones are not counted).
To get the real number, you have to open any _question_ this person asked and check Q/A numbers just under his/her picture. This is the real one.
In other words, hiding answers does not really help.
| | | | | Marjike has hit the nail on the head | Feb 10, 2005 |
Marijke Singer wrote: James Calder wrote:
...the stats are already there, why not make them more accessible?
Yes the "pseudo" stats are there and everybody has come to their own conclusions but the stats do not seem to actually provide any insights. ... This is dangerous in my viewpoint since we are encouraging other translators and customers to arrive at wrong conclusions. Do we really want this?
How can one rely on stats that themselves are not based on any standards. Anyone on the www. can ask, anyone signed up at Proz can answer and Asker is free to pick any answer he/she wishes for no other reason than finding it most helpful. Questions can go from the sublime to the ridiculous. Nowadays many newbees (and a few oldbees) seem to be using Proz as a handy substitute to doing their own research. So the same old questions are often asked (and answered) time and time again. No one squashes, because they are easy points.
After almost 3 years on Proz, I know which of my colleagues are reliable pros and which are not. And points have nothing to do with it. Some of the best people on Proz no longer do Kudoz for a variety of reasons. And some of the most reliable help is provided in private mails between proz colleagues. Take the kudoz points as the fun they are, but please don't talk about kudoz statistics as a serious matter.
| | | | | OK Kirill, thanks for the correction. | Feb 10, 2005 |
| | | | | Pages in topic: < [1 2 3] > | To report site rules violations or get help, contact a site moderator: You can also contact site staff by submitting a support request » KudoZ averages | Anycount & Translation Office 3000 | Translation Office 3000
Translation Office 3000 is an advanced accounting tool for freelance translators and small agencies. TO3000 easily and seamlessly integrates with the business life of professional freelance translators.
More info » |
| | Wordfast Pro | Translation Memory Software for Any Platform
Exclusive discount for ProZ.com users!
Save over 13% when purchasing Wordfast Pro through ProZ.com. Wordfast is the world's #1 provider of platform-independent Translation Memory software. Consistently ranked the most user-friendly and highest value
Buy now! » |
|
| | | | X Sign in to your ProZ.com account... | | | | | |
Login to reply/comment 
