| Pages in topic: [1 2] > | Selection of KudoZ answers based on number of "agrees" Thread poster: Muriel Vasconcellos
|
|---|
I recently went in to look at one of my KudoZ questions from the "outside" (i.e., just giving the # without the asker's ID), and saw for the first time an instruction telling me to choose a particular answer (i.e., the one with the most "agrees"). I was really surprised. For me, the number of "agrees" by itself doesn't mean a whole lot. It often happens that the first answer gets a lot of "agrees" because it's the first thought that came to mind and a lot of colleagues pounce on the question the... See more I recently went in to look at one of my KudoZ questions from the "outside" (i.e., just giving the # without the asker's ID), and saw for the first time an instruction telling me to choose a particular answer (i.e., the one with the most "agrees"). I was really surprised. For me, the number of "agrees" by itself doesn't mean a whole lot. It often happens that the first answer gets a lot of "agrees" because it's the first thought that came to mind and a lot of colleagues pounce on the question the moment it's posted. I look for second thoughts and afterthoughts, clear explanations, and good references. I am also influenced by "agrees" from colleagues whose work I know and trust. I think it's a mistake to encourage selection of answers based solely on the number of "agrees." I would urge that this message be removed. ▲ Collapse
| | | | | What words were used? | Nov 22, 2005 |
It has been a long time since I posted a question on KudoZ, and I cannot recall anything that specifically urged the selection of the answer with the most agree votes. I recall that the asker was advised to select the answer that was "most helpful." I wonder if the wording has changed. Can you post the wording that gave that impression?
I wonder if they just meant to say, "if you find yourself unable to choose, the number of agree votes may be a good indication to go by," or somethi... See more It has been a long time since I posted a question on KudoZ, and I cannot recall anything that specifically urged the selection of the answer with the most agree votes. I recall that the asker was advised to select the answer that was "most helpful." I wonder if the wording has changed. Can you post the wording that gave that impression?
I wonder if they just meant to say, "if you find yourself unable to choose, the number of agree votes may be a good indication to go by," or something to that effect. I say this because I know that one of the inherent problems in KudoZ is that the asker frequently has no clue. The target language frequently happens to be a language that the asker does not know at all (that is why he posted the question). So when the answers come in, such an asker is at a complete loss, and in such a situation, the advise to go by the agree votes is not bad. But it would concern me if the wording gives a different impression. It would be helpful if you could retrieve the wording and share it with us.
This is also an occasion for a suggestion to make: I suggest that, given the fact that some askers are unable to choose, the prompt should say something like:
If you can comfortably select the answer most helpful to you, please go ahead and make your selection now. If, however, you cannot choose, either because you do not know the target language, or because several answers have been equally helpful, please select "Let the KudoZ robot choose." The KudoZ robot will select the answer with the highest net score of agree votes by peers. The robot will make the selection and close the question after xx days from the posting of the last answer received.
Something like that. ▲ Collapse
| | | | Muriel Vasconcellos 
United States
Local time: 00:08
Member (2003)
Spanish to English
+ ...
TOPIC STARTER | This was NOT on the Asker's page | Nov 22, 2005 |
Hi! I found this message by accident because you don't see it on the Asker's link. I had asked a question on behalf of someone else, and I searched the link as an outsider would, to be sure I had given the correct URL. When I entered the question that way, the usual boxes didn't appear. Instead, I saw the prompt that I'm referring to underneath the question with the most "agrees." The ProZ software must have recognized me as the Asker nonetheless, even though I didn't go there via the Asker's li... See more Hi! I found this message by accident because you don't see it on the Asker's link. I had asked a question on behalf of someone else, and I searched the link as an outsider would, to be sure I had given the correct URL. When I entered the question that way, the usual boxes didn't appear. Instead, I saw the prompt that I'm referring to underneath the question with the most "agrees." The ProZ software must have recognized me as the Asker nonetheless, even though I didn't go there via the Asker's link.
I don't think the Robot should be encouraged, either. For the same reason that I disagree with urging colleagues to select the answer with the most "agrees." That's just one of several criteria that should enter into the equation. I think it would be much more educational to have a "Help" link with suggestions on what to look for in an answer. I would be more than happy to contribute some of the criteria that I follow. I use quite a few interesting criteria. The *number* of "agrees" alone, without taking into account the comments and who makes them, isn't even on my list.
Perhaps that could be a project! "Education" of this kind might help to improve the quality of the answers, stop answerers from filling up the site with *irrelevant* quotes, and better reward the more serious answers.
I can't give you the exact wording because once I graded the answer the prompt disappeared. I have no other questions pending.
[Edited at 2005-11-22 04:16] ▲ Collapse
| | | | | Human selection versus machine selection | Nov 22, 2005 |
When the selection is done by an asker who happens to know the target language, most askers probably go through the same list of criteria that you go through when you select the winning answer. ProZ.com suggests "the most helpful answer." Different askers may interpret this somewhat differently on different occasions, but in the end, everybody tends to choose an answer that is correct, full, well-documented, and well-presented.
The problem arises when there is no way for the asker t... See more When the selection is done by an asker who happens to know the target language, most askers probably go through the same list of criteria that you go through when you select the winning answer. ProZ.com suggests "the most helpful answer." Different askers may interpret this somewhat differently on different occasions, but in the end, everybody tends to choose an answer that is correct, full, well-documented, and well-presented.
The problem arises when there is no way for the asker to determine the relative merits of the various answers, or when a number of them seem of equal merit and helpfulness.
It is at this point that agree votes become a workable indication. Since the asker is unable to choose, peers are able to help, not only by their votes, but also by their comments.
But if, in the end, the asker still cannot make up his mind (and this should, ideally, represent a minority of cases) the robot should be allowed to do its work. The robot is utterly unable to do more than count votes (a major feat, if you ask me). It already does that when the asker fails to select an answer. All I am suggesting as a minor tweak is to change the wording so that the asker understands his role versus the robot's role, and so that the robot is, in essence, given a "go ahead" by the asker. In other words, there comes a point, sometimes, when the asker should be encouraged to explicitly defer to the robot.
[Edited at 2005-11-26 19:30] ▲ Collapse
| | |
|
|
|
Muriel Vasconcellos
United States
Local time: 00:08
Member (2003)
Spanish to English
+ ...
TOPIC STARTER | Use of the Robot | Nov 22, 2005 |
Though I've earned quite a few points thanks to the Robot, I still don't like the idea. To begin with, R2D2 only kicks in long after the fact, so he doesn't offer a solution for the asker. He just awards points and lets the answerer put something in the glossary. I think it would be better to leave the question unanswered--simply remove it when it gets stale.
I personally believe that translators should work only into languages that they are extremely familiar with, if not native in... See more Though I've earned quite a few points thanks to the Robot, I still don't like the idea. To begin with, R2D2 only kicks in long after the fact, so he doesn't offer a solution for the asker. He just awards points and lets the answerer put something in the glossary. I think it would be better to leave the question unanswered--simply remove it when it gets stale.
I personally believe that translators should work only into languages that they are extremely familiar with, if not native in. I shudder to think that they might not know the target language well enough to judge the quality of the answers based on knowledgeable criteria. This drags down the profession, and then we all lose.
It also worries me that people who are that severely handicapped in translation are "enfranchised" to grant points and put the selected answer in the glossary.
When I see that a newbie has a consistent pattern of picking irrelevant, non-native answers, I filter them through my dashboard. (I don't want to screen ALL non-native speakers, because some of them are excellent translators.) That solves my problem, but their judgments still contaminate the database.
Again, I'd like to suggest developing a "Help" link with some coaching for newbies on what to look for in an answer. I'm sure there would be some very rich and valuable input. ▲ Collapse
| | | | | Many KudoZ questions are posted by non-translators | Nov 22, 2005 |
There are many reasons some askers do not know the source or target language of their question. Sometimes, as you have observed, the reason is the unprofessional practice of accepting a translation job involving a language that the translator barely knows. But there are many occasions where the reason is perfectly understandable. Here are two common scenarios:
Scenario 1: A German-English translator, for instance, may be working with a subject matter involving political history. The... See more There are many reasons some askers do not know the source or target language of their question. Sometimes, as you have observed, the reason is the unprofessional practice of accepting a translation job involving a language that the translator barely knows. But there are many occasions where the reason is perfectly understandable. Here are two common scenarios:
Scenario 1: A German-English translator, for instance, may be working with a subject matter involving political history. The material may, on occasion, include expressions in Russian, Turkish, or Arabic -- language which the translator does not know at all. The translator may wish to verify his understanding of these expressions. Upon posting the question, he is less likely to be a good judge of the answers than if the question was about a German expression. The agree votes (and the comments of peers) are very helpful in this kind of situation.
Scenario 2: Someone has a linguistic question that has nothing to do with a translation project. The person may not even be a translator. For some reason, he just happens to wish to know how certain expressions are stated in Afrikaans, Swedish, or Japanese. He posts the question and receives several, different answers. He is understandably unable to choose an answer.
Scenario 1 involves unfamiliarity with the source language, while Scenario 2 involves unfamiliarity with the target language. Scenario 1 is fairly common and unavoidable in the course of many types of translation work. On the other hand, Scenario 2 arises from the simple fact that KudoZ is still open to the use of anybody, even non-members, albeit with limitations. So long as any user can post a question, there is no way to avoid this sort of situation.
So as you can see, although the overwhelming majority of questions that arise from the daily work of translators working in their language pairs normally do not call for delegating the selection process to the robot, the robot (relying on agree votes, the only indicator it can read) can perform a decent function in other situations.
It is not perfect (nothing about KudoZ is), but it is decent.
I do, however, encourage you to share any help tool that you have developed with the ProZ.com staff. I am sure we all can benefit from your insights, not just the newbies. ▲ Collapse
| | | | Muriel Vasconcellos
United States
Local time: 00:08
Member (2003)
Spanish to English
+ ...
TOPIC STARTER | Conclusion--or beginning? | Nov 22, 2005 |
Thanks for taking the time to answer. You write as if you have the final word on this subject, but I still think there's need for change. As I mentioned earlier, the Robot doesn't kick in until weeks after the fact, so it is of virtually not use to the Asker. I really think it's a waste, and I think there needs to be some solid discussion on this.
The present system leads to contamination of the database and frustrated professionals. When a seasoned translator (I have been in the fi... See more Thanks for taking the time to answer. You write as if you have the final word on this subject, but I still think there's need for change. As I mentioned earlier, the Robot doesn't kick in until weeks after the fact, so it is of virtually not use to the Asker. I really think it's a waste, and I think there needs to be some solid discussion on this.
The present system leads to contamination of the database and frustrated professionals. When a seasoned translator (I have been in the field more years than I care to admit and I taught translation at Georgetown University for many years) takes the time to help new people out, only to get the answer "rejected" in favor of some sort of alingual mish-mash that all the non-natives have agreed to. This happens ALL THE TIME EVERY DAY.
Suggestion: Perhaps a criterion could be set up before a person could award points and put entries in the glossary. Say, for example, that the Asker is free to pick an answer at any point and close the question. But the right award points and put entries in the glossary would be earned by having xx number of answers accepted by colleagues.
It's worth giving more attention to this issue, because the current situation is the source of a lot of contention on the site.
I'm sorry I won't be able to continue this discussion right now. I'm packing for a trip and will be leaving early in the morning. ▲ Collapse
| | | | | Believe me, I have no final word on any subject | Nov 22, 2005 |
I just happen to find your keen observations very helpful, and I wanted to echo your interest in the subject.
I was first intrigued by what you reported about being prompted to select the answer with the most agree votes. I was curious to find out what the actual wording was. I wondered aloud if perhaps they had meant something slightly different. I even ventured a wild suggestion to (1) tweak the wording, and (2) tweak the process (allow those who do not wish to choose an answer to... See more I just happen to find your keen observations very helpful, and I wanted to echo your interest in the subject.
I was first intrigued by what you reported about being prompted to select the answer with the most agree votes. I was curious to find out what the actual wording was. I wondered aloud if perhaps they had meant something slightly different. I even ventured a wild suggestion to (1) tweak the wording, and (2) tweak the process (allow those who do not wish to choose an answer to let the robot kick in sooner by declaring that they do not wish to make a selection).
I often make wild suggestions like that.
The next observation I made was about individuals who post questions in language pairs they may not know well. My point was simply that it is not always a case of doing translation work in a language outside one's mastered languages. I think the two scenarios I outlined are pretty common.
And in my last post, I encouraged you, as I still do, to share any methodology that you think would help make the selection more "appropriate." I, for one, could benefit from any advice you have on this subject.
I also appreciate what you said in your last post about the glossary.
So, as you can see, I have no final words, no real resolution, just a willingness to engage in a good, helpful conversation.
I wish you a good trip and happy holidays.
[Edited at 2005-11-22 10:33] ▲ Collapse
| | |
|
|
|
| Hear, hear ..... | Nov 22, 2005 |
Muriel said : When a seasoned translator (...) takes the time to help new people out, only to get the answer "rejected" in favor of some sort of alingual mish-mash that all the non-natives have agreed to. This happens ALL THE TIME EVERY DAY
Although Muriel is referring to the system in general, I'd like to discuss where things really can and do go haywire with robot grading/majority votes.
This happens, in particular, on the Portuguese to English site daily, so much so ... See more Muriel said : When a seasoned translator (...) takes the time to help new people out, only to get the answer "rejected" in favor of some sort of alingual mish-mash that all the non-natives have agreed to. This happens ALL THE TIME EVERY DAY
Although Muriel is referring to the system in general, I'd like to discuss where things really can and do go haywire with robot grading/majority votes.
This happens, in particular, on the Portuguese to English site daily, so much so that many of the seasoned translators Muriel refers to have just given up. The sub-site should more aptly be renamed Portuguese to Porglish as it presently stands. (no disrespect meant at ALL to those Portuguese colleagues who do participate and do give very sound answers - those of you reading this know which crowd I'm referring to) The sub-site is not moderated, our moderator apparently resigned and many of the glossary entries are sub-standard to say the least.
It serves no purpose to discuss these entries with an English speaking moderator who doesn't have a perfect command of Portuguese - in other words, the sub-site is currently rudderless.
Although I took the trouble of sending site management the names of some native English candidates who all have a really excellent command of the source language (Muriel, Todd, Sormane etc) and was told invitations would be sent out, the process doesn't seem to have moved along any. Maybe I'm wrong, but I haven't been answered since I requested an update some time ago. Possibly somebody could update us here.
My concern is if the robot is used to grade some of those answers and no moderating is being done, novice translators are going to rely on them and clients are going to suffer - I therefore back Muriel's reasoning 100%.
No system is perfect but if the robot system is used, the sub-site in question NEEDS stringent moderating. ▲ Collapse
| | | | Fred Neild (X)
English to Spanish
+ ...
| KudoZ is also a game | Nov 22, 2005 |
Muriel Vasconcellos wrote:
I don't think the Robot should be encouraged, either.
I think the Robot is another great ingredient that makes of KudoZ a stimulating game.
Of course, encouraging an asker to choose an answer based on the number of agrees is not a solution. If the asker cannot choose an answer on his/her own, then, maybe he shouldn't be translating in the first place.
But, the pollution in Google doesn´t change the fact that it is one of the most important tools for professional translators. The same can be said for KudoZ.
On the other hand, many times, I spend a lot of time trying to help a colleague. Then, you realize the asker probably didn't spend 1 minute to choose an answer, let alone use at least a couple of your 'filters'. Another case is when the asker didn't even bother to choose an answer or reject them all. Well, then the Robot comes in and awards the points to the winner so that the show can go on. Yes, this remains in the database, it's not perfect, but it is sometimes as useful as Google. As any other tool, you have to be skilled to use it. The problem is words don't chop hands, at least in a direct manner.
Is it specified in the database when the Robot chooses an answer automatically? Maybe it is time for a new 'label' Fuad... haha.
Finally, I believe the native/non-native issue should be treated separately.
| | | | Gina W
United States
Local time: 03:08
Member (2003)
French to English
Muriel Vasconcellos wrote:
I think it's a mistake to encourage selection of answers based solely on the number of "agrees." I would urge that this message be removed.
Yes, this is a mistake. While a large part of the time the answer with the most "agrees" might be correct, there are plenty of instances where that is not the case. I know that I have asked questions before and chosen an answer that was not the one with the most "agrees", and you are correct in that many times the answer that does have the most "agrees" is the first answer, whereas that is not necessarily the best answer.
| | | | Muriel Vasconcellos
United States
Local time: 00:08
Member (2003)
Spanish to English
+ ...
TOPIC STARTER | Case in point | Nov 26, 2005 |
Here is a perfect case of what I've been talking about. See http://www.proz.com/kudoz/1182987. The colleague who made the first suggestion (with 6 agrees), when she saw my suggestion, agreed that mine was better. NEVERTHELESS the Asker selected her answer. But the colleague, to emphasize her agreement with my answer, actually entered my answer in the glossary rather than hers.
This case cl... See more Here is a perfect case of what I've been talking about. See http://www.proz.com/kudoz/1182987. The colleague who made the first suggestion (with 6 agrees), when she saw my suggestion, agreed that mine was better. NEVERTHELESS the Asker selected her answer. But the colleague, to emphasize her agreement with my answer, actually entered my answer in the glossary rather than hers.
This case clearly shows that sometimes the answer with the most agrees is getting selected blindly with no attention whatsoever to the comments and discussion. ▲ Collapse
| | |
|
|
|
Nikki Graham
United Kingdom
Local time: 08:08
Spanish to English
Muriel Vasconcellos wrote: Here is a perfect case of what I've been talking about. See http://www.proz.com/kudoz/1182987. The colleague who made the first suggestion (with 6 agrees), when she saw my suggestion, agreed that mine was better. NEVERTHELESS the Asker selected her answer. But the colleague, to emphasize her agreement with my answer, actually entered my answer in the glossary rather than hers. This case clearly shows that sometimes the answer with the most agrees is getting selected blindly with no attention whatsoever to the comments and discussion.
Hi Muriel,
it could be a genuine mistake. The asker may have meant to select your answer and then clicked on the wrong box. I think it happens all the time. I've received notification before of a question being closed and the asker's comment has been "Thanks Nikki", yet I didn't get the points... I don't think the "close question" layout is very user friendly, it often confuses me.
On the other hand, I do agree with you. There is too much of the blind leading the blind in Kudoz and it is extremely frustrating.
| | | | | Where is that message | Nov 26, 2005 |
Dear Muriel
Since you launched this discussion I wanted to take part in it but you based the topic you want to discuss (Selection of KudoZ answers based on number of "agrees") on a sentence that none of us has seen anywhere.
As a old user of KudoZ I only can say that I have never seen any instruction telling to chose the answer with the most agrees.
It would not have been the same if you had brought the topic as a general thought, but discussing a text that... See more Dear Muriel
Since you launched this discussion I wanted to take part in it but you based the topic you want to discuss (Selection of KudoZ answers based on number of "agrees") on a sentence that none of us has seen anywhere.
As a old user of KudoZ I only can say that I have never seen any instruction telling to chose the answer with the most agrees.
It would not have been the same if you had brought the topic as a general thought, but discussing a text that nobody can find is useless. If it could be found it could be reworded or removed.
Now, referring to examples in which the asker has chosen the answer with most agrees is like trying to make your point valid. But then I wonder...Did I misunderstood your first message? Was your point a problem with a sentence you saw somewhere or with askers who chose the answer with most agrees? I'm a bit lost by the reasoning.
Claudia
Muriel Vasconcellos wrote:
I recently went in to look at one of my KudoZ questions from the "outside" (i.e., just giving the # without the asker's ID), and saw for the first time an instruction telling me to choose a particular answer (i.e., the one with the most "agrees"). I was really surprised. would urge that this message be removed.[/quote] ▲ Collapse
| | | | Muriel Vasconcellos
United States
Local time: 00:08
Member (2003)
Spanish to English
+ ...
TOPIC STARTER | Clarification | Nov 26, 2005 |
I, too, had never seen this message before. I found it by a fluke, as I explained above. Maybe the Robot put it there! To find it, I think, you would have to take a question you had asked that was still pending (I have none pending), copy the URL, and then open a new link to the Internet and search for that URL. Failing that, I guess there's no way to see it. I'm going to post a question and see if I can capture it again.
Seeing the message was a shocker for me and prompted me to ra... See more I, too, had never seen this message before. I found it by a fluke, as I explained above. Maybe the Robot put it there! To find it, I think, you would have to take a question you had asked that was still pending (I have none pending), copy the URL, and then open a new link to the Internet and search for that URL. Failing that, I guess there's no way to see it. I'm going to post a question and see if I can capture it again.
Seeing the message was a shocker for me and prompted me to raise the larger issue.
The example I cited was in support of my basic point--that the number of agrees can be misleading because later answers don't get seen by as many people. In this case, the asker was impressed by the number of agrees and failed to notice that the first answerer had deferred to the later answer.
[Edited at 2005-11-26 19:11]
[Edited at 2005-11-26 19:27] ▲ Collapse
| | | | | Pages in topic: [1 2] > | To report site rules violations or get help, contact a site moderator: You can also contact site staff by submitting a support request » Selection of KudoZ answers based on number of "agrees" | CafeTran Espresso | You've never met a CAT tool this clever!
Translate faster & easier, using a sophisticated CAT tool built by a translator / developer.
Accept jobs from clients who use Trados, MemoQ, Wordfast & major CAT tools.
Download and start using CafeTran Espresso -- for free
Buy now! » |
| | TM-Town | Manage your TMs and Terms ... and boost your translation business
Are you ready for something fresh in the industry? TM-Town is a unique new site for you -- the freelance translator -- to store, manage and share translation memories (TMs) and glossaries...and potentially meet new clients on the basis of your prior work.
More info » |
|
| | | | X Sign in to your ProZ.com account... | | | | | |
Login to reply/comment 
