How does automatic grading work? Thread poster: Sandra Petch
|
|---|
Hi
If an answer to a question gets two peer "agrees" and another answer gets two "agrees" and one "neutral" which is automatically graded as the "correct" answer?
In the case I have in mind, the answer with the two "agrees" and the "neutral" was automatically chosen. I'm curious as to why.
Best regards
Sandra
| | | | Natalie 
Poland
Local time: 21:19
Member (2002)
English to Russian
+ ...
MODERATOR SITE LOCALIZER | Sandra Petch
Local time: 21:19
French to English
+ ...
TOPIC STARTER | Strange rule? | Jul 12, 2009 |
Hi Natalie
Thanks for pointing me in the right direction. I figured the answer may be in the FAQs but couldn't find it!
This seems a strange rule though:
"The answer with the highest number of net agrees is selected. If two answers have the same number of net agrees, the last one entered is selected. The idea is that if two answers gather the same number of net agrees, the one that did so in a shorter period of time may be better."
One a... See more Hi Natalie
Thanks for pointing me in the right direction. I figured the answer may be in the FAQs but couldn't find it!
This seems a strange rule though:
"The answer with the highest number of net agrees is selected. If two answers have the same number of net agrees, the last one entered is selected. The idea is that if two answers gather the same number of net agrees, the one that did so in a shorter period of time may be better."
One answer got agrees after 2 minutes and 8 minutes.
One answer got agrees after 29 minutes, 1 day and 6 hours PLUS A NEUTRAL.
Logically then, answer number one should have been chosen but in fact answer number two was chosen.
I guess my bugbear really is with people who have time to post a question but no time or inclination to close it!
Well, as my cousin says, "if it's not life-threatening, don't worry about it!"
Sandra ▲ Collapse
| | | | Natalie
Poland
Local time: 21:19
Member (2002)
English to Russian
+ ...
MODERATOR SITE LOCALIZER
I am going to call the site staff to take a look at the situation.
Have a great Sunday,
Natalia
| | |
|
|
|
Sandra Petch
Local time: 21:19
French to English
+ ...
TOPIC STARTER
Thanks Natalia!
I'm not asking that points be reallocated in this case as I know it would cause hard feelings. I was just curious about "the rules."
Have a good Sunday too,
Sandra
[Edited at 2009-07-12 10:38 GMT]
| | | | | This is what the rule means | Jul 12, 2009 |
Sandra Petch wrote:
"The answer with the highest number of net agrees is selected. If two answers have the same number of net agrees, the last one entered is selected. The idea is that if two answers gather the same number of net agrees, the one that did so in a shorter period of time may be better."
One answer got agrees after 2 minutes and 8 minutes.
One answer got agrees after 29 minutes, 1 day and 6 hours PLUS A NEUTRAL.
Logically then, answer number one should have been chosen but in fact answer number two was chosen.
Sandra, you may have misunderstood the selection rule. What the selection rule says if two answers got the same number of agrees, then the answer that was posted later will be selected. (Not the one with the last peer-grade - perhaps this is what you thought.)
In your example, I assume answer No.1 was posted first, and answer No.2 was posted some time later. They collected the same number of agrees, and No. 2 was selected, as that was the answer posted later.
This is perfectly in line with the selection rule.
The rule is logical. Think about it: all those people that peer-graded the second answer definitely had a chance to look at the first answer, and they chose to agree with the second one. Another way of looking at it (and this is what written in the FAQ) is that the time elapsed since posting the first answer until the time of grading is longer than the time elapsed since posting the second answer until the time of grading. Therefore, the second answer collected the same amount of agrees in a shorter period of time.
I hope this helps understanding what is going on.
Katalin
| | | | Sandra Petch
Local time: 21:19
French to English
+ ...
TOPIC STARTER | Thanks Katalin | Jul 13, 2009 |
Yes, that makes it clear and it does make sense.
On the other hand, as the second answer also got a neutral (a strong neutral, more a disagree!), surely that should be taken into account?
I've started a second thread about users who systematically leave their questions to be automatically graded and it seems I'm not the only one to find this disagreeable.
Thanks again for your help.
| | | | | Neutral has no effect when computing net agrees | Jul 13, 2009 |
Sandra Petch wrote:
On the other hand, as the second answer also got a neutral (a strong neutral, more a disagree!), surely that should be taken into account?
A neutral is just neutral, it is not computed when evaluating the net agrees of an answer (agrees minus disagrees). If peer graders desire to leave a negative mark then they should use disagree instead of neutral.
Regards,
Enrique
| | | | To report site rules violations or get help, contact a site moderator: You can also contact site staff by submitting a support request » How does automatic grading work? | CafeTran Espresso | You've never met a CAT tool this clever!
Translate faster & easier, using a sophisticated CAT tool built by a translator / developer.
Accept jobs from clients who use Trados, MemoQ, Wordfast & major CAT tools.
Download and start using CafeTran Espresso -- for free
Buy now! » |
| | Anycount & Translation Office 3000 | Translation Office 3000
Translation Office 3000 is an advanced accounting tool for freelance translators and small agencies. TO3000 easily and seamlessly integrates with the business life of professional freelance translators.
More info » |
|
| | | | X Sign in to your ProZ.com account... | | | | | |
Login to reply/comment 
