How does automatic grading work?
Thread poster: Sandra Petch
Sandra Petch
Local time: 09:18
French to English
+ ...
Jul 11, 2009

Hi

If an answer to a question gets two peer "agrees" and another answer gets two "agrees" and one "neutral" which is automatically graded as the "correct" answer?

In the case I have in mind, the answer with the two "agrees" and the "neutral" was automatically chosen. I'm curious as to why.

Best regards
Sandra


Direct link Reply with quote
 

Natalie  Identity Verified
Poland
Local time: 09:18
Member (2002)
English to Russian
+ ...

MODERATOR
Hi Sandra Jul 11, 2009

The answer to your question may be found in KudoZ FAQ:
http://www.proz.com/faq/terminology_term_help/kudoz/kudoz:_general.html#what_happens_if_a_help_question_isn_t_graded_

HTH,
Natalia


Direct link Reply with quote
 
Sandra Petch
Local time: 09:18
French to English
+ ...
TOPIC STARTER
Strange rule? Jul 12, 2009

Hi Natalie

Thanks for pointing me in the right direction. I figured the answer may be in the FAQs but couldn't find it!


This seems a strange rule though:

"The answer with the highest number of net agrees is selected. If two answers have the same number of net agrees, the last one entered is selected. The idea is that if two answers gather the same number of net agrees, the one that did so in a shorter period of time may be better."

One answer got agrees after 2 minutes and 8 minutes.
One answer got agrees after 29 minutes, 1 day and 6 hours PLUS A NEUTRAL.

Logically then, answer number one should have been chosen but in fact answer number two was chosen.

I guess my bugbear really is with people who have time to post a question but no time or inclination to close it!

Well, as my cousin says, "if it's not life-threatening, don't worry about it!"

Sandra


Direct link Reply with quote
 

Natalie  Identity Verified
Poland
Local time: 09:18
Member (2002)
English to Russian
+ ...

MODERATOR
Hi Sandra Jul 12, 2009

I am going to call the site staff to take a look at the situation.

Have a great Sunday,
Natalia


Direct link Reply with quote
 
Sandra Petch
Local time: 09:18
French to English
+ ...
TOPIC STARTER
Points Jul 12, 2009

Thanks Natalia!

I'm not asking that points be reallocated in this case as I know it would cause hard feelings. I was just curious about "the rules."

Have a good Sunday too,
Sandra

[Edited at 2009-07-12 10:38 GMT]


Direct link Reply with quote
 

Katalin Horváth McClure  Identity Verified
United States
Local time: 03:18
Member (2002)
English to Hungarian
+ ...
This is what the rule means Jul 12, 2009

Sandra Petch wrote:


"The answer with the highest number of net agrees is selected. If two answers have the same number of net agrees, the last one entered is selected. The idea is that if two answers gather the same number of net agrees, the one that did so in a shorter period of time may be better."

One answer got agrees after 2 minutes and 8 minutes.
One answer got agrees after 29 minutes, 1 day and 6 hours PLUS A NEUTRAL.

Logically then, answer number one should have been chosen but in fact answer number two was chosen.


Sandra, you may have misunderstood the selection rule. What the selection rule says if two answers got the same number of agrees, then the answer that was posted later will be selected. (Not the one with the last peer-grade - perhaps this is what you thought.)
In your example, I assume answer No.1 was posted first, and answer No.2 was posted some time later. They collected the same number of agrees, and No. 2 was selected, as that was the answer posted later.
This is perfectly in line with the selection rule.

The rule is logical. Think about it: all those people that peer-graded the second answer definitely had a chance to look at the first answer, and they chose to agree with the second one. Another way of looking at it (and this is what written in the FAQ) is that the time elapsed since posting the first answer until the time of grading is longer than the time elapsed since posting the second answer until the time of grading. Therefore, the second answer collected the same amount of agrees in a shorter period of time.

I hope this helps understanding what is going on.
Katalin


Direct link Reply with quote
 
Sandra Petch
Local time: 09:18
French to English
+ ...
TOPIC STARTER
Thanks Katalin Jul 13, 2009

Yes, that makes it clear and it does make sense.

On the other hand, as the second answer also got a neutral (a strong neutral, more a disagree!), surely that should be taken into account?

I've started a second thread about users who systematically leave their questions to be automatically graded and it seems I'm not the only one to find this disagreeable.

Thanks again for your help.


Direct link Reply with quote
 

Enrique
Local time: 04:18
SITE STAFF
Neutral has no effect when computing net agrees Jul 13, 2009

Sandra Petch wrote:

On the other hand, as the second answer also got a neutral (a strong neutral, more a disagree!), surely that should be taken into account?



A neutral is just neutral, it is not computed when evaluating the net agrees of an answer (agrees minus disagrees). If peer graders desire to leave a negative mark then they should use disagree instead of neutral.

Regards,
Enrique


Direct link Reply with quote
 


To report site rules violations or get help, contact a site moderator:


You can also contact site staff by submitting a support request »

How does automatic grading work?

Advanced search






Wordfast Pro
Translation Memory Software for Any Platform

Exclusive discount for ProZ.com users! Save over 13% when purchasing Wordfast Pro through ProZ.com. Wordfast is the world's #1 provider of platform-independent Translation Memory software. Consistently ranked the most user-friendly and highest value

More info »
PDF Translation - the Easy Way
TransPDF converts your PDFs to XLIFF ready for professional translation.

TransPDF converts your PDFs to XLIFF ready for professional translation. It also puts your translations back into the PDF to make new PDFs. Quicker and more accurate than hand-editing PDF. Includes free use of Infix PDF Editor with your translated PDFs.

More info »



Forums
  • All of ProZ.com
  • Term search
  • Jobs
  • Forums
  • Multiple search