Off topic: Help with academic study on question answering
Thread poster: Sandra& Kenneth

Sandra& Kenneth  Identity Verified
Israel
Local time: 16:42
Member (2009)
French to English
+ ...
Sep 2, 2015

As part of his PhD, my son-in-law is trying to enable machines to better answer questions in a way that is easy for people to understand.

The intention is to get as many responses as possible to understand how people from diverse backgrounds understand the English language. It's OK to do the survey even if you're not a native English speaker. Also, the survey is completely anonymous. ​

Help us teach computers to do a better job answering questions, and hopefully, you may enjoy the results in your search engine in the future!

Here is the link to the survey:

http://irsrv2.cs.biu.ac.il:9992/

Your participation is much appreciated!

Sandra


 

564354352 (X)  Identity Verified
Denmark
Local time: 15:42
Danish to English
+ ...
Not so anonymous Sep 2, 2015

The first thing you are asked to do is to enter your name... Put me off straight away... sorry...

 

Sandra& Kenneth  Identity Verified
Israel
Local time: 16:42
Member (2009)
French to English
+ ...
TOPIC STARTER
This is just an optional field Sep 2, 2015

... which is pre-filled.

You can use a pseudonym or leave it as is. Most people just stay with the default name.

icon_smile.gif


 

Tom in London
United Kingdom
Local time: 14:42
Member (2008)
Italian to English
Why? Sep 2, 2015

Sandra& Kenneth wrote:

As part of his PhD, my son-in-law is trying to enable machines to better answer questions in a way that is easy for people to understand.



Why?


 

Phil Hand  Identity Verified
China
Local time: 22:42
Chinese to English
Too hard Sep 2, 2015

I tried, but there are too many questions, and the questionnaire is confusing.

For example, it offers both singular and plural verb forms. But some nouns are inherently plural, so I can't mark one or the other as grammatically incorrect. Also, there are several items on which I could go either way.

I would suggest that the questionnaire is redesigned, with real words instead of the dummy variables in angle brackets, and perhaps ask people to select the best answer out of 3 or 4, rather than this brain-melting list.


 

Amel Abdullah  Identity Verified
Jordan
Arabic to English
+ ...
I gave up Sep 2, 2015

I wanted to help out by responding, but I found the survey too confusing and finally gave up. I agree with the suggestions offered by Phil.

 

Maxi Schwarz
Local time: 08:42
German to English
+ ...
filled it, but Sep 2, 2015

Sandra& Kenneth wrote:

As part of his PhD, my son-in-law is trying to enable machines to better answer questions in a way that is easy for people to understand.

It might have to start with creating questions that people are able to understand, before worrying about machines being able to answer them. These questions were seriously messed up.


 

omerlevy
Israel
Thanks Sep 2, 2015

Hi everyone, thank you so much for helping with my research!

Phil, thanks for the valuable feedback; I will incorporate some of your suggestions in the next round of annotations.

Perhaps I should give some broader context. The questionnaire isolates a specific problem in automatic question answering - answer verification. In this subtask, the machine is given a well-formed question and an automatically-retrieved text fragment containing a candidate answer. The machine must then determine whether the text fragment indeed contains an answer to the question. This task is extremely difficult for machines, since many of these decisions are based on context. For example, in "Which drug cures infections?", the fragment "<DRUG> eradicates infections" will probably contain a relevant answer, because "eradicates" implies "cures" when the object is "infections". However, this is most certainly not the case in "Which person cures sick patients?".

The questionnaire is challenging in two aspects. First, because roughly 30% of the candidate answer fragments are ungrammatical or nonsensical. This is an artifact of the automatic fragment retrieval process, which relies on other error-prone automatic processes such as syntactic parsing. The other reason is that the answer-verification task itself is semantically gray; different people tend to interpret language in different manners, resulting in a mix of different annotations for each question. This is exactly why it is so important for me to get a diverse sample of annotators, and why your participation is so helpful to my cause.

Finally, I'd like to address Phil's point on <PLACEHOLDERS>. Placeholders are a necessary evil, because in the first round of annotations, where we had the actual answer in view, annotators used their own world knowledge to decide whether the fragment contains an answer. While populating the machine with world knowledge is a valid approach for automatic question answering, this research project is focused on teaching it whether the context is indicative of a correct answer, regardless of the answer itself.

Again, thank you so much for participating in my questionnaire and helping my research!


[Edited at 2015-09-03 05:46 GMT]


 

Sandra& Kenneth  Identity Verified
Israel
Local time: 16:42
Member (2009)
French to English
+ ...
TOPIC STARTER
Thanks everyone for commenting and participating! Sep 3, 2015

Your comments and participation are much appreciated.
The young scientist's reply is still being vetted by moderators, and hopefully will be released soon.

Thanks!

Sandra


 

Sandra& Kenneth  Identity Verified
Israel
Local time: 16:42
Member (2009)
French to English
+ ...
TOPIC STARTER
To change the form... Sep 4, 2015

If you don't feel comfortable with the questions, you can regenerate the form any number of times to a different set by refreshing the page!

Thanks everyone!

Sandra


 


To report site rules violations or get help, contact a site moderator:

Moderator(s) of this forum
Fernanda Rocha[Call to this topic]

You can also contact site staff by submitting a support request »

Help with academic study on question answering

Advanced search






memoQ translator pro
Kilgray's memoQ is the world's fastest developing integrated localization & translation environment rendering you more productive and efficient.

With our advanced file filters, unlimited language and advanced file support, memoQ translator pro has been designed for translators and reviewers who work on their own, with other translators or in team-based translation projects.

More info »
CafeTran Espresso
You've never met a CAT tool this clever!

Translate faster & easier, using a sophisticated CAT tool built by a translator / developer. Accept jobs from clients who use SDL Trados, MemoQ, Wordfast & major CAT tools. Download and start using CafeTran Espresso -- for free

More info »



Forums
  • All of ProZ.com
  • Term search
  • Jobs
  • Forums
  • Multiple search