Thursday, December 4, 2014

Participate in the Fact Ranking Challenge

Dear students!
We have developed a (fun) quiz which will allow us to tackle one of our main scientific research problems and we would really appreciate your participation and help in our experiment!

The problem is that, as we all know, there is too much information available on the web. How do we decide which facts are important or most relevant for a specific object?

Many fact ranking systems have already been developed, trying to solve this issue. To determine the relevance of a fact is a basic problem if you want to develop an intelligent recommender system that considers content-based relationships. However, the difficulty is that there does not exist any gold standard corpus which could serve as a ground truth for evaluating their performance.

How do you determine, which information or fact is more important or relevant than another? Just a quick example. Let's assume we have the following two facts:
(1) Albert Einstein is a physicist.
(2) Albert Einstein is a vegetarian.

Which of the two facts is more important or relevant? This is difficult to answer, simply because the truth often lies in the eye of the beholder. For a vegetarian, maybe the second fact is more important. But, what about the most common opinion? What would the mainstream think? Probably, most people would say that fact (1) in general is more important.

Our quiz can be found here. The registration takes just 1 minute and after that the task will be explained to you in detail.

We considered about 500 popular concepts from Wikipedia and you have 
(1) to think about the most important facts about these concepts that come to your mind and then 
(2) to rate the (new) facts presented to you according to their relevance. 
There is no right or wrong answer. Just vote as you think it seems right for you. Afterwards, we will aggregate all votes from all participants to determine a general (mainstream) relevance for the presented facts.

You may interrupt your rating of the presented facts at any time you like and continue later. As you go, you will get score points which depend on how well you've answered the questions and how many concepts you've covered. 

To make it more challenging, we have reserved a prize for the student that gets most points by Christmas time! The student who collects the most points will receive a copy of the book "Logicomix: An epic search for truth" in our last lecture before Christmas on Dec. 16, 2014. Logicomix is an engaging and exciting graphic novel that will walk you through the extraordinary world of logics in a witty way.
(Please register with your full name, or send us your chosen username by email, so that we can identify the most successful players.)

The task is already running for quite a while. So please don't get scared by existing highscores. The competition and the prize is only reserved to our students. 

We would really appreciate your help in this task! Please do also spread the word and share the task with your friends and family. The more participants, the more valid our ground truth will be. While you are playing this game, your inputs will contribute to our scientific experiment. Together, we will build a first corpus that will help our scientific community in evaluating different fact ranking strategies!

For more information, see: