There is a strong compatibility argument
for researchers to use the same methods as each
other unless there is very good reason to depart
from the norm.
Stephen Robertson, "Evaluation in information retrieval". European Summer School on Information Retrieval (ESSIR), Lecture Notes in Computer Science 1980, Springer, pages 81-92, 2000. |
Name Last modified Size Description
Parent Directory -
README.html 15-Nov-2005 09:34 1.3K
cityuni/ 10-Jul-2006 14:41 -
cmu/ 10-Jul-2006 14:41 -
ibmhaifa/ 10-Jul-2006 14:42 -
irit/ 10-Jul-2006 14:42 -
lip6/ 10-Jul-2006 14:42 -
maxplanck/ 10-Jul-2006 14:42 -
ouc/ 10-Jul-2006 14:42 -
qutau/ 10-Jul-2006 14:43 -
rmit/ 10-Jul-2006 14:43 -
uamsterdam/ 10-Jul-2006 14:43 -
uberkeley/ 10-Jul-2006 14:44 -
ucalif/ 10-Jul-2006 14:44 -
uhebrew/ 10-Jul-2006 14:44 -
uhelsinki/ 10-Jul-2006 14:44 -
ukyung/ 10-Jul-2006 14:45 -
uminnesota/ 10-Jul-2006 14:45 -
umontes/ 10-Jul-2006 14:45 -
uniKaislau/ 10-Jul-2006 14:46 -
uniisys/ 10-Jul-2006 14:45 -
utampere/ 10-Jul-2006 14:46 -
utrecht/ 10-Jul-2006 14:46 -
utwente/ 10-Jul-2006 14:46 -
uwaterloo/ 10-Jul-2006 14:46 -
valoria/ 10-Jul-2006 14:46 -
xtra-utrecht/ 10-Jul-2006 14:46 -
xtra-utwente/ 10-Jul-2006 14:47 -
xtra-utwente2004-2/ 10-Jul-2006 14:47 -
xtra-utwente2004/ 10-Jul-2006 14:47 -
This directory contains alternative evaluation results for the INEX 2005 adhoc runs. Many thanks to the INEX organisation for making runs and assessments available at an early stage. The files contain precision and overlap at several document cut-off values. The files [x].full.txt contain measurements for every single topic including a summary of the results (preceeded with '|') reporting the average precision and average overlap (averaged over the topics). The files [x].summary.txt only contain the precision and overlap averages. The files report precision for 5 different quantisation functions. All quantisations are binary (either relevant or not). The quantisation functions are: 0) very strict (which corresponds to the INEX 2005 official strict quantisiation), 1) strict, 2) liberal, 3) exhaustiveness oriented, and 4) specificity oriented. For more information, read the publication below. Djoerd Hiemstra and Vojkan Mihajlovic, ``The Simplest Evaluation Measures for XML Information Retrieval that Could Possibly Work'', In Proceedings of the INEX 2005 Workshop on Element Retrieval Methodology, University of Otago, New Zealand, July 30, ISBN 0-473-10071-1, pages 6-13, 2005 http://www.cs.utwente.nl/~hiemstra/papers/inexmw.pdf