OATAO - Open Archive Toulouse Archive Ouverte Open Access Week

Informativeness for Adhoc IR evaluation: a measure that prevents assessing individual documents

Deveaud, Romain and Moriceau, Véronique and Mothe, Josiane and San Juan, Eric Informativeness for Adhoc IR evaluation: a measure that prevents assessing individual documents. (2016) In: 38th European Conference on Information Retrieval (ECIR 2016), 20 March 2016 - 23 March 2016 (Padoue, Italy).

(Document in English)

PDF (Author's version) - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader

Official URL: http://dx.doi.org/10.1007/978-3-319-30671-1_73


Informativeness measures have been used in interactive in- formation retrieval and automatic summarization evaluation. Indeed, as opposed to adhoc retrieval, these two tasks cannot rely on the Cranfield evaluation paradigm in which retrieved documents are compared to static query relevance document lists. In this paper, we explore the use of informativeness measures to evaluate adhoc task. The advantage of the proposed evaluation framework is that it does not rely on an exhaustive reference and can be used in a changing environment in which new documents occur, and for which relevance has not been assessed. We show that the correlation between the official system ranking and the informativeness measure is specifically high for most of the TREC adhoc tracks.

Item Type:Conference or Workshop Item (Paper)
Additional Information:Thanks to Springer editor. This papers appears in Volume 9626 of Lecture Notes in Computer Science ISSN : 0302-9743 The original PDF is available at: https://link.springer.com/chapter/10.1007/978-3-319-30671-1_73
HAL Id:hal-01567065
Audience (conference):International conference proceedings
Uncontrolled Keywords:
Institution:French research institutions > Centre National de la Recherche Scientifique - CNRS (FRANCE)
Université de Toulouse > Institut National Polytechnique de Toulouse - Toulouse INP (FRANCE)
Université de Toulouse > Université Toulouse III - Paul Sabatier - UT3 (FRANCE)
Université de Toulouse > Université Toulouse - Jean Jaurès - UT2J (FRANCE)
Université de Toulouse > Université Toulouse 1 Capitole - UT1 (FRANCE)
Other partners > Université d'Avignon et des Pays de Vaucluse (FRANCE)
Other partners > Université Paris-Sud 11 (FRANCE)
Laboratory name:
Deposited On:27 Jun 2017 09:54

Repository Staff Only: item control page