OATAO - Open Archive Toulouse Archive Ouverte Open Access Week

Why do you Think this Query is Difficult? A User Study on Human Query Prediction

Mizzaro, Stefano and Mothe, Josiane Why do you Think this Query is Difficult? A User Study on Human Query Prediction. (2016) In: 39th International ACM SIGIR conference on research and development in Information Retrieval (SIGIR 2016), 17 July 2016 - 21 July 2016 (Pisa, Italy).

(Document in English)

PDF (Author's version) - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader

Official URL: http://dx.doi.org/10.1145/2911451.2914696


Predicting if a query will be difficult for a system is important to improve retrieval effectiveness by implementing specific processing. There have been several attempts to predict difficulty, both automatically and manually; but without high accuracy at a preretrieval stage. In this paper, we focus rather on understanding why a query is perceived by humans as difficult. We ran two separated but related experiments in which we asked humans to provide both a query difficulty prediction and reasons to explain their prediction. Results show that: (i) reasons can be categorized into 4 classes; (ii) reasons can be framed into closed questions to be answered on a Likert scale; and (iii) some reasons correlate in a coherent way with the human predicted numerical difficulty. On the basis of these results it is possible to derive hints to be provided to help users when formulating their queries and to avoid them to rely on their wrong perception of difficulty.

Item Type:Conference or Workshop Item (Paper)
Additional Information:Thanks to ACM editor. The definitive version is available at http://dl.acm.org This papers appears in SIGIR '16 : Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval. ISBN : 978-1-4503-4069-4 The original PDF is available at: http://dl.acm.org/citation.cfm?id=2914696&CFID=760673810&CFTOKEN=63426868
HAL Id:hal-04109565
Audience (conference):International conference proceedings
Uncontrolled Keywords:
Institution:French research institutions > Centre National de la Recherche Scientifique - CNRS (FRANCE)
Université de Toulouse > Institut National Polytechnique de Toulouse - Toulouse INP (FRANCE)
Université de Toulouse > Université Toulouse III - Paul Sabatier - UT3 (FRANCE)
Université de Toulouse > Université Toulouse - Jean Jaurès - UT2J (FRANCE)
Université de Toulouse > Université Toulouse 1 Capitole - UT1 (FRANCE)
Other partners > Università degli studi di Udine - UNIUD (ITALY)
Laboratory name:
Deposited On:10 May 2017 07:39

Repository Staff Only: item control page