OATAO - Open Archive Toulouse Archive Ouverte Open Access Week

Studying the Variability of System Setting Effectiveness by Data Analytics and Visualization

Déjean, Sebastien and Mothe, Josiane and Ullah, Md Zia Studying the Variability of System Setting Effectiveness by Data Analytics and Visualization. (2019) In: Conference and Labs of the Evaluation Forum, Living Labs (CLEF 2019), 9 September 2019 - 12 September 2019 (Lugano, Switzerland).

(Document in English)

PDF (Author's version) - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader

Official URL: https://doi.org/10.1007/978-3-030-28577-7_3


Search engines differ from their modules and parameters; defining the optimal system setting is challenging the more because of the complexity of a retrieval stream. The main goal of this study is to determine which are the most important system components and parameters in system setting, thus which ones should be tuned as the first priority. We carry out an extensive analysis of 20, 000 different system settings applied to three TREC ad-hoc collections. Our analysis includes zooming in and out the data using various data analysis methods such as ANOVA, CART, and data visualization. We found that the query expansion model is the most significant component that changes the system effectiveness, consistently across collections. Zooming in the queries, we show that the most significant component changes to the retrieval model when considering easy queries only. The results of our study are directly re-usable for the system designers and for system tuning.

Item Type:Conference or Workshop Item (Paper)
Additional Information:Thanks to Springer editor. This papers appears in volume 11696 of Lecture Notes in Computer Science ISSN : 0302-9743 ISBN: 978-3-030-28576-0 The original PDF is available at: https://link.springer.com/chapter/10.1007/978-3-030-28577-7_3
HAL Id:hal-02930098
Audience (conference):International conference proceedings
Uncontrolled Keywords:
Institution:French research institutions > Centre National de la Recherche Scientifique - CNRS (FRANCE)
Université de Toulouse > Institut National Polytechnique de Toulouse - Toulouse INP (FRANCE)
Université de Toulouse > Université Toulouse III - Paul Sabatier - UT3 (FRANCE)
Université de Toulouse > Université Toulouse - Jean Jaurès - UT2J (FRANCE)
Université de Toulouse > Université Toulouse 1 Capitole - UT1 (FRANCE)
Other partners > Institut National des Sciences Appliquées - INSA (FRANCE)
Laboratory name:
Deposited On:10 Jul 2020 14:56

Repository Staff Only: item control page