Hubert, Gilles
and Moreno, José G. and Pinel-Sauvagnat, Karen and Pitarch, Yoann
Some thoughts from IRIT about the scenario A of the TREC RTS 2016 and 2017 tracks.
(2017)
In: 26th Text REtrieval Conference (TREC 2017), 15 November 2017 - 17 November 2017 (Gaithersburg, Maryland, United States).
|
(Document in English)
PDF (Author's version) - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader 368kB |
Official URL: https://trec.nist.gov/pubs/trec26/papers/IRIT2-RT.pdf
Abstract
The TREC Real-Time Summarization (RTS) track provides a framework for evaluating systems monitoring the Twitter stream and pushing tweets to users according to given profiles. It includes metrics, files, settings and hypothesis provided by the organizers. In this work, we perform a thorough analysis of each component of the framework used for batch evaluation of scenario A in 2016 and 2017. We found some weaknesses of the metrics and took advantage of these limitations to submit our official run in the 2017 edition. The good evaluation results validate our findings. This paper also gives clear recommendations to fairly reuse the collection.
| Item Type: | Conference or Workshop Item (Paper) |
|---|---|
| Additional Information: | NIST: National Institute of Standards and Technology |
| HAL Id: | hal-02319737 |
| Audience (conference): | International conference proceedings |
| Uncontrolled Keywords: | |
| Institution: | Université de Toulouse > Institut National Polytechnique de Toulouse - Toulouse INP (FRANCE) French research institutions > Centre National de la Recherche Scientifique - CNRS (FRANCE) Université de Toulouse > Université Toulouse III - Paul Sabatier - UT3 (FRANCE) Université de Toulouse > Université Toulouse - Jean Jaurès - UT2J (FRANCE) Université de Toulouse > Université Toulouse 1 Capitole - UT1 (FRANCE) |
| Laboratory name: | |
| Statistics: | download |
| Deposited On: | 04 Oct 2019 13:50 |
Repository Staff Only: item control page
