OATAO - Open Archive Toulouse Archive Ouverte Open Access Week

Mining Discourse Markers for Unsupervised Sentence Representation Learning

Sileo, Damien and Van de Cruys, Tim and Pradel, Camille and Muller, Philippe Mining Discourse Markers for Unsupervised Sentence Representation Learning. (2019) In: Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL 2019), 1 June 2019 - 5 June 2019 (Minneapolis, United States).

(Document in English)

PDF (Author's version) - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader

Official URL: https://doi.org/10.18653/v1/N19-1351


Current state of the art systems in NLP heavily rely on manually annotated datasets, which are expensive to construct. Very little work adequately exploits unannotated data – such as discourse markers between sentences – mainly because of data sparseness and ineffective extraction methods. In the present work, we propose a method to automatically discover sentence pairs with relevant discourse markers, and apply it to massive amounts of data. Our resulting dataset contains 174 discourse markers with at least 10k examples each, even for rare markers such as “coincidentally” or “amazingly”. We use the resulting data as supervision for learning transferable sentence embeddings. In addition, we show that even though sentence representation learning through prediction of discourse marker yields state of the art results across different transfer tasks, it’s not clear that our models made use of the semantic relation between sentences, thus leaving room for further improvements.

Item Type:Conference or Workshop Item (Paper)
Additional Information:ACL : Association for Computational Linguistics https://www.aclweb.org/anthology/N19-1351.pdf
HAL Id:hal-02397473
Audience (conference):International conference proceedings
Uncontrolled Keywords:
Institution:French research institutions > Centre National de la Recherche Scientifique - CNRS (FRANCE)
Université de Toulouse > Institut National Polytechnique de Toulouse - Toulouse INP (FRANCE)
Université de Toulouse > Université Toulouse III - Paul Sabatier - UT3 (FRANCE)
Université de Toulouse > Université Toulouse - Jean Jaurès - UT2J (FRANCE)
Université de Toulouse > Université Toulouse 1 Capitole - UT1 (FRANCE)
Other partners > Synapse Développement (FRANCE)
Laboratory name:
Deposited On:25 Nov 2019 13:22

Repository Staff Only: item control page