OATAO - Open Archive Toulouse Archive Ouverte Open Access Week

Composition of Embeddings : Lessons from Statistical Relational Learning

Sileo, Damien and Van de Cruys, Tim and Pradel, Camille and Muller, Philippe Composition of Embeddings : Lessons from Statistical Relational Learning. (2019) In: 8th Joint Conference on Lexical and Computational Semantics (SEM 2019), 6 June 2019 - 7 June 2019 (Minneapolis, United States).

(Document in English)

PDF (Author's version) - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader

Official URL: https://doi.org/10.18653/v1/s19-1004


Various NLP problems -- such as the prediction of sentence similarity, entailment, and discourse relations -- are all instances of the same general task: the modeling of semantic relations between a pair of textual elements. A popular model for such problems is to embed sentences into fixed size vectors, and use composition functions (e.g. concatenation or sum) of those vectors as features for the prediction. At the same time, composition of embeddings has been a main focus within the field of Statistical Relational Learning (SRL) whose goal is to predict relations between entities (typically from knowledge base triples). In this article, we show that previous work on relation prediction between texts implicitly uses compositions from baseline SRL models. We show that such compositions are not expressive enough for several tasks (e.g. natural language inference). We build on recent SRL models to address textual relational problems, showing that they are more expressive, and can alleviate issues from simpler compositions. The resulting models significantly improve the state of the art in both transferable sentence representation learning and relation prediction.

Item Type:Conference or Workshop Item (Paper)
Additional Information:ACL : Association for Computational Linguistics ISBN 978-1-948087-93-3 https://www.aclweb.org/anthology/S19-1004/
HAL Id:hal-02397476
Audience (conference):International conference proceedings
Uncontrolled Keywords:
Institution:French research institutions > Centre National de la Recherche Scientifique - CNRS (FRANCE)
Université de Toulouse > Institut National Polytechnique de Toulouse - Toulouse INP (FRANCE)
Université de Toulouse > Université Toulouse III - Paul Sabatier - UT3 (FRANCE)
Université de Toulouse > Université Toulouse - Jean Jaurès - UT2J (FRANCE)
Université de Toulouse > Université Toulouse 1 Capitole - UT1 (FRANCE)
Other partners > Synapse Développement (FRANCE)
Laboratory name:
Deposited On:25 Nov 2019 13:41

Repository Staff Only: item control page