Label-consistent sparse auto-encoders

Abstract

Auto-encoders (AE) is a particular type of unsupervised neural networks that aim at providing a compact representation of a signal or an image [1]. Such AEs are useful for data compression but most of the time the representations they provide are not appropriate as is for a downstream classification task. This is due to the fact that they are trained to minimize a reconstruction error and not a classification loss. Classification attempts with AEs have already been proposed such as contractive AEs [2], correspondence AEs [3] and stacked similarity-aware AEs [4], for instance. Inspired by label-consistent K-SVD (LC-KSVD) [5], we propose a novel supervised version of AEs that integrates class information within the encoded representations.