Reference : IMPROVING THE CAPACITY OF VERY DEEP NETWORKS WITH MAXOUT UNITS
Scientific congresses, symposiums and conference proceedings : Paper published in a book
Engineering, computing & technology : Computer science
http://hdl.handle.net/10993/34968
IMPROVING THE CAPACITY OF VERY DEEP NETWORKS WITH MAXOUT UNITS
English
Oyedotun, Oyebade mailto [University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > >]
Shabayek, Abd El Rahman mailto [University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > >]
Aouada, Djamila mailto [University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > >]
Ottersten, Björn mailto [University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > >]
21-Feb-2018
2018 IEEE International Conference on Acoustics, Speech and Signal Processing
Yes
International
2018 IEEE International Conference on Acoustics, Speech and Signal Processing
15–20 April 2018
IEEE
Calgary, Alberta
Canada
[en] Image classification ; deep networks ; residual learning ; neural networks
[en] Deep neural networks inherently have large representational power for approximating complex target functions. However, models based on rectified linear units can suffer reduction in representation capacity due to dead units. Moreover, approximating very deep networks trained with dropout at test time can be more inexact due to the several layers of non-linearities. To address the aforementioned problems, we propose to learn the activation functions of hidden units for very deep networks via maxout. However, maxout units increase the model parameters, and therefore model may suffer from overfitting; we alleviate this problem by employing elastic net regularization. In this paper, we propose very deep networks with maxout units and elastic net regularization and show that the features learned are quite linearly separable. We perform extensive experiments and reach state-of-the-art results on the USPS and MNIST datasets. Particularly, we reach an error rate of 2.19% on the USPS dataset, surpassing the human performance error rate of 2.5% and all previously reported results, including those that employed training data augmentation. On the MNIST dataset, we reach an error rate of 0.36% which is competitive with the state-of-the-art results.
Interdisciplinary Centre for Security, Reliability and Trust (SnT) > SIGCOM
This work was funded by the National Research Fund (FNR), Luxembourg, under the project reference R-AGR-0424-05-D/Bjorn Ottersten
http://hdl.handle.net/10993/34968

File(s) associated to this reference

Fulltext file(s):

FileCommentaryVersionSizeAccess
Open access
Oyebade_ICASSP2018_CR_V01.pdfAuthor preprint607.84 kBView/Open

Bookmark and Share SFX Query

All documents in ORBilu are protected by a user license.