References of "Oyedotun, Oyebade 50025901"
     in
Bookmark and Share    
Full Text
Peer Reviewed
See detailHighway Network Block with Gates Constraints for Training Very Deep Networks
Oyedotun, Oyebade UL; Shabayek, Abd El Rahman UL; Aouada, Djamila UL et al

in 2018 IEEE International Conference on Computer Vision and Pattern Recognition Workshop, June 18-22, 2018 (2018, June 19)

In this paper, we propose to reformulate the learning of the highway network block to realize both early optimization and improved generalization of very deep networks while preserving the network depth ... [more ▼]

In this paper, we propose to reformulate the learning of the highway network block to realize both early optimization and improved generalization of very deep networks while preserving the network depth. Gate constraints are duly employed to improve optimization, latent representations and parameterization usage in order to efficiently learn hierarchical feature transformations which are crucial for the success of any deep network. One of the earliest very deep models with over 30 layers that was successfully trained relied on highway network blocks. Although, highway blocks suffice for alleviating optimization problem via improved information flow, we show for the first time that further in training such highway blocks may result into learning mostly untransformed features and therefore a reduction in the effective depth of the model; this could negatively impact model generalization performance. Using the proposed approach, 15-layer and 20-layer models are successfully trained with one gate and a 32-layer model using three gates. This leads to a drastic reduction of model parameters as compared to the original highway network. Extensive experiments on CIFAR-10, CIFAR-100, Fashion-MNIST and USPS datasets are performed to validate the effectiveness of the proposed approach. Particularly, we outperform the original highway network and many state-ofthe- art results. To the best our knowledge, on the Fashion-MNIST and USPS datasets, the achieved results are the best reported in literature. [less ▲]

Detailed reference viewed: 97 (8 UL)
Full Text
Peer Reviewed
See detailIMPROVING THE CAPACITY OF VERY DEEP NETWORKS WITH MAXOUT UNITS
Oyedotun, Oyebade UL; Shabayek, Abd El Rahman UL; Aouada, Djamila UL et al

in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (2018, February 21)

Deep neural networks inherently have large representational power for approximating complex target functions. However, models based on rectified linear units can suffer reduction in representation ... [more ▼]

Deep neural networks inherently have large representational power for approximating complex target functions. However, models based on rectified linear units can suffer reduction in representation capacity due to dead units. Moreover, approximating very deep networks trained with dropout at test time can be more inexact due to the several layers of non-linearities. To address the aforementioned problems, we propose to learn the activation functions of hidden units for very deep networks via maxout. However, maxout units increase the model parameters, and therefore model may suffer from overfitting; we alleviate this problem by employing elastic net regularization. In this paper, we propose very deep networks with maxout units and elastic net regularization and show that the features learned are quite linearly separable. We perform extensive experiments and reach state-of-the-art results on the USPS and MNIST datasets. Particularly, we reach an error rate of 2.19% on the USPS dataset, surpassing the human performance error rate of 2.5% and all previously reported results, including those that employed training data augmentation. On the MNIST dataset, we reach an error rate of 0.36% which is competitive with the state-of-the-art results. [less ▲]

Detailed reference viewed: 113 (19 UL)
Full Text
Peer Reviewed
See detailFacial Expression Recognition via Joint Deep Learning of RGB-Depth Map Latent Representations
Oyedotun, Oyebade UL; Demisse, Girum UL; Shabayek, Abd El Rahman UL et al

in 2017 IEEE International Conference on Computer Vision Workshop (ICCVW) (2017, August 21)

Humans use facial expressions successfully for conveying their emotional states. However, replicating such success in the human-computer interaction domain is an active research problem. In this paper, we ... [more ▼]

Humans use facial expressions successfully for conveying their emotional states. However, replicating such success in the human-computer interaction domain is an active research problem. In this paper, we propose deep convolutional neural network (DCNN) for joint learning of robust facial expression features from fused RGB and depth map latent representations. We posit that learning jointly from both modalities result in a more robust classifier for facial expression recognition (FER) as opposed to learning from either of the modalities independently. Particularly, we construct a learning pipeline that allows us to learn several hierarchical levels of feature representations and then perform the fusion of RGB and depth map latent representations for joint learning of facial expressions. Our experimental results on the BU-3DFE dataset validate the proposed fusion approach, as a model learned from the joint modalities outperforms models learned from either of the modalities. [less ▲]

Detailed reference viewed: 241 (47 UL)
Full Text
Peer Reviewed
See detailPrototype Incorporated Emotional Neural Network (PI-EmNN)
Oyedotun, Oyebade UL; Khashman, Adnan

in IEEE Transactions on Neural Networks and Learning Systems (2017)

Artificial neural networks (ANNs) aim to simulate the biological neural activities. Interestingly, many ‘engineering’ prospects in ANN have relied on motivations from cognition and psychology studies. So ... [more ▼]

Artificial neural networks (ANNs) aim to simulate the biological neural activities. Interestingly, many ‘engineering’ prospects in ANN have relied on motivations from cognition and psychology studies. So far, two important learning theories that have been subject of active research are the prototype and adaptive learning theories. The learning rules employed for ANNs can be related to adaptive learning theory, where several examples of the different classes in a task are supplied to the network for adjusting internal parameters. Conversely, prototype learning theory uses prototypes (representative examples); usually, one prototype per class of the different classes contained in the task. These prototypes are supplied for systematic matching with new examples so that class association can be achieved. In this paper, we propose and implement a novel neural network algorithm based on modifying the emotional neural network (EmNN) model to unify the prototype and adaptive learning theories. We refer to our new model as “PI-EmNN” (Prototype-Incorporated Emotional Neural Network). Furthermore, we apply the proposed model to two real-life challenging tasks, namely; static hand gesture recognition and face recognition, and compare the result to those obtained using the popular back propagation neural network (BPNN), emotional back propagation neural network (EmNN), deep networks and an exemplar classification model, k-nearest neighbor (k-NN). [less ▲]

Detailed reference viewed: 56 (15 UL)
Full Text
Peer Reviewed
See detailTraining Very Deep Networks via Residual Learning with Stochastic Input Shortcut Connections
Oyedotun, Oyebade UL; Shabayek, Abd El Rahman UL; Aouada, Djamila UL et al

in 24th International Conference on Neural Information Processing, Guangzhou, China, November 14–18, 2017 (2017, July 31)

Many works have posited the benefit of depth in deep networks. However, one of the problems encountered in the training of very deep networks is feature reuse; that is, features are ’diluted’ as they are ... [more ▼]

Many works have posited the benefit of depth in deep networks. However, one of the problems encountered in the training of very deep networks is feature reuse; that is, features are ’diluted’ as they are forward propagated through the model. Hence, later network layers receive less informative signals about the input data, consequently making training less effective. In this work, we address the problem of feature reuse by taking inspiration from an earlier work which employed residual learning for alleviating the problem of feature reuse. We propose a modification of residual learning for training very deep networks to realize improved generalization performance; for this, we allow stochastic shortcut connections of identity mappings from the input to hidden layers.We perform extensive experiments using the USPS and MNIST datasets. On the USPS dataset, we achieve an error rate of 2.69% without employing any form of data augmentation (or manipulation). On the MNIST dataset, we reach a comparable state-of-the-art error rate of 0.52%. Particularly, these results are achieved without employing any explicit regularization technique. [less ▲]

Detailed reference viewed: 144 (42 UL)