Revisiting the Training of Very Deep Neural Networks without Skip Connections

Poster (2021)

Home-based rehabilitation system for strokesurvivors: a clinical evaluation

in Journal of medical systems (2020)

Improved Highway Network Block for Training Very Deep Neural Networks

in IEEE Access (2020)

GOING DEEPER WITH NEURAL NETWORKS WITHOUT SKIP CONNECTIONS

in IEEE International Conference on Image Processing (ICIP 2020), Abu Dhabi, UAE, Oct 25–28, 2020 (2020, May 30)

Towards Automatic CAD Modeling from 3D Scan Sketch based Representation

in Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2020), GRAPP (2020, February)

An adaptive framework for real-time data reduction in AMI

in Journal of King Saud University - Computer and Information Sciences (2019), 31(3), 392-402

In existing Advanced Metering Infrastructure (AMI), data collection intervals for each smart meter (SM) typically vary from 15 to 60 min. If we have 1 million SMs that transmit data every 15 min, these ... [more ▼]

In existing Advanced Metering Infrastructure (AMI), data collection intervals for each smart meter (SM) typically vary from 15 to 60 min. If we have 1 million SMs that transmit data every 15 min, these SMs will export 4 million records per hour. This leads to dramatically increasing bandwidth usage, energy consumption, traffic cost and I/O congestion. In this work, we present an adaptive framework for minimizing the amount of data transfer from SMs. The reduction in the framework is forecasting-based; when an SM reading is close to the forecasted value, the SM does not transmit the reading. In order for the framework to be adaptive to the ever-changing pattern of SM data, it is provided with a pool of forecasting methods. A supervised-learning scheme is employed to switch in real-time to the forecasting method most suitable to the current data pattern. The experimental results demonstrate that the proposed framework achieves data reduction rates up to 98% with accuracy 96%, depending on the operational parameters of the framework and consumer behavior (statistical features of SM data). [less ▲]

Highway Network Block with Gates Constraints for Training Very Deep Networks

in 2018 IEEE International Conference on Computer Vision and Pattern Recognition Workshop, June 18-22, 2018 (2018, June 19)

In this paper, we propose to reformulate the learning of the highway network block to realize both early optimization and improved generalization of very deep networks while preserving the network depth ... [more ▼]

In this paper, we propose to reformulate the learning of the highway network block to realize both early optimization and improved generalization of very deep networks while preserving the network depth. Gate constraints are duly employed to improve optimization, latent representations and parameterization usage in order to efficiently learn hierarchical feature transformations which are crucial for the success of any deep network. One of the earliest very deep models with over 30 layers that was successfully trained relied on highway network blocks. Although, highway blocks suffice for alleviating optimization problem via improved information flow, we show for the first time that further in training such highway blocks may result into learning mostly untransformed features and therefore a reduction in the effective depth of the model; this could negatively impact model generalization performance. Using the proposed approach, 15-layer and 20-layer models are successfully trained with one gate and a 32-layer model using three gates. This leads to a drastic reduction of model parameters as compared to the original highway network. Extensive experiments on CIFAR-10, CIFAR-100, Fashion-MNIST and USPS datasets are performed to validate the effectiveness of the proposed approach. Particularly, we outperform the original highway network and many state-ofthe- art results. To the best our knowledge, on the Fashion-MNIST and USPS datasets, the achieved results are the best reported in literature. [less ▲]

IMPROVING THE CAPACITY OF VERY DEEP NETWORKS WITH MAXOUT UNITS

in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (2018, February 21)

Deep neural networks inherently have large representational power for approximating complex target functions. However, models based on rectified linear units can suffer reduction in representation ... [more ▼]

Deep neural networks inherently have large representational power for approximating complex target functions. However, models based on rectified linear units can suffer reduction in representation capacity due to dead units. Moreover, approximating very deep networks trained with dropout at test time can be more inexact due to the several layers of non-linearities. To address the aforementioned problems, we propose to learn the activation functions of hidden units for very deep networks via maxout. However, maxout units increase the model parameters, and therefore model may suffer from overfitting; we alleviate this problem by employing elastic net regularization. In this paper, we propose very deep networks with maxout units and elastic net regularization and show that the features learned are quite linearly separable. We perform extensive experiments and reach state-of-the-art results on the USPS and MNIST datasets. Particularly, we reach an error rate of 2.19% on the USPS dataset, surpassing the human performance error rate of 2.5% and all previously reported results, including those that employed training data augmentation. On the MNIST dataset, we reach an error rate of 0.36% which is competitive with the state-of-the-art results. [less ▲]

Towards Automatic Human Body Model Fitting to a 3D Scan

in D'APUZZO, Nicola (Ed.) Proceedings of 3DBODY.TECH 2017 - 8th International Conference and Exhibition on 3D Body Scanning and Processing Technologies, Montreal QC, Canada, 11-12 Oct. 2017 (2017, October)

This paper presents a method to automatically recover a realistic and accurate body shape of a person wearing clothing from a 3D scan. Indeed, in many practical situations, people are scanned wearing ... [more ▼]

This paper presents a method to automatically recover a realistic and accurate body shape of a person wearing clothing from a 3D scan. Indeed, in many practical situations, people are scanned wearing clothing. The underlying body shape is thus partially or completely occluded. Yet, it is very desirable to recover the shape of a covered body as it provides non-invasive means of measuring and analysing it. This is particularly convenient for patients in medical applications, customers in a retail shop, as well as in security applications where suspicious objects under clothing are to be detected. To recover the body shape from the 3D scan of a person in any pose, a human body model is usually fitted to the scan. Current methods rely on the manual placement of markers on the body to identify anatomical locations and guide the pose fitting. The markers are either physically placed on the body before scanning or placed in software as a postprocessing step. Some other methods detect key points on the scan using 3D feature descriptors to automate the placement of markers. They usually require a large database of 3D scans. We propose to automatically estimate the body pose of a person from a 3D mesh acquired by standard 3D body scanners, with or without texture. To fit a human model to the scan, we use joint locations as anchors. These are detected from multiple 2D views using a conventional body joint detector working on images. In contrast to existing approaches, the proposed method is fully automatic, and takes advantage of the robustness of state-of-art 2D joint detectors. The proposed approach is validated on scans of people in different poses wearing garments of various thicknesses and on scans of one person in multiple poses with known ground truth wearing close-fitting clothing. [less ▲]

Training Very Deep Networks via Residual Learning with Stochastic Input Shortcut Connections

in 24th International Conference on Neural Information Processing, Guangzhou, China, November 14–18, 2017 (2017, July 31)

Many works have posited the benefit of depth in deep networks. However, one of the problems encountered in the training of very deep networks is feature reuse; that is, features are ’diluted’ as they are ... [more ▼]

Many works have posited the benefit of depth in deep networks. However, one of the problems encountered in the training of very deep networks is feature reuse; that is, features are ’diluted’ as they are forward propagated through the model. Hence, later network layers receive less informative signals about the input data, consequently making training less effective. In this work, we address the problem of feature reuse by taking inspiration from an earlier work which employed residual learning for alleviating the problem of feature reuse. We propose a modification of residual learning for training very deep networks to realize improved generalization performance; for this, we allow stochastic shortcut connections of identity mappings from the input to hidden layers.We perform extensive experiments using the USPS and MNIST datasets. On the USPS dataset, we achieve an error rate of 2.69% without employing any form of data augmentation (or manipulation). On the MNIST dataset, we reach a comparable state-of-the-art error rate of 0.52%. Particularly, these results are achieved without employing any explicit regularization technique. [less ▲]