Impact of Disentanglement on Pruning Neural Networks

Deploying deep learning neural networks on edge devices,
to accomplish task specific objectives in the real-world, requires a
reduction in their memory footprint, power consumption, and latency.
This can be realized via efficient model compression. Disentangled latent
representations produced by variational autoencoder (VAE) networks are
a promising approach for achieving model compression because they
mainly retain task-specific information, discarding useless information
for the task at hand. We make use of the Beta-VAE framework combined
with a standard criterion for pruning to investigate the impact of forcing
the network to learn disentangled representations on the pruning process
for the task of classification. In particular, we perform experiments on
MNIST and CIFAR10 datasets, examine disentanglement challenges, and
propose a path forward for future works.