Paper published in a journal (Scientific congresses, symposiums and conference proceedings)
Evolving a Deep Neural Network Training Time Estimator
Pinel, Frédéric; Yin, Jian-xiong; Hundt, Christian et al.
2020In Communications in Computer and Information Science
Peer reviewed
 

Files


Full Text
Pinel2020_Chapter_EvolvingADeepNeuralNetworkTrai.pdf
Publisher postprint (309.06 kB)
Download

All documents in ORBilu are protected by a user license.

Send to



Details



Abstract :
[en] We present a procedure for the design of a Deep Neural Net- work (DNN) that estimates the execution time for training a deep neural network per batch on GPU accelerators. The estimator is destined to be embedded in the scheduler of a shared GPU infrastructure, capable of providing estimated training times for a wide range of network architectures, when the user submits a training job. To this end, a very short and simple representation for a given DNN is chosen. In order to compensate for the limited degree of description of the basic network representation, a novel co-evolutionary approach is taken to fit the estimator. The training set for the estimator, i.e. DNNs, is evolved by an evolutionary algorithm that optimizes the accuracy of the estimator. In the process, the genetic algorithm evolves DNNs, generates Python-Keras programs and projects them onto the simple representation. The genetic operators are dynamic, they change with the estimator’s accuracy in order to balance accuracy with generalization. Results show that despite the low degree of information in the representation and the simple initial design for the predictor, co-evolving the training set performs better than near random generated population of DNNs.
Disciplines :
Computer science
Author, co-author :
Pinel, Frédéric ;  University of Luxembourg > Faculty of Science, Technology and Communication (FSTC) > Computer Science and Communications Research Unit (CSC)
Yin, Jian-xiong
Hundt, Christian ;  University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT)
Kieffer, Emmanuel ;  University of Luxembourg > Faculty of Science, Technology and Communication (FSTC) > Computer Science and Communications Research Unit (CSC)
Varrette, Sébastien ;  University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > Computer Science and Communications Research Unit (CSC)
Bouvry, Pascal ;  University of Luxembourg > Faculty of Science, Technology and Communication (FSTC) > Computer Science and Communications Research Unit (CSC)
See, Simon
External co-authors :
yes
Language :
English
Title :
Evolving a Deep Neural Network Training Time Estimator
Publication date :
February 2020
Event name :
Third International Conference on Optimization and Learning (OLA)
Event organizer :
University of Cadiz
Event place :
Cadiz, Spain
Event date :
from 17-02-2020 to 19-02-2020
Audience :
International
Journal title :
Communications in Computer and Information Science
ISSN :
1865-0929
Publisher :
Springer, Berlin, Germany
Peer reviewed :
Peer reviewed
Focus Area :
Computational Sciences
Available on ORBilu :
since 27 March 2020

Statistics


Number of views
183 (21 by Unilu)
Number of downloads
432 (16 by Unilu)

Scopus citations®
 
1
Scopus citations®
without self-citations
0
OpenCitations
 
1
WoS citations
 
0

Bibliography


Similar publications



Contact ORBilu