Paper published in a book (Scientific congresses, symposiums and conference proceedings)
Convergence time analysis of Asynchronous Distributed Artificial Neural Networks
DALLE LUCCA TOSI, Mauro; Ellampallil Venugopal, Vinu; THEOBALD, Martin
2022In 5th Joint International Conference on Data Science Management of Data (9th ACM IKDD CODS and 27th COMAD)
Peer reviewed
 

Files


Full Text
2022-CODS_COMAD.pdf
Publisher postprint (721.17 kB)
Request a copy

All documents in ORBilu are protected by a user license.

Send to



Details



Abstract :
[en] Artificial Neural Networks (ANNs) have drawn academy and industry attention for their ability to represent and solve complex problems. Researchers are studying how to distribute their computation to reduce their training time. However, the most common approaches in this direction are synchronous, letting computational resources sub-utilized. Asynchronous training does not have this drawback but is impacted by staled gradient updates, which have not been extended researched yet. Considering this, we experimentally investigate how stale gradients affect the convergence time and loss value of an ANN. In particular, we analyze an asynchronous distributed implementation of a Word2Vec model, in which the impact of staleness is negligible and can be ignored considering the computational speedup we achieve by allowing the staleness.
Disciplines :
Computer science
Author, co-author :
DALLE LUCCA TOSI, Mauro  ;  University of Luxembourg > Faculty of Science, Technology and Medicine (FSTM) > Department of Computer Science (DCS)
Ellampallil Venugopal, Vinu
THEOBALD, Martin ;  University of Luxembourg > Faculty of Science, Technology and Medicine (FSTM) > Department of Computer Science (DCS)
External co-authors :
no
Language :
English
Title :
Convergence time analysis of Asynchronous Distributed Artificial Neural Networks
Publication date :
2022
Event name :
CODS-COMAD 2022: 5th Joint International Conference on Data Science & Management of Data (9th ACM IKDD CODS and 27th COMAD)
Event date :
from 07-01-2022 to 10-01-2022
Audience :
International
Main work title :
5th Joint International Conference on Data Science Management of Data (9th ACM IKDD CODS and 27th COMAD)
Pages :
314--315
Peer reviewed :
Peer reviewed
FnR Project :
FNR12252781 - Data-driven Computational Modelling And Applications, 2017 (01/09/2018-28/02/2025) - Andreas Zilian
Available on ORBilu :
since 06 September 2022

Statistics


Number of views
140 (18 by Unilu)
Number of downloads
0 (0 by Unilu)

Bibliography


Similar publications



Contact ORBilu