Reference : Convergence time analysis of Asynchronous Distributed Artificial Neural Networks

	Document type :	Scientific congresses, symposiums and conference proceedings : Paper published in a book
	Discipline(s) :	Engineering, computing & technology : Computer science
	To cite this reference:	http://hdl.handle.net/10993/52020

Title :	Convergence time analysis of Asynchronous Distributed Artificial Neural Networks
Language :	English
Author, co-author :	Dalle Lucca Tosi, Mauro [University of Luxembourg > Faculty of Science, Technology and Medicine (FSTM) > Department of Computer Science (DCS)]
	Ellampallil Venugopal, Vinu [> >]
	Theobald, Martin [University of Luxembourg > Faculty of Science, Technology and Medicine (FSTM) > Department of Computer Science (DCS)]
Publication date :	2022
Main document title :	5th Joint International Conference on Data Science Management of Data (9th ACM IKDD CODS and 27th COMAD)
Pages :	314--315
Peer reviewed :	Yes
Audience :	International
Event name :	CODS-COMAD 2022: 5th Joint International Conference on Data Science & Management of Data (9th ACM IKDD CODS and 27th COMAD)
Event date :	from 07-01-2022 to 10-01-2022
Abstract :	[en] Artificial Neural Networks (ANNs) have drawn academy and industry attention for their ability to represent and solve complex problems. Researchers are studying how to distribute their computation to reduce their training time. However, the most common approaches in this direction are synchronous, letting computational resources sub-utilized. Asynchronous training does not have this drawback but is impacted by staled gradient updates, which have not been extended researched yet. Considering this, we experimentally investigate how stale gradients affect the convergence time and loss value of an ANN. In particular, we analyze an asynchronous distributed implementation of a Word2Vec model, in which the impact of staleness is negligible and can be ignored considering the computational speedup we achieve by allowing the staleness.
Permalink :	http://hdl.handle.net/10993/52020
FnR project :	FnR ; FNR12252781 > Andreas Zilian > DRIVEN > Data-driven Computational Modelling And Applications > 01/09/2018 > 28/02/2025 > 2017

	File(s) associated to this reference
	Fulltext file(s): File Commentary Version Size Access Limited access 2022-CODS_COMAD.pdf Publisher postprint 704.27 kB Request a copy

All documents in ORBi^lu are protected by a user license.

 

University of Luxembourg Library |  Feedback |  Legal notices Site Map