Machine learning

OLIVEIRA KUHFUSS DE MENDONÇA, Marcele; Netto, Sergio L.; Diniz, Paulo S.R.; Theodoridis, Sergios

doi:10.1016/B978-0-32-391772-8.00019-3

No full text

Contribution to collective works (Parts of books)

Machine learning

OLIVEIRA KUHFUSS DE MENDONÇA, Marcele; Netto, Sergio L.; Diniz, Paulo S.R. et al.

2023 • In Signal Processing and Machine Learning Theory

Peer reviewed

Permalink
https://hdl.handle.net/10993/63460

DOI
10.1016/B978-0-32-391772-8.00019-3

Files (0)Send to Details Statistics Bibliography Similar publications

Files

Full Text

No document available.

Send to

RIS BibTex APA Chicago Permalink X Linkedin

Details

Keywords :

adversarial learning; artificial intelligence; deep learning; machine learning; neural networks; supervised learning; support vector machines; unsupervised learning; Engineering (all); Computer Science (all)

Abstract :

[en] Machine learning (ML) entails a set of tools and structures to acquire information from data. This chapter explains a wide range of tools to learn from data originating from distinct sources. The chapter reviews established learning concepts and details some classical tools to perform unsupervised and supervised learning. Then, deep learning algorithms and their structural variations are discussed, along with their suitability to solve specific problems. Complementing the remaining chapters of the book, we highlight some recent topics about ML, such as adversarial training and federated learning, including many illustrative examples. The aim is to equip the reader with a broad view of the current ML techniques and set the stage to access the details discussed in the remaining parts of the book. This chapter presents some fundamental concepts of ML that are broadly utilized and discusses some current ongoing investigations.

Disciplines :

Electrical & electronics engineering

Author, co-author :

OLIVEIRA KUHFUSS DE MENDONÇA, Marcele ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > SigCom

Netto, Sergio L.; Program of Electrical Engineering, Department of Electronics & Computer Engineering, COPPE/Poli/Universidade Federal do Rio de Janeiro, Rio de Janeiro, Brazil

Diniz, Paulo S.R.; Program of Electrical Engineering, Department of Electronics & Computer Engineering, COPPE/Poli/Universidade Federal do Rio de Janeiro, Rio de Janeiro, Brazil

Theodoridis, Sergios; Department of Informatics and Telecommunications, National and Kapodistrian University of Athens, Athens, Greece ; Electronic Systems Department, Aalborg University, Aalborg, Denmark

External co-authors :

yes

Language :

English

Title :

Machine learning

Publication date :

2023

Main work title :

Signal Processing and Machine Learning Theory

Publisher :

Elsevier

ISBN/EAN :

978-0-323-91772-8
978-0-323-97225-3

Peer reviewed :

Peer reviewed

Additional URL :

https://api.elsevier.com/content/article/PII:B9780323917728000193?httpAccept=text/xml

Available on ORBilu :

since 06 January 2025

Statistics

Number of views

44 (1 by Unilu)

Number of downloads

0 (0 by Unilu)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

OpenCitations

OpenAlex citations

Bibliography

A. Antoniou and W.-S. Lu. Practical Optimization - Algorithms and Engineering Applications. 2nd Edition, New York, NY: Springer, 2021.
Y. S. Abu-Mostafa, M. Magdon-Ismail, and H.-T. Lin. Learning from Data. AMLbook.com, Pasadena, CA, 2012.
C. M. Bishop. Pattern Recognition and Machine Learning. Springer, New York, NY, 2006.
T. Chen and C. Guestrin. “XGBoost: A scalable tree boosting system”. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2016), pp. 785794.
G. Chen. “A gentle tutorial of recurrent neural network with error backpropagation”. ArXiv abs/1610.02583v3 (2018).
G. Cohen et al. “EMNIST: an extension of MNIST to handwritten letters”. arXiv preprint arXiv:1702.05373 (2017).
J. M. Danskin. “The theory of max-min, with applications”. SIAM Journal on Applied Mathematics 14 (4) (1966), pp. 641-664.
H. Dbouk et al. “Low-complexity fixed-Point convolutional neural networks for automatic target recognition”. In: International Conference on Acoustics, Speech, and Signal Processing (ICASSP-2020). 2020, pp. 1598-1602.
J. Deng et al. “ImageNet: A large-scale hierarchichal image dataset”. in: Proc. Conference on Computer Vision and Pattern Recognition (2009).
M. P Deisenroth, A. A. Faisal, and C. S. Ong. Mathematics for Machine Learning. Cambridge, UK: Cambridge University Press, 2020.
J. C. Duchi, E. Hazan, and Y. Singer. “Adaptive subgradient methods for online learning and stochastic optimization”. Journal of Machine Learning Research 12 (2011), pp. 2121-2159.
C. T. Dinh et al. “Federated Learning over Wireless Networks: Covergence Analysis and Resource Allocation”. arXiv:1910.13067v4 (2020), pp. 1-15.
P. S. R. Diniz. Adaptive Filtering: Algorithms and Practical Implementation. 5th Edition, Springer, Cham, Switzerland, 2020.
R. Dey and F. M. Salem. “Gate-variants of Gated Recurrent Unit (GRU) neural networks”. In: 2017IEEE 60th 2017IEEE 60th International Midwest Symposium on Circuits and Systems (MWSCAS). 2017, pp. 1597-1600. doi:10.1109/MWSCAS.2017.8053243.
B. Efron and T. Hastie. Computer Age Statistical Inference. Cambridge, UK: Cambridge University Press, 2016.
J. H. Friedman. “Greedy function approximation: A gradient boosting machine”. The Annals of Statistics 29 (5) (2001), pp. 11891232.
Y. Freund and R. E. Schapire. “A decision-theoretic generalization of on-line learning and an application to boosting”. Journal of Computer and System Sciences 55 (1) (1997), pp. 119-139.
T. Gafni et al. “Federated Learning: An signal processing perspective”. IEEE Signal Processing Magazine (2022), pp. 14-41. doi:10.1109/MSP.2021.3125282.
X. Glorot and Y. Bengio. “Understanding the difficulty of training deep feedforward neural networks”. In: Proc. International Conference on Artificial Intelligence and Statistics. Sardinia, Italy, 2010, pp. 249-256.
X. Glorot, A. Bordes, and Y. Bengio. “Deep sparse rectifier neural networks”. In: Proceedings of the fourteenth international conference on artificial intelligence and statistics. 2011, pp. 315-323.
I. Goodfellow, Y. Bengio, and A. Courville. Deep Learning. Cambridge, MA: MIT Press, 2016.
I. J. Goodfellow, J. Shlens, and C. Szegedy. “Explaining and harnessing adversarial examples”. arXiv preprint arXiv:1412.6572 (2014).
G. H. Golub and C. F. Van Loan. Matrix Computations. 4th ed., Baltimore: John Hopkins University Press, 2013. isbn:978-1-421407944.
S. Haykin. Neural Networks and Learning Machines. Englewood Cliffs, NJ: Prentice-Hall, 2009.
K. He et al. “Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification”. In: Proc. International Conference on Artificial Intelligence and Statistics. Santiago, Chile, 2015, pp. 1026-1034.
D. O. Hebb. The Organization of Behavior: A Neural Psychological Theory. New York: John Wiley & sons, 1949. isbn: 9780415654531.
R. Hecht-Nielsen. “Theory of the backpropagation neural network”. In: Neural networks for perception. Elsevier, 1992, pp. 65-93.
S. Hochreiter and J. Schmidhuber. “Long short-term memory”. Neural Computation 9 (1997), pp. 1735-1780.
K. Hornik, M. Stinchcombe, and H. White. “Multilayer feedforward networks are universal approximators”. Neural Networks 2 (1989), pp. 359-366.
T. Hastie, R. Tibshirani, and J. Friedman. Elements of Statistical Learning: Data Mining, Inference, and Prediction. New York, NY: Springer, 2016.
K. Jarrett et al. “What is the best multi-stage architecture for object recognition?” In: Proc. International Conference on Computer Vision. Kyoto, Japan, 2009, pp. 2146-2153.
J. Kaddour et al. “Causal machine learning: A survey and open problems”. ArXiv https://arxiv.org/abs/2206.15475 (2022).
D. Kingma and J. Ba. “Adam: A method for stochastic optimization”. In: International Conference on Learning Representations. 2014, pp. 1-15. https://arxiv.org/pdf/1412.6980.pdf.
D. P. Kingma and J. Ba. “Adam: A method for stochastic optimization”. In: Proc. International Conference for Learning Representations. San Diego, USA, 2015, pp. 1-15.
J. Krohn, G. Beyleveld, and A. Bassens. Deep Learning Illustrated. USA: Pearson Addison-Wesley, 2020.
T. Kohonen. “Self-organized formation of topologically correct feature maps”. Biological Cybernetics 43 (1982), pp. 59-69.
T. Kohonen. “The self-organizing map”. Proceedings of the IEEE 78 (9) (1990), pp. 1464-148.
A. Krizhevsky, I. Sutskever, and G. E. Hinton. “ImageNet classification with deep convolutional networks”. Advances in Neural Information Processing Systems 25 (2) (2012).
Y. LeCun, C. Cortes, and C. J. C. Burges. MNIST dataset of handwritten digit. http://yann.lecun.com/exdb/mnist/.
Y. LeCun, L. Bottou, et al. “Gradient-based learning applied to document recognition”. Proceedings of the IEEE 86 (11) (1998), pp. 2278-2324.
X. Li et al. “On the Covergence of FedAvg on Non-IID Data”. arXiv:1907.02189v4 (2020), pp. 1-26.
A. Madry et al. “Towards deep learning models resistant to adversarial attacks”. arXiv preprint arXiv:1706.06083 (2017).
W. S. McCulloch and W. Pitts. “A logical calculus of the activities immanent in nervous activities”. Bulletin of Mathematical Biophysics 5 (1943), pp. 115-133.
W. J. Murdoch et al. “Definitions, methods, and applications in interpretable machine learning”. Proceedings of the National Academy of Science (PNAS) 116 (44) (2019), pp. 22071-22080.
K. P. Murphy. Machine Learning: A Probabilistic Perspective. Cambridge, MA: MIT Press, 2012.
V. Nair and G. E. Hinton. “Rectified linear units improve restricted Boltzmann machines”. In: Proc. International Conference on Machine Learning. Haifa, Israel, 2010.
A.B. Novikoff, On convergence proofs for perceptrons. Tech. rep. Stanford Research Institute, 1963. https://apps.dtic.mil/sti/pdfs/AD0298258.pdf.
G. Ortiz-Jimenez et al. “Optimism in the face of adversity:Optimism in the face of adversity: Understanding and improving deep learning through adversarial robustness”. Proceedings of the IEEE 109 (2021), pp. 635-659. doi:10.1109/JPROC.2021.3050042.
F. Rosenblatt. “The perceptron: A probabilistic model for information storage and organization in the brain”. Psychological Review 65 (1958), pp. 386-408.
A. Rahimi and B. Recht. “Random features for large-scale kernel machines”. In: NIPS'07: Proceedings of the 20th International Conference on Neural Information Processing Systems, Curran Associates, Inc., 2008, pp. 1-8.
S. Ruder. “An overview of gradient descent optimization algorithms”. ArXiv abs/1609.04747 (2016).
P. Ramachandran, B. Zoph, and Q. V. Le. “Searching for activation functions”. ArXiv abs/1710.05941 (2017).
B. Schölkopf, F. Locatello, et al. “Towards causal representation learning”. Proceedings of the IEEE 109 (2021), pp. 612-634. doi:10.1109/JPROC.2021.3058954.
J. Schmidhuber. “Deep learning in neural networks: An overview”. Neural networks 61 (2015), pp. 85-117.
National Institute of standards and Technology (NIST). Emnist Letters dataset. https://www.nist.gov/node/1298471/emnist-dataset.
J. B. O. Souza Filho et al. Online Component Analysis, Architectures and Applications. Foundations and Trends® in Signal Processing 16 (3-4) (2022), pp 224–429,. http://dx.doi.org/10.1561/2000000112. Boston, MA, USA: NOW Publishers
N. Srivastava et al. “Dropout: A simple way to prevent neural networks from overfitting”. Journal of Machine Learning Research 15 (2014), pp. 1929-1958.
B. Scholkopf and A. J. Smola. Learning with Kernels: State Vector Machines, Regularization, Optimization, and Beyond. Cambridge, MA: MIT Press, 2002.
Jiawei Su, Danilo Vasconcellos Vargas, and Kouichi Sakurai. “One pixel attack for fooling deep neural networks”. IEEE Transactions on Evolutionary Computation 23(5) (2019), pp. 828-841.
C. Szegedy et al. “Intriguing properties of neural networks”. arXiv preprint arXiv:1312.6199 (2013).
E. Tjoa and C. Guan. “A survey on explainable artificial intelligence (XAI): Toward medical XAI”. IEEE Transactions on Neural Networks and Learning Systems 32(11) (2021), pp. 4793-4813.
S. Theodoridis. Machine Learning: A Bayesian Optimization Perspective. 2nd Edition: Oxford, UK, Academic Press, 2020.
S. Theodoridis and K. Koutroumbas. Patter Recognition. London, UK: Academic Press, 2009.
V. N. Vapnik. Statistical Learning Theory. New Delhi, India: Wiley Interscience, 1998.
J. Watt, R. Borhani, and A. K. Katsaggelos. Machine Learning Refined: Foundations, Algorithms, and Applications. 2nd Edition, Cambridge, UK, Cambridge University Press, 2020.
D. West. “Neural network credit scoring models”. Computers & Operations Research 27(11-12) (2000), pp. 1131-1152.
R. Xu and D. Wunsch II. “Survey of clustering algorithms”. IEEE Trans. Neural Networks 16 (3) (2005), pp. 645-678.
M. D. Zeiler. “ADADELTA: an adaptive learning rate method”. arXiv preprint arXiv:1212.5701 (2012).