Task-Oriented Communication Design in Cyber-Physical Systems: A Survey on Theory and Applications

MOSTAANI, Arsham; VU, Thang Xuan; SHARMA, Shree Krishna; Nguyen, Van-Dinh; Liao, Qi; CHATZINOTAS, Symeon

doi:10.1109/ACCESS.2022.3231039

Download

Article (Scientific journals)

Task-Oriented Communication Design in Cyber-Physical Systems: A Survey on Theory and Applications

MOSTAANI, Arsham; VU, Thang Xuan; SHARMA, Shree Krishna et al.

2022 • In IEEE Access

Peer Reviewed verified by ORBi

Permalink
https://hdl.handle.net/10993/53991

DOI
10.1109/ACCESS.2022.3231039

Files (1)Send to Details Statistics Bibliography Similar publications

Files

Full Text

Task-Oriented_Communication_Design_in_Cyber-Physical_Systems_A_Survey_on_Theory_and_Applications.pdf

Author preprint (9.1 MB)

Download

All documents in ORBilu are protected by a user license.

Send to

RIS BibTex APA Chicago Permalink X Linkedin

Details

Keywords :

Multi-agent communications; cyber-physical systems; task-effective communications

Abstract :

[en] Communication system design has been traditionally guided by task-agnostic principles, which aim at efficiently transmitting as many correct bits as possible through a given channel. However, in the era of cyber-physical systems, the effectiveness of communications is not dictated simply by the bit rate, but most importantly by the efficient completion of the task in hand, e.g., controlling remotely a robot, automating a production line or collaboratively sensing through a drone swarm. In parallel, it is projected that by 2023, half of the worldwide network connections will be among machines rather than humans. In this context, it is crucial to establish a new paradigm for designing communication strategies for multi-agent cyber-physical systems. This is a daunting task, since it requires a combination of principles from information, communication, control theories and computer science in order to formalize a general framework for task-oriented communication designs. In this direction, this paper reviews and structures the relevant theoretical work across a wide range of scientific communities. Subsequently, it proposes a general conceptual framework for task-oriented communication design, along with its specializations according to targeted use cases. Furthermore, it provides a survey of relevant contributions in dominant applications, such as industrial internet of things, multi-unmanned aerial vehicle (UAV) systems, autonomous vehicles, distributed learning systems, smart manufacturing plants, 5G and beyond self-organizing networks, and tactile internet. Finally, this paper also highlights the most important open research topics from both the theoretical framework and application points of view.

Research center :

- Interdisciplinary Centre for Security, Reliability and Trust (SnT) > SIGCOM - Signal Processing & Communications

Disciplines :

Electrical & electronics engineering

Author, co-author :

MOSTAANI, Arsham ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > SigCom

VU, Thang Xuan ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > SigCom

SHARMA, Shree Krishna ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > SigCom

Nguyen, Van-Dinh; VinUniversity > College of Engineering and Computer Science

Liao, Qi; Nokia Bell-Labs > Stuttgart

CHATZINOTAS, Symeon ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > SigCom

External co-authors :

yes

Language :

English

Title :

Task-Oriented Communication Design in Cyber-Physical Systems: A Survey on Theory and Applications

Publication date :

21 December 2022

Journal title :

IEEE Access

ISSN :

2169-3536

Publisher :

Institute of Electrical and Electronics Engineers, Piscataway, United States - New Jersey

Peer reviewed :

Peer Reviewed verified by ORBi

Focus Area :

Computational Sciences

Additional URL :

https://ieeexplore.ieee.org/document/9994683

European Projects :

H2020 - 742648 - AGNOSTIC - Actively Enhanced Cognition based Framework for Design of Complex Systems

Name of the research project :

Agnostic

Funders :

ERC Advanced Grant - 742648
CE - Commission Européenne

Available on ORBilu :

since 21 January 2023

Statistics

Number of views

74 (10 by Unilu)

Number of downloads

80 (5 by Unilu)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

OpenAlex citations

WoS citations^™

Bibliography

U. Cisco, "Cisco annual internet report (2018-2023) white paper, " Cisco, San Jose, CA, USA, 2020.
R. Dobrushin, "General formulation of Shannon's basic theorems of information theory, " AMS Transl., vol. 33, pp. 323-438, Jan. 1959.
S. Vembu, S. Verdú, and Y. Steinberg, "The source-channel separation theorem revisited, " IEEE Trans. Inf. Theory, vol. 41, no. 1, pp. 44-54, Jan. 1995.
P. Popovski, O. Simeone, F. Boccardi, D. Gündüz, and O. Sahin, "Semantic-effectiveness filtering and control for post-5G wireless connectivity, " J. Indian Inst. Sci., vol. 100, no. 2, pp. 435-443, Apr. 2020.
W.Weaver, "Recent contributions to the mathematical theory of communication, " in The Mathematical Theory of Communication, C. E. Shannon and W. Weaver, Eds. Champaign, IL, USA: Univ. Illinois Press, 1949, ch. 10.
E. Calvanese Strinati and S. Barbarossa, "6G networks: Beyond Shannon towards semantic and goal-oriented communications, " Comput. Netw., vol. 190, May 2021, Art. no. 107930.
G. Shi, Y. Xiao, Y. Li, and X. Xie, "From semantic communication to semantic-Aware networking: Model, architecture, and open problems, " IEEE Commun. Mag., vol. 59, no. 8, pp. 44-50, Aug. 2021.
M. Kountouris and N. Pappas, "Semantics-empowered communication for networked intelligent systems, " 2020, arXiv:2007.11579.
N. Pappas and M. Kountouris, "Goal-oriented communication for realtime tracking in autonomous systems, " in Proc. IEEE Int. Conf. Auto. Syst. (ICAS), Aug. 2021, pp. 1-5.
X. Luo, H.-H. Chen, and Q. Guo, "Semantic communications: Overview, open issues, and future research directions, " IEEE Wireless Commun., vol. 29, no. 1, pp. 210-219, Feb. 2022.
T.-Y. Tung, S. Kobus, J. P. Roig, and D. Gunduz, "Effective communications: A joint learning and communication framework for multiagent reinforcement learning over noisy channels, " IEEE J. Sel. Areas Commun., vol. 39, no. 8, pp. 2590-2603, Aug. 2021.
E. Uysal, O. Kaya, A. Ephremides, J. Gross, M. Codreanu, P. Popovski, M. Assaad, G. Liva, A. Munari, T. Soleymani, B. Soret, and K. H. Johansson, "Semantic communications in networked systems, " 2021, arXiv:2103.05391.
J. Bao, P. Basu, M. Dean, C. Partridge, A. Swami, W. Leland, and J. A. Hendler, "Towards a theory of semantic communication, " in Proc. IEEE Netw. Sci. Workshop, Jun. 2011, pp. 110-117.
M. Kalfa, M. Gok, A. Atalik, B. Tegin, T. M. Duman, and O. Arikan, "Towards goal-oriented semantic signal processing: Applications and future challenges, " Digit. Signal Process., vol. 119, Dec. 2021, Art. no. 103134.
I. F. Akyildiz, P. Wang, and S.-C. Lin, "Softair: A software defined networking architecture for 5G wireless systems, " Comput. Netw., vol. 85, pp. 1-18, Jul. 2015.
O. Alliance, "O-ran use cases and deployment scenarios, " O-RAN Alliance, Bonn, Germany, White Paper, Feb. 2020.
A. Ghosh, A. Maeder, M. Baker, and D. Chandramouli, "5G evolution: A view on 5G cellular technology beyond 3GPP release 15, " IEEE Access, vol. 7, pp. 127639-127651, 2019.
A. Aijaz, "Private 5G: The future of industrial wireless, " IEEE Ind. Electron. Mag., vol. 14, no. 4, pp. 136-145, Dec. 2020.
Q. Zhou, C.-X. Wang, S. McLaughlin, and X. Zhou, "Network virtualization and resource description in software-defined wireless networks, " IEEE Commun. Mag., vol. 53, no. 11, pp. 110-117, Nov. 2015.
X. Zhou, R. Li, T. Chen, and H. Zhang, "Network slicing as a service: Enabling enterprises' own software-defined cellular networks, " IEEE Commun. Mag., vol. 54, no. 7, pp. 146-153, Jul. 2016.
X. Foukas, N. Nikaein, M. M. Kassem, M. K. Marina, and K. Kontovasilis, "FlexRAN: A flexible and programmable platform for software-defined radio access networks, " in Proc. 12th Int. Conf. Emerg. Netw. Exp. Technol., New York, NY, USA, 2016, pp. 427-441.
A. Wyner and J. Ziv, "Bounds on the rate-distortion function for stationary sources with memory, " IEEE Trans. Inf. Theory, vol. IT-17, no. 5, pp. 508-513, Sep. 1971.
M. Sudan, H. Tyagi, and S. Watanabe, "Communication for generating correlation: A unifying survey, " IEEE Trans. Inf. Theory, vol. 66, no. 1, pp. 5-37, Jan. 2020.
P. W. Cuff, H. H. Permuter, and T. M. Cover, "Coordination capacity, " IEEE Trans. Inf. Theory, vol. 56, no. 9, pp. 4181-4206, Sep. 2010.
M. Mylonakis, P. A. Stavrou, and M. Skoglund, "Empirical coordination subject to a fldelity criterion, " in Proc. IEEE Inf. TheoryWorkshop (ITW), Aug. 2019, pp. 1-5.
A. Sahai and S. Mitter, "The necessity and sufficiency of anytime capacity for stabilization of a linear system over a noisy communication link-Part I: Scalar systems, " IEEE Trans. Inf. Theory, vol. 52, no. 8, pp. 3369-3395, Aug. 2006.
N. C. Martins, M. A. Dahleh, and N. Elia, "Feedback stabilization of uncertain systems using a stochastic digital link, " in Proc. 43rd IEEE Conf. Decis. Control (CDC), vol. 2, 2004, pp. 1889-1895, doi: 10.1109/CDC.2004.1430323.
R. Gilad-Bachrach, A. Navot, and N. Tishby, "An information theoretic tradeoff between complexity and accuracy, " in Learning Theory and Kernel Machines. Springer, 2003, pp. 595-609.
R. A. Amjad and B. C. Geiger, "Learning representations for neural network-based classification using the information bottleneck principle, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 42, no. 9, pp. 2225-2239, Sep. 2020.
P. Harremoes and N. Tishby, "The information bottleneck revisited or how to choose a good distortion measure, " in Proc. IEEE Int. Symp. Inf. Theory, Jun. 2007, pp. 566-570.
D. Gondek and T. Hofmann, "Conditional information bottleneck clustering, " in Proc. 3rd IEEE Int. Conf. Data Mining, Workshop Clustering Large Data Sets, Nov. 2003, pp. 36-42.
B. Larrousse, S. Lasaulce, and M. R. Bloch, "Coordination in distributed networks via coded actions with application to power control, " IEEE Trans. Inf. Theory, vol. 64, no. 5, pp. 3633-3654, May 2018.
S. Lasaulce and S. Tarbouriech, "Information constraints in multiple agent problems with I.I.D states, " in Control Subject to Computational and Communication Constraints. Springer, 2018, pp. 311-323.
J. W. Overstreet and A. Tzes, "An internet-based real-Time control engineering laboratory, " IEEE Control Syst. Mag., vol. 19, no. 5, pp. 19-34, Oct. 1999.
B. Aktan, C. A. Bohus, L. A. Crowl, and M. H. Shor, "Distance learning applied to control engineering laboratories, " IEEE Trans. Educ., vol. 39, no. 3, pp. 320-326, Aug. 1996.
R.W. Brocket and D. Liberzon, "Quantized feedback stabilization of linear systems, " IEEE Trans. Autom. Control, vol. 45, no. 7, pp. 1279-1289, Jul. 2000.
S. Tatikonda and S. Mitter, "Control under communication constraints, " IEEE Trans. Autom. Control, vol. 49, no. 7, pp. 1056-1068, Jul. 2004.
G. N. Nair and R. J. Evans, "Stabilizability of stochastic linear systems with finite feedback data rates, " SIAM J. Control Optim., vol. 43, no. 2, pp. 413-436, Jul. 2004.
N. C. Martins, M. A. Dahleh, and N. Elia, "Feedback stabilization of uncertain systems in the presence of a direct link, " IEEE Trans. Autom. Control, vol. 51, no. 3, pp. 438-447, Mar. 2006.
P. Minero, M. Franceschetti, S. Dey, and G. Nair, "Data rate theorem for stabilization over fading channels, " in Proc. 45th Ann. Allerton Conf. Commun., Control Comput., 2007, pp. 1-8.
M. Andreasson, D. V. Dimarogonas, H. Sandberg, and K. H. Johansson, "Distributed control of networked dynamical systems: Static feedback, integral action and consensus, " IEEE Trans. Autom. Control, vol. 59, no. 7, pp. 1750-1764, Jul. 2014.
D. Antunes and W. P. M. H. Heemels, "Rollout event-Triggered control: Beyond periodic control performance, " IEEE Trans. Autom. Control, vol. 59, no. 12, pp. 3296-3311, Dec. 2014.
W. Liu, G. Nair, Y. Li, D. Nesic, B. Vucetic, and H. V. Poor, "On the latency, rate, and reliability tradeoff in wireless networked control systems for IIoT, " IEEE Internet Things J., vol. 8, no. 2, pp. 723-733, Jan. 2021.
K. Huang, W. Liu, Y. Li, A. Savkin, and B. Vucetic, "Wireless feedback control with variable packet length for industrial IoT, " IEEE Wireless Commun. Lett., vol. 9, no. 9, pp. 1586-1590, Sep. 2020.
V. Kostina and B. Hassibi, "Rate-cost tradeoffs in control, " IEEE Trans. Autom. Control, vol. 64, no. 11, pp. 4525-4540, Nov. 2019.
A. S. Matveev and A. V. Savkin, Estimation and Control Over Commu-nication Networks. Springer, 2009.
G. N. Nair, S. Dey, and R. J. Evans, "In-mum data rates for stabilising Markov jump linear systems, " in Proc. 42nd IEEE Int. Conf. Decis. Control, vol. 2, Dec. 2003, pp. 1176-1181.
Y. Song, J. Yang, M. Zheng, and C. Peng, "Disturbance attenuation for Markov jump linear system over an additive white Gaussian noise channel, " Int. J. Control, vol. 89, no. 12, pp. 2482-2491, Dec. 2016.
H. Zhenglong, S. Yang, Z. Min, J. Li, and T. C. Yang, "Stabilization of Markov jump linear systems over Gaussian relay channel, " in Proc. UKACC 11th Int. Conf. Control (CONTROL), Aug. 2016, pp. 1-6.
C. Zhang, K. Chen, and G. E. Dullerud, "Stabilization of Markovian jump linear systems with limited information-A convex approach, " in Proc. Amer. Control Conf., Jun. 2009, pp. 4013-4019.
V. S. Borkar, S. K. Mitter, and S. Tatikonda, "Optimal sequential vector quantization of Markov sources, " SIAM J. Control Optim., vol. 40, no. 1, pp. 135-148, 2001.
D. P. Bertsekas and D. A. Castanon, "Adaptive aggregation methods for infinite horizon dynamic programming, " IEEE Trans. Autom. Control, vol. 34, no. 6, pp. 589-598, Jun. 1989.
D. Bertsekas, "Biased aggregation, rollout, and enhanced policy improvement for reinforcement learning, " 2019, arXiv:1910.02426.
M. L. Puterman and M. C. Shin, "Modified policy iteration algorithms for discounted Markov decision problems, " Manage. Sci., vol. 24, no. 11, pp. 1127-1137, Jul. 1978.
M. L. Puterman and M. C. Shin, "Action elimination procedures for modi fied policy iteration algorithms, " Oper. Res., vol. 30, no. 2, pp. 301-318, Apr. 1982.
M. A. Voelkel, A.-L. Sachs, and U. W. Thonemann, "An aggregationbased approximate dynamic programming approach for the periodic review model with random yield, " Eur. J. Oper. Res., vol. 281, no. 2, pp. 286-298, Mar. 2020.
F. Chatelin and W. L. Miranker, "Acceleration by aggregation of successive approximation methods, " Linear Algebra Appl., vol. 43, pp. 17-47, Mar. 1982.
E. Shaeepoorfard, M. Raginsky, and S. P. Meyn, "Rationally inattentive control of Markov processes, " SIAM J. Control Optim., vol. 54, no. 2, pp. 987-1016, Jan. 2016.
D. Maity, M. H. Mamduhi, S. Hirche, K. H. Johansson, and J. S. Baras, "Optimal LQG control under delay-dependent costly information, " IEEE Control Syst. Lett., vol. 3, no. 1, pp. 102-107, Jan. 2019.
S. Yüksel and T. Linder, "Optimization and convergence of observation channels in stochastic control, " SIAM J. Control Optim., vol. 50, no. 2, pp. 864-887, Jan. 2012.
C. A. Sims, "Implications of rational inattention, " J. Monetary Econ., vol. 50, no. 3, pp. 665-690, Apr. 2003.
A. Mostaani, T. X. Vu, S. Chatzinotas, and B. Ottersten, "Task-oriented data compression for multi-Agent communications over bit-budgeted channels, " IEEE Open J. Commun. Soc., vol. 3, pp. 1867-1886, 2022.
A. Mostaani, T. X. Vu, S. Chatzinotas, and B. Ottersten, "State aggregation for multiagent communication over rate-limited channels, " in Proc. IEEE Global Commun. Conf. (GLOBECOM), Dec. 2020, pp. 1-7.
A. Mostaani, O. Simeone, S. Chatzinotas, and B. Ottersten, "Learningbased physical layer communications for multiagent collaboration, " in Proc. IEEE 30th Annu. Int. Symp. Pers., Indoor Mobile Radio Commun. (PIMRC), Sep. 2019, pp. 1-6.
J. N. Tsitsiklis and B. Van Roy, "An analysis of temporal-difference learning with function approximation, " IEEE Trans. Autom. Control, vol. 42, no. 5, pp. 674-690, May 1997.
D. P. Bertsekas and J. N. Tsitsiklis, "Neuro-dynamic programming: An overview, " in Proc. 34th IEEE Conf. Decis. Control, vol. 1, Dec. 1995, pp. 560-564.
M. Riedmiller, "Neural fitted Q iteration-first experiences with a data efficient neural reinforcement learning method, " in Proc. Eur. Conf. Mach. Learn. Springer, 2005, pp. 317-328.
A. Antos, C. Szepesvári, and R. Munos, "Fitted Q-iteration in continuous action-space MDPs, " in Proc. Adv. Neural Inf. Process. Syst., 2008, pp. 9-16.
F. S. Melo, S. P. Meyn, and M. I. Ribeiro, "An analysis of reinforcement learning with function approximation, " in Proc. 25th Int. Conf. Mach. Learn. (ICML), 2008, pp. 664-671.
V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, "Playing atari with deep reinforcement learning, " 2013, arXiv:1312.5602.
V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, and J. Veness, "Humanlevel control through deep reinforcement learning, " Nature, vol. 518, no. 7540, pp. 529-533, 2015.
F. A. Oliehoek, M. T. J. Spaan, and N. Vlassis, "Optimal and approximate Q-value functions for decentralized POMDPs, " J. Artif. Intell. Res., vol. 32, no. 1, pp. 289-353, 2008.
F. A. Oliehoek and N. Vlassis, "Q-value functions for decentralized POMDPs, " in Proc. 6th Int. Joint Conf. Auto. Agents Multiagent Syst. (AAMAS), 2007, pp. 1-8.
F. A. Oliehoek, "Lossless clustering of histories in decentralized POMDPs, " in Proc. AAMAS, vol. 1, 2009, pp. 577-584.
R. Lowe, Y.Wu, A. Tamar, J. Harb, O. P. Abbeel, and I. Mordatch, "Multiagent actor-critic for mixed cooperative-competitive environments, " in Proc. Adv. Neural Inf. Process. Syst., Long Beach, CA, USA, 2017, pp. 6382-6393.
T. Chen, Y. Sun, and W. Yin, "LASG: Lazily aggregated stochastic gradients for communication-efficient distributed learning, " 2020, arXiv:2002.11360.
Y. Liu, Y. Sun, and W. Yin, "Decentralized learning with lazy and approximate dual gradients, " IEEE Trans. Signal Process., vol. 69, pp. 1362-1377, 2021.
D. Abel, D. Hershkowitz, and M. Littman, "Near optimal behavior via approximate state abstraction, " Proc. 33rd Int. Conf. Mach. Learn., vol. 48, M. F. Balcan K. Q. Weinberger, Eds. 2016, pp. 2915-2923.
L. Li, T. J. Walsh, and M. L. Littman, "Towards a unified theory of state abstraction for MDPs, " in Proc. Int. Symp. Artif. Intell. Math. (ISAIM), Fort Lauderdale, FL, USA, Jan. 2006, pp. 1-10. [Online]. Available: http://anytime.cs.umass.edu/aimath06/proceedings/P21.pdf
O. Nachum, S. Gu, H. Lee, and S. Levine, "Near-optimal representation learning for hierarchical reinforcement learning, " 2018, arXiv:1810.01257.
P. Ioannou and J. Sun, "Theory and design of robust direct and indirect adaptive-control schemes, " Int. J. Control, vol. 47, no. 3, pp. 775-813, 1988.
K. S. Narendra and L. S. Valavani, "Direct and indirect adaptive control, " IFAC Proc. Volumes, vol. 11, no. 1, pp. 1981-1987, 1978.
S. Sedighi, K. V. Mishra, M. R. B. Shankar, and B. Ottersten, "Localization with one-bit passive radars in narrowband Internet-of-Things using multivariate polynomial optimization, " IEEE Trans. Signal Process., vol. 69, pp. 2525-2540, 2021.
J. Liu, W. Zhang, and H. Vincent Poor, "A rate-distortion framework for characterizing semantic information, " 2021, arXiv:2105.04278.
A. Kipnis, S. Rini, and A. J. Goldsmith, "The rate-distortion risk in estimation from compressed data, " IEEE Trans. Inf. Theory, vol. 67, no. 5, pp. 2910-2924, May 2021.
N. Shlezinger and Y. C. Eldar, "Deep task-based quantization, " Entropy, vol. 23, no. 1, p. 104, 2021.
J. Dommel, Z. Utkovski, O. Simeone, and S. Stanczak, "Joint sourcechannel coding for semantics-Aware grant-free radio access in IoT fog networks, " IEEE Signal Process. Lett., vol. 28, pp. 728-732, 2021.
E. Raei, M. Alaee-Kerahroodi, and M. B. Shankar, "Spatial-And range-ISLR trade-off in MIMO radar via waveform correlation optimization, " IEEE Trans. Signal Process., vol. 69, pp. 3283-3298, 2021.
Z. Cheng, S. Shi, Z. He, and B. Liao, "Transmit sequence design for dualfunction radar-communication system with one-bit DACs, " IEEE Trans. Wireless Commun., vol. 20, no. 9, pp. 5846-5860, Sep. 2021.
P. Minero, M. Franceschetti, S. Dey, and G. N. Nair, "Data rate theorem for stabilization over time-varying feedback channels, " IEEE Trans. Autom. Control, vol. 54, no. 2, pp. 243-255, Feb. 2009.
W. Liu, P. Popovski, Y. Li, and B. Vucetic, "Wireless networked control systems with coding-free data transmission for industrial IoT, " IEEE Internet Things J., vol. 7, no. 3, pp. 1788-1801, Mar. 2020.
W. Liu, P. Popovski, Y. Li, and B. Vucetic, "Real-Time wireless networked control systems with coding-free data transmission, " in Proc. IEEE Global Commun. Conf. (GLOBECOM), Dec. 2019, pp. 1-6.
M. Pezzutto, F. Tramarin, S. Dey, and L. Schenato, "Adaptive transmission rate for LQG control overWi-Fi: A cross-layer approach, " Automat-ica, vol. 119, Sep. 2020, Art. no. 109092.
Y. Wu, H.-N. Dai, and H. Tang, "Graph neural networks for anomaly detection in industrial Internet of Things, " IEEE Internet Things J., vol. 9, no. 12, pp. 9214-9231, Jun. 2022.
V.-D. Nguyen and O.-S. Shin, "Cooperative prediction-And-sensingbased spectrum sharing in cognitive radio networks, " IEEE Trans. Cogn. Commun. Netw., vol. 4, no. 1, pp. 108-120, Mar. 2018.
Y. Zhen, W. Chen, L. Zheng, X. Li, and D. Mu, "Multiagent cooperative caching policy in industrial Internet of Things, " IEEE Internet Things J., vol. 9, no. 18, pp. 16770-16779, Sep. 2022.
D. Ullmann, S. Rezaeifar, O. Taran, T. Holotyak, B. Panos, and S. Voloshynovskiy, "Information bottleneck classification in extremely distributed systems, " Entropy, vol. 22, no. 11, p. 1237, Oct. 2020.
Y. Lin, S. Han, H. Mao, Y. Wang, and W. J. Dally, "Deep gradient compression: Reducing the communication bandwidth for distributed training, " 2017, arXiv:1712.01887.
F. Seide, H. Fu, J. Droppo, G. Li, and D. Yu, "1-bit stochastic gradient descent and its application to data-parallel distributed training of speech dnns, " in Proc. 15th Annu. Conf. Int. Speech Commun. Assoc., 2014.
W.Wen, C. Xu, F. Yan, C.Wu, Y.Wang, Y. Chen, and H. Li, "TernGrad: Ternary gradients to reduce communication in distributed deep learning, " in Proc. Adv. Neural Inf. Process. Syst., 2017, pp. 1509-1519.
S. Zhou, Y. Wu, Z. Ni, X. Zhou, H. Wen, and Y. Zou, "DoReFa-Net: Training low bitwidth convolutional neural networks with low bitwidth gradients, " 2016, arXiv:1606.06160.
M. Mohammadi Amiri, D. Gunduz, S. R. Kulkarni, and H. Vincent Poor, "Convergence of federated learning over a noisy downlink, " 2020, arXiv:2008.11141.
M. M. Amiri and D. Gündüz, "Machine learning at the wireless edge: Distributed stochastic gradient descent over-The-Air, " IEEE Trans. Signal Process., vol. 68, pp. 2155-2169, 2020.
M. Mohri, G. Sivek, and A. Theertha Suresh, "Agnostic federated learning, " 2019, arXiv:1902.00146.
Z. Li, C. Wang, and C.-J. Jiang, "User association for load balancing in vehicular networks: An online reinforcement learning approach, " IEEE Trans. Intell. Transp. Syst., vol. 18, no. 8, pp. 2217-2228, Aug. 2017.
A. Mohajer, M. Bavaghar, and H. Farrokhi, "Mobility-Aware load balancing for reliable self-organization networks: Multi-Agent deep reinforcement learning, " Rel. Eng. Syst. Saf., vol. 202, Oct. 2020, Art. no. 107056.
Y. Xu, W. Xu, Z.Wang, J. Lin, and S. Cui, "Load balancing for ultradense networks:Adeep reinforcement learning-based approach, " IEEE Internet Things J., vol. 6, no. 6, pp. 9399-9412, Dec. 2019.
R. Amiri, M. A. Almasi, J. G. Andrews, and H. Mehrpouyan, "Reinforcement learning for self organization and power control of two-Tier heterogeneous networks, " IEEE Trans. Wireless Commun., vol. 18, no. 8, pp. 3933-3947, Aug. 2019.
S. S.Mwanje, L. C. Schmelz, and A. Mitschele-Thiel, "Cognitive cellular networks: A Q-learning framework for self-organizing networks, " IEEE Trans. Netw. Service Manag., vol. 13, no. 1, pp. 85-98, Mar. 2016.
P. Muñoz, R. Barco, and I. de la Bandera, "Load balancing and handover joint optimization in LTE networks using fuzzy logic and reinforcement learning, " Comput. Netw., vol. 76, pp. 112-125, Jan. 2015.
W. Liu, X. Zang, Y. Li, and B. Vucetic, "Over-The-Air computation systems: Optimization, analysis and scaling laws, " IEEE Trans. Wireless Commun., vol. 19, no. 8, pp. 5488-5502, Aug. 2020.
J. Dong, Y. Shi, and Z. Ding, "Blind over-The-Air computation and data fusion via provableWirtinger flow, " IEEE Trans. Signal Process., vol. 68, pp. 1136-1151, 2020.
M. Frey, I. Bjelakovic, and S. Stanczak, "Over-The-Air computation in correlated channels, " 2020, arXiv:2007.02648.
P. G. Otanez, J. R. Moyne, and D. M. Tilbury, "Using deadbands to reduce communication in networked control systems, " in Proc. Amer. Control Conf., vol. 4, May 2002, pp. 3015-3020.
E. Steinbach, S. Hirche, J. Kammerl, I. Vittorias, and R. Chaudhari, "Haptic data compression and communication, " IEEE Signal Process. Mag., vol. 28, no. 1, pp. 87-96, Jan. 2011.
S. Hirche and M. Buss, "Transparent data reduction in networked telepresence and teleaction systems. Part II: Time-delayed communication, " Presence: Teleoperators Virtual Environ., vol. 16, no. 5, pp. 532-542, 2007.
M. Mukherjee, M. Guo, J. Lloret, and Q. Zhang, "Leveraging intelligent computation offloading with fog/edge computing for tactile internet: Advantages and limitations, " IEEE Netw., vol. 34, no. 5, pp. 322-329, Sep. 2020.
U. Aÿmann, C. Baier, C. Dubslaff, D. Grzelak, S. Hanisch, A. P. P. Hartono, S. Köpsell, T. Lin, and T. Strufe, "Tactile computing: Essential building blocks for the tactile internet, " in Tactile Internet. Amsterdam, The Netherlands: Elsevier, 2021, pp. 293-317.
C. Shahabi, A. Ortega, and M. R. Kolahdouzan, "A comparison of different haptic compression techniques, " in Proc. IEEE Int. Conf. Multimedia Expo, vol. 1, Aug. 2002, pp. 657-660.
P. Hinterseer, S. Hirche, S. Chaudhuri, and E. Steinbach, "Perceptionbased data reduction and transmission of haptic data in telepresence and teleaction systems, " IEEE Trans. Signal Process., vol. 56, no. 2, pp. 588-597, Feb. 2008.
J. Cui, Y. Liu, and A. Nallanathan, "Multi-Agent reinforcement learningbased resource allocation for UAV networks, " IEEE Trans. Wireless Commun., vol. 19, no. 2, pp. 729-743, Feb. 2020.
J. Hu, H. Zhang, and L. Song, "Reinforcement learning for decentralized trajectory design in cellular UAV networks with sense-And-send protocol, " IEEE Internet Things J., vol. 6, no. 4, pp. 6177-6189, Aug. 2019.
Q. Wang, W. Zhang, Y. Liu, and Y. Liu, "Multi-UAV dynamic wireless networking with deep reinforcement learning, " IEEE Commun. Lett., vol. 23, no. 12, pp. 2243-2246, Dec. 2019.
H. Qie, D. Shi, T. Shen, X. Xu, Y. Li, and L. Wang, "Joint optimization of multi-UAV target assignment and path planning based on multi-Agent reinforcement learning, " IEEE Access, vol. 7, pp. 146264-146272, 2019.
X. Liu, Y. Liu, and Y. Chen, "Reinforcement learning in multiple-UAV networks: Deployment and movement design, " IEEE Trans. Veh. Tech-nol., vol. 68, no. 8, pp. 8036-8049, Aug. 2019.
N. H. Chu, D. T. Hoang, D. N. Nguyen, N. Van Huynh, and E. Dutkiewicz, "Joint speed control and energy replenishment optimization forUAV-Assisted IoT data collection with deep reinforcement transfer learning, " IEEE Internet Things J., early access, Feb. 14, 2022, doi: 10.1109/JIOT.2022.3151201.
Z. Chang, H. Deng, L. You, G. Min, S. Garg, and G. Kaddoum, "Trajectory design and resource allocation for multi-UAV networks: Deep reinforcement learning approaches, " IEEE Trans. Netw. Sci. Eng., early access, May 3, 2022, doi: 10.1109/TNSE.2022.3171600.
W. Wang, Y. Liu, R. Srikant, and L. Ying, "3M-RL: Multi-resolution, multi-Agent, mean-field reinforcement learning for autonomous UAV routing, " IEEE Trans. Intell. Transp. Syst., vol. 23, no. 7, pp. 8985-8996, Jul. 2022.
T. Ren, J. Niu, B. Dai, X. Liu, Z. Hu, M. Xu, and M. Guizani, "Enabling efficient scheduling in large-scale UAV-Assisted mobile-edge computing via hierarchical reinforcement learning, " IEEE Internet Things J., vol. 9, no. 10, pp. 7095-7109, May 2022.
H. Chang, Y. Chen, B. Zhang, and D. Doermann, "Multi-UAV mobile edge computing and path planning platform based on reinforcement learning, " IEEE Trans. Emerg. Topics Comput. Intell., vol. 6, no. 3, pp. 489-498, Jun. 2022.
R. Zhang, Q. Zong, X. Zhang, L. Dou, and B. Tian, "Game of drones: Multi-UAV pursuit-evasion game with online motion planning by deep reinforcement learning, " IEEE Trans. Neural Netw. Learn. Syst., early access, Feb. 14, 2022, doi: 10.1109/TNNLS.2022.3146976.
L.Wang, K.Wang, C. Pan, W. Xu, N. Aslam, and A. Nallanathan, "Deep reinforcement learning based dynamic trajectory control forUAV-Assisted mobile edge computing, " IEEE Trans. Mobile Comput., vol. 21, no. 10, pp. 3536-3550, Oct. 2022.
C. Zhan and Y. Zeng, "Energy minimization for cellular-connected UAV: From optimization to deep reinforcement learning, " IEEE Trans.Wireless Commun., vol. 21, no. 7, pp. 5541-5555, Jul. 2022.
X. Zhong and Y. Zhou, "A reinforcement learning trained fuzzy neural network controller for maintaining wireless communication connections in multi-robot systems, " Proc. SPIE, vol. 9119, May 2014, Art. no. 91190A.
S. H. Alsamhi, O. Ma, and M. S. Ansari, "Convergence of machine learning and robotics communication in collaborative assembly: Mobility, connectivity and future perspectives, " J. Intell. Robotic Syst., vol. 98, nos. 3-4, pp. 541-566, Jun. 2020.
M. P. Mota, A. Valcarce, J.-M. Gorce, and J. Hoydis, "The emergence of wireless MAC protocols with multi-Agent reinforcement learning, " 2021, arXiv:2108.07144.
D. O. Pop, A. Rogozan, C. Chatelain, F. Nashashibi, and A. Bensrhair, "Multi-Task deep learning for pedestrian detection, action recognition and time to cross prediction, " IEEE Access, vol. 7, pp. 149318-149327, 2019.
X. X. Na and D. J. Cole, "Modelling of a human driver's interaction with vehicle automated steering using cooperative game theory, " IEEE/CAA J. Autom. Sinica, vol. 6, no. 5, pp. 1095-1107, Sep. 2019.
K. Li, W. Ni, E. Tovar, and A. Jamalipour, "On-board deep Q-network for UAV-Assisted online power transfer and data collection, " IEEE Trans. Veh. Technol., vol. 68, no. 12, pp. 12215-12226, Dec. 2019.
S. Tomashevich and B. Andrievsky, "Improved adaptive coding procedure for transferring the navigation data between UAVs in formation, " in Proc. AIP Conf., 2018, vol. 2046, no. 1, Art. no. 020102.
M. Hüttenrauch, A. Ošić, and G. Neumann, "Deep reinforcement learning for swarm systems, " 2018, arXiv:1807.06613.
D. Baldazo, J. Parras, and S. Zazo, "Decentralized multi-Agent deep reinforcement learning in swarms of drones for-ood monitoring, " in Proc. 27th Eur. Signal Process. Conf. (EUSIPCO), Sep. 2019, pp. 1-5.
Y. Sung, A. K. Budhiraja, R. K. Williams, and P. Tokekar, "Distributed assignment with limited communication for multi-robot multi-Target tracking, " Auto. Robots, vol. 44, no. 1, pp. 57-73, Jan. 2020.
S. W. Loke, "Cooperative automated vehicles: A review of opportunities and challenges in socially intelligent vehicles beyond networking, " IEEE Trans. Intell. Vehicles, vol. 4, no. 4, pp. 509-518, Dec. 2019.
S. Kar, J. M. F. Moura, and H. V. Poor, "QD-learning: A collaborative distributed strategy for multi-Agent reinforcement learning through consensus C innovations, " IEEE Trans. Signal Process., vol. 61, no. 7, pp. 1848-1862, Apr. 2013.
L. Schenato, B. Sinopoli, M. Franceschetti, K. Poolla, and S. S. Sastry, "Foundations of control and estimation over lossy networks, " Proc. IEEE, vol. 95, no. 1, pp. 163-187, Jan. 2007.
A. S. Matveev and A. V. Savkin, "The problem of LQG optimal control via a limited capacity communication channel, " Syst. Control Lett., vol. 53, no. 1, pp. 51-64, Sep. 2004.
S. Chai and V. K. N. Lau, "Online trajectory and radio resource optimization of cache-enabled UAV wireless networks with content and energy recharging, " IEEE Trans. Signal Process., vol. 68, pp. 1286-1299, 2020.
G. Faraci, C. Grasso, and G. Schembra, "Design of a 5G network slice extension with MEC UAVs managed with reinforcement learning, " IEEE J. Sel. Areas Commun., vol. 38, no. 10, pp. 2356-2371, Oct. 2020.
F. Wu, H. Zhang, J. Wu, and L. Song, "Cellular UAV-To-device communications: Trajectory design and mode selection by multi-Agent deep reinforcement learning, " IEEE Trans. Commun., vol. 68, no. 7, pp. 4175-4189, Jul. 2020.
C. H. Liu, X. Ma, X. Gao, and J. Tang, "Distributed energy-efficient multi-UAV navigation for long-Term communication coverage by deep reinforcement learning, " IEEE Trans. Mobile Comput., vol. 19, no. 6, pp. 1274-1285, Jun. 2020.
Z. Fang, J.Wang, Y. Ren, Z. Han, H.V. Poor, and L. Hanzo, "Age of information in energy harvesting aided massive multiple access networks, " IEEE J. Sel. Areas Commun., vol. 40, no. 5, pp. 1441-1456, May 2022.
W. Wei, J. Wang, Z. Fang, J. Chen, Y. Ren, and Y. Dong, "3U: Joint design of UAV-USV-UUV networks for cooperative target hunting, " IEEE Trans. Veh. Technol., early access, Nov. 9, 2022, doi: 10.1109/TVT.2022.3220856.
F. Dressler, F. Klingler, M. Segata, and R. L. Cigno, "Cooperative driving and the tactile internet, " Proc. IEEE, vol. 107, no. 2, pp. 436-446, Feb. 2019.
A. Benloucif, A.-T. Nguyen, C. Sentouh, and J.-C. Popieul, "Cooperative trajectory planning for haptic shared control between driver and automation in highway driving, " IEEE Trans. Ind. Electron., vol. 66, no. 12, pp. 9846-9857, Dec. 2019.
M. During and K. Lemmer, "Cooperative maneuver planning for cooperative driving, " IEEE Intell. Transp. Syst. Mag., vol. 8, no. 3, pp. 8-22, Fall 2016.
M. Segata, "Toward communication strategies for platooning: Simulative and experimental evaluation, " IEEE Trans. Veh. Technol., vol. 64, no. 12, pp. 5411-5423, Dec. 2015.
A. Ghosh and S. Huang, "Cooperative traffic control where autonomous cars meet human drivers, " in Proc. SoutheastCon, Apr. 2019, pp. 1-6.
K. Sonoda and T. Wada, "Displaying system situation awareness increases driver trust in automated driving, " IEEE Trans. Intell. Vehicles, vol. 2, no. 3, pp. 185-193, Sep. 2017.
F. Yan, K.Wang, B. Zou, L. Tang, W. Li, and C. Lv, "LiDAR-based multitask road perception network for autonomous vehicles, " IEEE Access, vol. 8, pp. 86753-86764, 2020.
S. Li and R. Li, "Task allocation based on task deployment in autonomous vehicular cloud, " in Proc. IEEE 9th Int. Conf. Electron. Inf. Emergency Commun. (ICEIEC), Jul. 2019, pp. 450-454.
R. Shokri and V. Shmatikov, "Privacy-preserving deep learning, " in Proc. 22nd ACM SIGSAC Conf. Comput. Commun. Secur., 2015, pp. 1310-1321.
J. Konečný, H. Brendan McMahan, F. X. Yu, P. Richtárik, A. Theertha Suresh, and D. Bacon, "Federated learning: Strategies for improving communication efficiency, " 2016, arXiv:1610.05492.
S. J. Reddi, J. Konečný, P. Richtárik, B. Póczós, and A. Smola, "AIDE: Fast and communication efficient distributed optimization, " 2016, arXiv:1608.06879.
B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. Y. Arcas, "Communication-efficient learning of deep networks from decentralized data, " in Proc. 20th Int. Conf. Artif. Intell. Statist., 2017, pp. 1273-1282.
D. Wen, X. Li, Q. Zeng, J. Ren, and K. Huang, "An overview of dataimportance aware radio resource management for edge machine learning, " J. Commun. Inf. Netw., vol. 4, no. 4, pp. 1-14, Dec. 2019.
D. Liu, G. Zhu, J. Zhang, and K. Huang, "Data-importance aware user scheduling for communication-efficient edge machine learning, " IEEE Trans. Cogn. Commun. Netw., vol. 7, no. 1, pp. 265-278, Mar. 2021.
A. Molin, H. Esen, and K. H. Johansson, "Scheduling networked state estimators based on value of information, " Automatica, vol. 110, Dec. 2019, Art. no. 108578.
Z. Goldfeld and Y. Polyanskiy, "The information bottleneck problem and its applications in machine learning, " IEEE J. Sel. Areas Inf. Theory, vol. 1, no. 1, pp. 19-38, May 2020.
R. Shwartz-Ziv and N. Tishby, "Opening the black box of deep neural networks via information, " 2017, arXiv:1703.00810.
K. Yang, T. Jiang, Y. Shi, and Z. Ding, "Federated learning via overthe-Air computation, " IEEE Trans. Wireless Commun., vol. 19, no. 3, pp. 2022-2035, Mar. 2020.
A. Lobinger, S. Stefanski, T. Jansen, and I. Balan, "Coordinating handover parameter optimization and load balancing in LTE self-optimizing networks, " in Proc. IEEE 73rd Veh. Technol. Conf. (VTC Spring), May 2011, pp. 1-5.
S. K. Sharma, I. Woungang, A. Anpalagan, and S. Chatzinotas, "Toward tactile internet in beyond 5G era: Recent advances, current issues, and future directions, " IEEE Access, vol. 8, pp. 56948-56991, 2020.
K. Antonakoglou, X. Xu, E. Steinbach, T. Mahmoodi, and M. Dohler, "Toward haptic communications over the 5G tactile internet, " IEEE Commun. Surveys Tuts., vol. 20, no. 4, pp. 3034-3059, 4th Quart., 2018.
H. Zhang, W. Chen, Z. Huang, M. Li, Y. Yang, W. Zhang, and J. Wang, "Bi-level actor-critic for multi-Agent coordination, " in Proc. AAAI Conf. Artif. Intell., 2020, vol. 34, no. 5, pp. 7325-7332.
C. E. Shannon, "A mathematical theory of communication, " Bell Syst. Tech. J., vol. 27, no. 3, pp. 379-423, Jul./Oct. 1948.
T.-Y. Tung, S. Kobus, J. Roig Pujol, and D. Gunduz, "Effective communications: A joint learning and communication framework for multi-Agent reinforcement learning over noisy channels, " 2021, arXiv:2101.10369.
E. Vinogradov, H. Sallouha, S. De Bast, M. Mahdi Azari, and S. Pollin, "Tutorial on UAV: A blue sky view on wireless communication, " 2019, arXiv:1901.02306.
M. M. Azari, F. Rosas, K.-C. Chen, and S. Pollin, "Ultra reliable UAV communication using altitude and cooperation diversity, " IEEE Trans. Commun., vol. 66, no. 1, pp. 330-344, Jan. 2018.
Y. Yuan, L. Lei, T. X. Vu, S. Chatzinotas, S. Sun, and B. Ottersten, "Energy minimization in UAV-Aided networks: Actor-critic learning for constrained scheduling optimization, " IEEE Trans. Veh. Technol., vol. 70, no. 5, pp. 5028-5042, May 2021.
P. Kairouz et al., "Advances and open problems in federated learning, " 2019, arXiv:1912.04977.
J. Park, S. Samarakoon, M. Bennis, and M. Debbah, "Wireless network intelligence at the edge, " Proc. IEEE, vol. 107, no. 11, pp. 2204-2239, Nov. 2019.
J. Park, S. Samarakoon, A. Elgabli, J. Kim, M. Bennis, S.-L. Kim, and M. Debbah, "Communication-efficient and distributed learning over wireless networks: Principles and applications, " Proc. IEEE, vol. 109, no. 5, pp. 796-819, May 2021.
S. U. Stich, "Local SGD converges fast and communicates little, " in Proc. Int. Conf. Learn. Represent., 2019, pp. 1-19.
Y. Zhang, J. C. Duchi, and M. J. Wainwright, "Communication-efficient algorithms for statistical optimization, " J. Mach. Learn. Res., vol. 14, pp. 3321-3363, Nov. 2013.
S. Wang, T. Tuor, T. Salonidis, K. K. Leung, C. Makaya, T. He, and K. Chan, "Adaptive federated learning in resource constrained edge computing systems, " IEEE J. Sel. Areas Commun., vol. 37, no. 6, pp. 1205-1221, Jun. 2019.
V.-D. Nguyen, S. K. Sharma, T. X. Vu, S. Chatzinotas, and B. Ottersten, "Efficient federated learning algorithm for resource allocation in wireless IoT networks, " IEEE Internet Things J., vol. 8, no. 5, pp. 3394-3409, Mar. 2021.
V.-D. Nguyen, S. Chatzinotas, B. Ottersten, and T. Q. Duong, "Fed-Fog: Network-Aware optimization of federated learning over wireless fog-cloud systems, " IEEE Trans. Wireless Commun., vol. 21, no. 10, pp. 8581-8599, Oct. 2022.
J. C. Duchi, M. I. Jordan, and M. J. Wainwright, "Privacy aware learning, " J. ACM, vol. 61, pp. 1-57, Dec. 2014, doi: 10.1145/2666468.
H. Brendan McMahan, D. Ramage, K. Talwar, and L. Zhang, "Learning differentially private recurrent language models, " 2017, arXiv:1710.06963.
Z. Cao, P. Zhou, R. Li, S. Huang, and D. Wu, "Multiagent deep reinforcement learning for joint multichannel access and task offloading of mobile-edge computing in industry 4.0, " IEEE Internet Things J., vol. 7, no. 7, pp. 6201-6213, Jul. 2020.
Telecommunication Management; Self-Organizing Networks (SON); Concepts and Requirements, Rel-16, document 3GPP TS 32.500 Jul. 2020. [Online]. Available: https://www.3gpp.org
H. Xu, S. Feng, Y. Zhang, and L. Li, "A grouping-based cooperative driving strategy for CAVs merging problems, " IEEE Trans. Veh. Technol., vol. 68, no. 6, pp. 6125-6136, Jun. 2019.
J. He, Z. Tang, X. Fu, S. Leng, F. Wu, K. Huang, J. Huang, J. Zhang, Y. Zhang, A. Radford, L. Li, and Z. Xiong, "Cooperative connected autonomous vehicles (CAV): Research, applications and challenges, " in Proc. IEEE 27th Int. Conf. Netw. Protocols (ICNP), Oct. 2019, pp. 1-6.
S. Aradi, "Survey of deep reinforcement learning for motion planning of autonomous vehicles, " IEEE Trans. Intell. Transp. Syst., vol. 23, no. 2, pp. 740-759, Feb. 2022.
S. Arora and P. Doshi, "A survey of inverse reinforcement learning: Challenges, methods and progress, " Artif. Intell., vol. 297, Aug. 2021, Art. no. 103500.
T. Fernando, S. Denman, S. Sridharan, and C. Fookes, "Deep inverse reinforcement learning for behavior prediction in autonomous driving: Accurate forecasts of vehicle motion, " IEEE Signal Process. Mag., vol. 38, no. 1, pp. 87-96, Jan. 2021.
J. García and F. Fernández, "A comprehensive survey on safe reinforcement learning, " J. Mach. Learn. Res., vol. 16, no. 1, pp. 1437-1480, 2015.
K. Zhang, Z. Yang, and T. Başar, "Multi-Agent reinforcement learning: A selective overview of theories and algorithms, " in Handbook of Reinforcement Learning and Control, K. G. Vamvoudakis, Y. Wan, F. L. Lewis, and D. Cansever, Eds. Cham, Switzerland: Springer, 2021, pp. 321-384, doi: 10.1007/978-3-030-60990-0-12.
T. T. Nguyen, N. D. Nguyen, and S. Nahavandi, "Deep reinforcement learning for multiagent systems: A review of challenges, solutions, and applications, " IEEE Trans. Cybern., vol. 50, no. 9, pp. 3826-3839, Sep. 2020.