SIGNAL PROCESSING FOR PHYSICAL LAYER SECURITY WITH APPLICATION IN SATELLITE COMMUNICATIONS

Wireless broadcast allows widespread and easy information transfer. However, it may expose the information to unintended receivers, which could include eavesdroppers. As a solution, cryptography at the higher network levels has been used to encrypt and protect data. Cryptography relies on the fact that the computational power of the adversary is not enough to break the encryption. However, due to increasing computing power, the adversary power also increases. To further strengthen the security and complement the encryption, the concept of physical layer security has been introduced and surged an enormous amount of research. Widely speaking, the research in physical layer security can be divided into two directions: the information-theoretic and signal processing paradigms. This thesis starts with an overview of the physical layer security literature and continues with the contributions which are divided into the two following parts.

In the first part, we investigate the information-theoretic secrecy rate. In the first scenario, we study the confidentiality of a bidirectional satellite network consisting of two mobile users who exchange two messages via a multibeam satellite using the XOR network coding protocol. We maximize the sum secrecy rate by designing the optimal beamforming vector along with optimizing the return and forward link time allocation. In the second scenario, we study the effect of interference on the secrecy rate. We investigate the secrecy rate in a two-user interference network where one of the users, namely user 1, requires to establish a confidential connection. User 1 wants to prevent an unintended user of the network to decode its transmission. User 1 has to adjust its transmission power such that its secrecy rate is maximized while the quality of service at the destination of the other user, user 2, is satisfied. We obtain closed-form solutions for optimal joint power control. In the third scenario, we study secrecy rate over power ratio, namely ``secrecy energy efficiency''. We design the optimal beamformer for a multiple-input single-output system with and without considering the minimum required secrecy rate at the destination.
In the second part, we follow the signal processing paradigm to improve the security. We employ the directional modulation concept to enhance the security of a multi-user multiple-input multiple-output communication system in the presence of a multi-antenna eavesdropper. Enhancing the security is accomplished by increasing the symbol error rate at the eavesdropper without the eavesdropper's CSI. We show that when the eavesdropper has less antennas than the users, regardless of the received signal SNR, it cannot recover any useful information; in addition, it has to go through extra noise enhancing processes to estimate the symbols when it has more antennas than the users. Finally, we summarize the conclusions and discuss the promising research directions in the physical layer security.