AI-Powered Defense: Reinforcement Learning in the Battle Against Hardware Trojans
Board Location: #71
Discipline: Computer Sciences and Information Management
Subcategory: Computer Engineering
Session: 3
Juan Torres, - New Mexico State University
Co-Author(s): Amin Sarihi, Abdel-Hameed A. Badawy, New Mexico State University
The ubiquity of computers in our daily lives has led to an exponential demand for microelectronic components. This surging demand has stretched the limits of supply chain oversight. With approximately 88% of production and 98% of assembly occurring (Hicks 2022), opportunities for malicious actors to exploit the vast and unmonitored production scale have emerged. The usage of unreliable 3PIPs (third-party intellectual properties) has also exacerbated the security aspects of chips. The insertion of hardware Trojans, defined as any unauthorized modification in integrated circuits, leading to erroneous outputs, denial of service or unauthorized leakage of sensitive information, poses a significant security risk.To address the pressing challenge of combating hardware Trojans, we try to expand thecurrent hardware Trojan benchmarks by adopting various Reinforcement Learning algorithms toinsert new unseen Trojan instances. This research builds upon the previously publishedimplementation from “Hardware Trojan Insertion Using Reinforcement Learning” (Sarihi et al.2022). In this expansion, we employ three reinforcement learning algorithms: recurrent proximity policy optimization (RPPO), advantage actor-critic (A2C), and soft actor-critic (SAC).Each algorithm uses a distinct strategy. RPPO uses recurrent neural networks to stay in a trusted region and improve learning gradually. A2C adopts an actor-critic framework, optimizing policy and value functions for efficient learning through exploration and exploitation. SAC focuses on exploration, advantageous for this problem. These algorithms undergo rigorous training and evaluation in a simulated environment, replicating the hardware Trojan insertion process.Our models are trained on a high-performance machine: AMD EPYC 7313 CPU (16 cores, 64 threads), 512GB DDR4 RAM (2600MHz), running openSUSE 15.4.We parallelized the original implementation using vectorization, training 60 parallel environments, enabling simultaneous Trojan insertions.Findings show RPPO consistently outperforms other algorithms by an average 7% in average episode reward, making it the preferred choice. RPPO’s recurrent neural networks offer slight but significant advantages over A2C and SAC.In future research, we’ll fine-tune SAC hyperparameters, utilize GPUs for faster training, evaluate insertions with our detection tool, and create an integrated model for combined insertion and detection with a reinforcing feedback loop.References:- A. Sarihi, A. Patooghy, P. Jamieson, and A.-H. A. Badawy, “Hardware trojan insertion using reinforcement learning,” in Proceedings of the Great Lakes Symposium on VLSI 2022, ser. GLSVLSI ’22. New York, NY, USA: Association for Computing Machinery, 2022, p. 139–142. – K. H. Hicks, “Securing defense-critical supply chains: An action plan developed in response to President Biden’s executive order 14017,” Office of the Deputy Secretary of Defense, Washington DC, Tech. Rep., February 2022.
Funder Acknowledgement(s): This work has received support from the National Science Foundation under grant number 1950121
Faculty Advisor: Abdel-Hameed A. Badawy, badawy@nmsu.edu
Role: In this research, my contributions involved implementing all three reinforcement learning algorithms into the existing framework, parallelizing the environments, training the models, and actively participating in data collection during the evaluation phase.