Strengthening Cyber Security in US Defense: The AI Bill of Materials

Recently the Pentagon has unveiled a comprehensive cybersecurity strategy aimed at bolstering the defense industrial base (DIB) against cyber threats. The DIB itself is a network of over 100,000 companies and subcontractors involved in the design, production, delivery, and maintenance of military weapons systems. The strategy, covering fiscal years 2024 through 2027, responds to the evolving cyber threats targeting this critical infrastructure, outlining goals to strengthen DOD governance, improve the cybersecurity posture of the DIB, preserve the resiliency of critical capabilities, and enhance collaboration within the DIB. Generative AI explosive growth has highlighted even greater potential for adoption of AI tools within industrial production and management.

The ARMY’s AI Vanguard

Parallel to these efforts, the U.S. Army considers a groundbreaking initiative: the AI Bill of Materials (AI BOM). This concept, inspired by software bill of materials practices, aims to ensure transparency and manage cybersecurity risks associated with the integration of artificial intelligence algorithms in defense technologies. By demanding a detailed account of the digital ingredients that compose these AI systems, the Army seeks to preempt potential vulnerabilities, such as Trojans, triggers, and poisoned data sets, without infringing on proprietary intellectual property.

Security Challenges of IoT — Mazhar, T.;etc.

AI Threats expose the need for BoM

In the rapidly evolving landscape of artificial intelligence (AI) and machine learning (ML), the integrity and security of AI systems have become paramount. Two significant threats to these systems are poisoned data sets and malicious text string bombs. These threats not only compromise the reliability of AI applications but also pose serious security risks. This research delves into the nature of these threats, their potential impact on AI systems, and explores the role of AI bill-of-materials (AI BoM) in mitigating these dangers.

Yansong Gao,et al.

Poisoned Data Sets

Poisoned data sets refer to data that has been intentionally tampered with or corrupted to manipulate the behavior of AI models. This tampering can be subtle, such as slightly altering the features of a dataset, or more overt, such as injecting malicious data that causes the model to fail or behave unpredictably.

Impact on AI Systems

The impact of poisoned data sets on AI systems can be profound. They can lead to incorrect or biased decision-making, reduce the accuracy of predictions, and in some cases, allow attackers to gain unauthorized access or control over AI-driven systems. For instance, in security-sensitive applications like fraud detection or autonomous vehicles, poisoned data sets can lead to catastrophic outcomes.

Attack payload generation

Malicious text string bombs are specially crafted inputs designed to trigger unexpected or harmful behavior in AI systems processing natural language data. These inputs exploit vulnerabilities in the AI’s parsing or processing mechanisms, causing it to crash, slow down significantly, or execute unintended actions.

Impact on AI Systems

The impact of malicious text strings filled with poisoned data is particularly severe in systems that rely heavily on natural language processing (NLP). They can disrupt services, lead to denial-of-service (DoS) attacks, or compromise data integrity and confidentiality. For businesses, this can translate into financial losses, reputation damage, and legal liabilities.

AI Bill-of-Materials (AI BoM)

An AI Bill-of-Materials (AI BoM) is a comprehensive inventory that lists all the components, data sets, and dependencies used in the development and deployment of an AI system. It serves as a critical tool for understanding the provenance and composition of AI models, thereby enabling better security and integrity checks.

Mitigating Risks

The AI BOM plays a pivotal role in identifying and safeguarding against the risks posed by poisoned data sets and malicious text string bombs. By providing transparency into the data sources, libraries, and third-party components used in AI systems, AI BoM allows developers and security professionals to:

• Traceability: Identify and trace the origins of data and components, making it easier to spot potentially malicious or compromised elements. This requires using trusted and vetted source of data only.
• Vulnerability Assessment: Conduct thorough vulnerability assessments and audits to detect and mitigate security risks before deployment.
• Compliance and Integrity Checks: Ensure compliance with data protection regulations and maintain the integrity of AI systems by verifying the authenticity and security of all components.

Effectiveness

Supply Chain Optimization: The AI BoM will create greater transparency across the defense industrial base with the exchange of operational AI and Generative AI’s data and model capabilities and components. This will increase the real-time adjustments in supply chain operations, ensuring agility and resilience in the face of dynamic market demands of military industrial production coordination.

This will strengthen data and security control for countering the risks of malicious data attacks against operational AI and generative AI systems. This hinges on its thoroughness and the commitment of organizational governance to maintain a high state of cybersecurity. Best practices such as Zero Trust Architecture and SASE can be better implemented controls for AI usage in military industrial networks and device systems.

It requires a collaborative effort among developers, security teams, senior management of both military and civilian and stakeholders to ensure that the AI BoM is comprehensive and up-to-date. When properly implemented, AI BoM can significantly enhance the security posture of AI systems, making them more resilient to attacks and ensuring their reliability and trustworthiness.

Conclusion

The AI Bill-of-Materials emerges as a crucial tool in mitigating these risks, offering a systematic approach to ensuring the security and integrity of AI systems. Also it will greatly enhance the effectiveness and adoption of ISO/IEC 42001:2023 framework for AI governance which emphasizes ethical integrity, reliability, and continuous improvement throughout the AI system lifecycle.

By fostering transparency, traceability, and accountability, AI BoM plays an indispensable role in protecting AI systems against the evolving landscape of cyber threats. The integration of AI and cybersecurity within the defense domain presents both unparalleled opportunities and formidable challenges for the United States of America and its allies.

Through initiatives like the AI BOM and the Pentagon’s cybersecurity strategy for the DIB, a new pathway should emerge, guiding the defense community towards a future where technology and security converge in the service of a safer, more resilient military.

Resources

Yigit, Y., Buchanan, W. J., Tehrani, M. G., & Maglaras, L. (2024). Review of generative AI methods in cybersecurity. arXiv. https://arxiv.org/abs/2403.08701v2

• National Telecommunications and Information Administration. (2024, March 27). NTIA calls for audits and investments in trustworthy AI systems. https://www.ntia.gov/press-release/2024/ntia-calls-audits-and-investments-trustworthy-ai-systems
• International Organization for Standardization. (2023). ISO/IEC 42001:2023, Artificial intelligence — Management system. Retrieved from https://www.iso.org/standard/81230.html[1]
• Mazhar, T.; Talpur, D.B.; Shloul, T.A.; Ghadi, Y.Y.; Haq, I.; Ullah, I.; Ouahada, K.; Hamam, H. Analysis of IoT Security Challenges and Its Solutions Using Artificial Intelligence. Brain Sci. 2023, 13, 683. https://doi.org/10.3390/brainsci13040683
• Demarest, C. (2023, May 31). US Army may ask defense industry to disclose AI algorithms. C4ISRNET. https://www.c4isrnet.com/artificial-intelligence/2023/05/31/us-army-may-ask-defense-industry-todisclose-ai-algorithms/
• Gao, Y., Doan, B. G., Zhang, Z., Ma, S., Zhang, J., Fu, A., Nepal, S., & Kim, H. (n.d.). Backdoor attacks and countermeasures on deep learning: A comprehensive review. arXiv. https://doi.org/10.48550/arXiv.2007.10760
• Tafas, J. (2024, January 15). Data poisoning: Safeguarding trust in AI innovation. LinkedIn. https://www.linkedin.com/pulse/data-poisoning-safeguarding-trust-ai-innovation-john-tafas-ux6fc/?trk=article-ssr-frontend-pulse_more-articles_related-content-card
• Simms, N. (n.d.). Data poisoning: A new threat to artificial intelligence. La Salle University Digital Commons. https://digitalcommons.lasalle.edu/cgi/viewcontent.cgi?article=1051&context=mathcompcapstones