AI BOM for Compliance and Security

Abstract

The rapid adoption of AI systems, particularly large language models, introduces significant challenges in regulatory compliance, transparency, and reproducibility.  Building on the concept of Software Bills of Materials, this project develops an AI Bill of Materials (AI BOM) as a mechanism to support traceability, auditability, and compliance across the AI lifecycle. Many compliance-relevant properties such as training data characteristics, model lineage, build configurations, and training-time modifications can only be reliably captured during training or build time and may not be available after deployment. Addressing this gap may require instrumentation or extensions of training frameworks such as PyTorch. The project therefore focuses on defining practical guidelines for manual collection of AI BOM data, with a longer-term goal of enabling automated and standardized generation to support reproducible and compliant AI systems.

From a security perspective, the project emphasizes verification and validation of AI BOM claims rather than model-specific vulnerabilities. This includes validating parent models of fine-tuned systems, quantization methods, and detecting inconsistencies caused by prompt-based jailbreaking or other integrity violations. To achieve model-agnostic security guarantees, the project develops both a test infrastructure – capable of interacting with deployed models or analyzing model weights and a plugin-based validation framework for detecting security-relevant deviations. Since real-world tainted models are rarely available, the project incorporates structured red teaming of language models to evaluate detection mechanisms.  Through active collaboration with universities and research groups, academic results and proof-of-concepts are re-implemented to pre-production quality, with the goal of establishing a community-driven, open-source foundation for AI compliance and security.

Learn More

• https://github.com/aibom-squad/SBOM-for-AI-Use-Cases
• https://www.linuxfoundation.org/research/ai-bom

Core Project Team

Marek Grac, Red Hat Research

Nora Haxidautiova, Red Hat Research

Donald Hunter, Emerging Technologies, Red Hat

Gabriela Dozortsev, Emerging Technologies, Red Hat