Research · AI Security
AI Supply Chain Security
AI applications depend on models, data sets, libraries and services built by others. AI supply chain security is about knowing what those components are, where they came from and whether they can be trusted.
What is in the AI supply chain
- Pretrained and open-weight models downloaded from public repositories.
- Training, fine-tuning and evaluation data sets.
- AI frameworks, libraries and plug-ins.
- Model APIs and hosted AI services.
- Tools and connectors that agents use.
Threats
- Models that contain malicious code in their files or hidden backdoor behavior.
- Poisoned or mislabeled data sets.
- Vulnerable or compromised open-source AI libraries.
- Unvetted plug-ins and tool connectors with broad access.
- Provider changes to hosted models that alter behavior.
Controls that help
Apply software supply chain practices to AI. Maintain an inventory of models and data sets, sometimes called an AI bill of materials; the CycloneDX standard, for example, supports machine learning components. Prefer model formats that do not execute code on load, scan artifacts before use, pin versions, verify sources and review licenses. Approve tool connectors the same way you approve software that accesses sensitive systems.
Common pitfalls
- Developers downloading models directly into production environments.
- No record of which model version is running where.
- Ignoring licenses and usage restrictions on models and data.
How to get started
- Create an approved source list for models and data sets.
- Scan models and AI libraries as part of the build pipeline.
- Record model and data provenance for every AI system.
- Review agent tools and plug-ins before enabling them.
Questions leaders should ask
- Where did the models we run come from, and who approved them?
- Do we scan model files before loading them?
- Which agent plug-ins and connectors are enabled, and what can they access?
- Could we list every model version running in production?