Research
AI Security
AI creates new attack surfaces and gives attackers new tools. This area covers securing AI systems, the AI supply chain, deepfakes and AI-enabled fraud, and the use of AI in security operations.
Topics in AI Security
Securing AI Systems (prompt injection, data poisoning)
Defending models, prompts, data and agents from attack.
Deepfakes & AI-Enabled Fraud
Countering synthetic media and AI-assisted social engineering.
Overview
AI systems can be manipulated in ways traditional software cannot. Prompt injection can redirect a model through hidden instructions, poisoned data can corrupt what a model learns, and agents with broad permissions can be turned against the systems they serve. Resources such as the OWASP Top 10 for Large Language Model Applications and MITRE ATLAS catalog these threats.
At the same time, attackers use generative AI to produce convincing phishing, synthetic voices and deepfake video. Defenders are also adopting AI to triage alerts and investigate incidents faster.
What leaders need to get right
- Threat modeling. Include AI-specific attacks in security design reviews.
- Least privilege. Treat models and agents as identities with minimal access.
- Supply chain. Know where models, data sets and AI libraries come from.
- Verification. Update payment and approval processes so a convincing voice or video is never enough.
Questions leaders should ask
- Have our AI applications been tested for prompt injection?
- Do we know the origin of every model and data set we use?
- Could a deepfake of an executive trigger a payment in our organization?
- Where is AI helping our security team, and how is it supervised?
Other research areas
Latest insights on AI Security
New articles in this area are on the way. In the meantime, explore the Research library.