Research · AI Security

Deepfakes & AI-Enabled Fraud

Generative AI makes it cheap to fake voices, faces and writing. Attackers use it to impersonate executives, bypass identity checks and scale phishing. Defenses depend as much on process as on detection technology.

The Impersonation Exposure Matrix

Generative AI has removed the cost barrier to faking a voice or a face. The weak point is rarely the detection technology. It is a business process that treats a familiar voice, or a face on a video call, as proof of identity. The Impersonation Exposure Matrix, a CorpExcellence framework, ranks each process on two questions: how much it relies on seeing or hearing a person, and how much damage a fraudulent request could cause. The answer places it in one of four quadrants, each with its own control set.

Verify out of band: high reliance, high impact

This quadrant holds wire approvals, vendor bank-detail changes, help desk password and MFA resets, and urgent executive requests made on calls. Controls here must work even when nobody spots the fake. Call back on a number from your own records, require a second approver who was not on the original call, and state in policy that no executive can waive these steps. Small teams can add a pre-agreed code phrase for verbal requests. Train help desk staff to expect pressure and to follow the script anyway.

Harden the process: low reliance, high impact

Payments and account openings that run through documents and systems are still exposed, because forged invoices and synthetic identity documents are now easy to produce. The FinCEN alert on deepfake media lists red flags for identity checks, such as photos that conflict with other identifying information and third-party webcam plugins used during verification. Vendor master changes deserve the same scrutiny as payments. Use dual control and change logs, and choose liveness checks that are tested against virtual cameras and injected video.

Train and label, then monitor: lower impact

Where stakes are lower, focus on awareness and fast reporting. Show staff real examples of cloned voices and face swaps, explain the pressure tactics attackers rely on (urgency and secrecy), and make reporting an attempt quick and blame-free. For images and video your organization publishes, attach Content Credentials so customers and partners can check where the media came from. Review attempt reports for patterns that point to a targeted campaign, and share confirmed attempts with your bank.

Where the industry is now

Provenance is becoming a normal expectation. The C2PA released version 2.3 of its Content Credentials specification in February 2026, adding live video support, and runs a conformance program for products that display credentials. In the EU, the AI Act’s Article 50 transparency duties, including deepfake disclosure by deployers, apply from August 2, 2026. Provenance only proves where labeled content came from; an unlabeled file proves nothing, and detection tools miss some fakes. Keep verification steps in the process itself.

Common pitfalls

• Buying a detection tool and leaving payment and reset procedures unchanged.
• Exempting senior executives from callback rules because it slows them down.
• Assuming a live video call cannot be faked in real time.