An AI service can produce a correct answer and still expose information to the wrong person. It can protect information and still recommend an unsafe action. It can pass a demonstration and still leave nobody responsible for correcting a problem. Trust requires examining these properties separately.
For enterprise teams, responsible AI delivery becomes practical when requirements are translated into architecture, tests, permissions, and operating decisions. A policy statement provides direction, but users experience the system's actual behavior. The organization needs evidence that its stated boundaries are enforced where information and actions move.
Start with the use case and its consequences
Describe what the application will do, who will use it, and what could happen if it fails. A document-search assistant and a system that changes account permissions may use similar models while creating very different consequences. Risk assessment should reflect those differences.
NIST's AI Risk Management Framework 1.0 organizes activity around Govern, Map, Measure, and Manage. [1] Those functions offer a way to connect ownership, context, evaluation, and response. They are not a certificate of compliance or a guarantee that a particular deployment is safe.
In a benefits assistant, for example, map ordinary mistakes and serious failures separately. An awkward summary, an incorrect eligibility explanation, and disclosure of another employee's information require different preventive and corrective measures. The team should avoid treating them as interchangeable entries in a general error rate.
Distinguish reliability from security
Reliability concerns whether the service performs its intended function acceptably. Security includes protecting information and systems against unauthorized access or misuse. A secure service can be consistently inaccurate, while an accurate service can have excessive access.
NIST's Generative AI Profile identifies risks including confabulation, data privacy, information security, and human-AI configuration, which covers automation bias and over-reliance. [2] This breadth matters because a successful accuracy test cannot establish all the properties an organization needs.
Set separate acceptance criteria for different failure types. Test whether the assistant supports its conclusions, whether users can retrieve only permitted material, and whether the system correctly refuses an unauthorized action. A combined score can supplement these checks, but should not conceal a critical failure.
Treat external content as potentially hostile
A system may encounter instructions inside retrieved documents, emails, or websites that attempt to redirect its behavior. OWASP describes prompt injection as input that alters a model's intended behavior, including indirect attacks delivered through external content. [3]
The important implementation question is what happens after such content enters the workflow. The application should minimize unnecessary exposure, constrain available tools, and validate proposed actions. Asking the model to ignore malicious text is not sufficient evidence that the system is protected.
For the benefits example, a retrieved document should not be able to authorize exporting employee records. That prohibition should be enforced by the application and resource permissions. Test adversarial documents in a controlled environment and inspect both the final answer and attempted tool use. Our research on securing AI systems goes deeper on these threats.
Limit authority and preserve a meaningful approval
OWASP's excessive-agency guidance highlights risks from excessive functionality, permissions, and autonomy. [4] A useful review therefore asks three separate questions: which operations are exposed, which records are accessible, and which decisions can proceed without review?
Narrow tools can help. A read-only policy lookup has a different exposure from a general-purpose command interface. A draft update can be reviewed before commitment. Where approval is required, show the approver the proposed action, affected records, and important consequences.
An approval that appears after an irreversible action is only a notification. Similarly, a reviewer overwhelmed by routine prompts may be unable to exercise meaningful judgment. Reserve interruption for decisions that need it, and establish clear rejection and escalation paths.
Keep secure development and operational evidence
NIST SP 800-218A extends secure development guidance for generative AI and foundation-model contexts and is intended to be used with the SSDF. [5] AI-specific concerns therefore sit alongside established software responsibilities such as dependency management, testing, and vulnerability handling.
Retain enough evidence to investigate incidents: the relevant configuration, source identifiers, access decisions, tool outcomes, and release version. Apply appropriate retention and access rules to that evidence. Logs containing sensitive prompts or records need protection themselves.
Practice recovery. A team might need to disable an integration, revoke credentials, restore a previous configuration, or correct affected records. Identify who can authorize each response and how business users will learn that a service has been restricted.
Make accountability visible
Assign an owner for the business outcome and an owner for operating the service. Involve data owners, security specialists, and relevant policy experts in decisions that fall within their responsibilities. A shared project does not require ambiguous accountability.
Controls belong before, during, and after an interaction, and no single control establishes trust. The model is useful because it links each control to evidence and an owner, allowing the organization to examine gaps.
A responsible release decision should state what was tested, what remains uncertain, and why the remaining exposure is acceptable for the intended use. That decision must be revisited when the use case, information, model, permissions, or operating conditions change.
References
- NIST. Artificial Intelligence Risk Management Framework 1.0. January 2023.
- NIST. Generative Artificial Intelligence Profile, AI 600-1. July 2024.
- OWASP. LLM01:2025 Prompt Injection. 2025 edition.
- OWASP. LLM06:2025 Excessive Agency. 2025 edition.
- NIST. Secure Software Development Practices for Generative AI and Dual-Use Foundation Models: An SSDF Community Profile, SP 800-218A. July 2024.
Sources checked October 2026. CorpExcellence.com articles are best-effort research and analysis, not professional advice.
Leave A Comment