Use case
This independent case study applies the NIST AI Risk Management Framework to a hypothetical generative-AI assistant that drafts responses for customer-service agents. Agents review and send responses; the system does not communicate autonomously.
GOVERN
Governance establishes the conditions for every other activity.
- Name the accountable business owner, model/service owner and independent reviewers.
- Define risk appetite, prohibited uses, approval authority and exception process.
- Integrate AI risk with privacy, cyber, conduct, operational risk and vendor governance.
- Establish workforce competence, incident responsibilities and evidence retention.
- Maintain an inventory record with approval conditions and review date.
MAP
Mapping defines the context and people who may be affected.
- Intended purpose: help agents draft accurate, consistent responses using approved knowledge.
- Affected parties: customers, agents, vulnerable users, complaints teams and regulators.
- Foreseeable harms: misinformation, unfair treatment, privacy exposure, inappropriate tone, over-reliance and service exclusion.
- Dependencies: foundation-model provider, retrieval index, customer platform, identity controls and knowledge owners.
- Context limits: no final decisions on complaints, eligibility, vulnerability or legal rights.
MEASURE
Measurement should connect technical evaluation to real-world harm.
| Measure | Why it matters | Example threshold response | |---|---|---| | Unsupported-claim rate | Indicates hallucination exposure | Restrict or suspend affected intent | | Retrieval-source accuracy | Tests grounding quality | Remove stale source; re-index and retest | | Privacy leakage tests | Tests sensitive-data controls | Incident review and control redesign | | Outcome difference by group | Identifies unfair performance | Investigate data, prompts and workflow | | Human override rate | Signals usability or quality issues | Sample cases and interview agents | | Complaint and incident trends | Connects system to operational harm | Escalate by severity and recurrence |
Evaluation sets should reflect actual languages, customer intents, vulnerable circumstances and difficult edge cases. A single average accuracy number is insufficient.
MANAGE
Management turns evidence into prioritised action.
- Limit the pilot to selected intents and trained agents.
- Require source display and agent confirmation before sending.
- Route sensitive or consequential cases to established specialist teams.
- Set thresholds for rollback, suspension and governance escalation.
- Track remediation owners and due dates in the risk register.
- Reassess after model, data, process or regulatory change.
Residual-risk statement
With a constrained purpose, grounded sources, competent human review, representative testing and active monitoring, the use case may be suitable for a time-limited pilot. Residual misinformation, unfairness and automation-bias risk remains and requires explicit acceptance by the accountable owner.
Assurance questions
- Does evidence cover the real deployment context rather than a vendor demo?
- Can reviewers reproduce the evaluation and trace data, model and prompt versions?
- Are control failures linked to clear actions and decision authority?
- Do incident and complaint routes capture AI contribution?
- Has the organisation retained the ability to restrict, replace or stop the service?
The NIST AI RMF is used here as a decision framework, not as a badge or certification claim.