Chapter 8
Run a controlled pilot
Updated
Put the system into a real workflow with a named owner.
Read
- How WeightWatchers embraces AI to engage members with empathy, at scale Domain language, policy integration, and a limited initial rollout.
- Morgan Stanley uses AI evals to shape the future of financial services Expert evaluation, retrieval, adoption, and reviewed meeting summaries.
- LLM Prompt Injection Prevention Cheat Sheet Separate untrusted content from authority to execute actions.Focus: Read indirect injection, least privilege, action checks, and the stated limits of model-based guardrails. Apply: Threat-model a retrieved document that asks for a write or data export; enforce permissions outside the model.
- MCP Security Best Practices · draft Token audience validation, proxy consent, and token-passthrough hazards.Focus: Read confused-deputy attacks, per-client consent, token audience checks, and the prohibition on token passthrough. Apply: Draw who issues each token, its audience, which component validates it, and the downstream authorization boundary.
Do
- Choose a pilot cohort and document eligibility and consent needs.
- Train users, observe actual sessions, and log support needs.
- Review quality and failure thresholds before expanding traffic.
You are done when: A pilot review with usage, failures, user feedback, and an explicit go/no-go decision.