Chapter 9
Measure adoption and value
Updated
Connect usage to outcomes and account for ongoing work.
Read
- How Sonos elevates the listener experience with Sierra Customer effort and time-to-music connect support engineering to the user’s goal.
- Rakuten accelerates development with Claude Code A customer account of coding-agent adoption and development throughput.
- Online Controlled Experiments at Large Scale PaperExperiment trustworthiness, surprising results, and organizational decision making.Focus: Read organizational adoption, experiment interactions, and skepticism toward surprising results. Apply: Specify the comparison, unit of assignment, guardrails, and what would invalidate a claimed lift.
- Leo Mehr: Ramp’s $44B Bet on Services PodcastHow Ramp’s FDE team ships agents inside large customers, and why data access is the hard part.Focus: The parts on data access, unwritten tribal knowledge, and why the model is usually the easy part. Apply: Before your next build, list the data you need, who controls access to it, and how long access will take.
Do
- Measure eligible users, active users, successful tasks, and repeat use.
- Estimate realized time or cost change including review and support.
- Check whether case mix, seasonality, or other changes explain the result.
You are done when: An impact memo with denominators, a comparison period, and uncertainty.