Chapter 7
Engineer for production
Updated
Handle partial failure and make operations understandable.
Read
- Timeouts, retries, and backoff with jitter Recover from partial failures without amplifying load or repeating side effects.Focus: Read timeout selection, retry amplification, backoff, and jitter. Apply: Give a dependency a time budget and bounded retry policy; identify the one layer responsible for retrying.
- Designing robust and predictable APIs with idempotency A concrete integration pattern for retryable operations.Focus: Read request identifiers, server behavior, and retryable operations. Apply: Specify key scope, request-parameter matching, concurrent duplicate handling, and result retention.
- Service Level Objectives Choose service indicators that reflect what users actually experience.
- Canarying Releases · The Site Reliability Workbook BookA partial rollout needs a meaningful control and a release decision.Focus: Read representative canaries, comparison signals, and automated rollout decisions. Apply: Define cohort, exposure, observation window, rollback trigger, and how low traffic affects detection.
- GuideLLM · workload and inference benchmarking Evaluate distributions of latency and throughput under representative load.Focus: Inspect load patterns, input/output length distributions, time-to-first-token, and end-to-end latency. Apply: Report latency distributions against request rate and task length; state model, hardware, and workload.
Do
- Add timeouts, bounded retries, idempotent writes, and useful traces.
- Define service objectives and a rollback or human-fallback path.
- Run a failure walkthrough with the future maintainer.
You are done when: A runbook demonstrating how to detect, contain, and recover from failure.