Chapter 4
Make the data usable
Updated
Turn permissions, schemas, and dirty records into explicit constraints.
Read
- Reflections on Palantir A former FDE’s firsthand account of field work, data integration, and product development.Focus: Focus on integrating data, working alongside users, and the tension between local usefulness and generality. Apply: Describe one belief about the workflow that changed after observing it.
- Designing robust and predictable APIs with idempotency A concrete integration pattern for retryable operations.Focus: Read request identifiers, server behavior, and retryable operations. Apply: Specify key scope, request-parameter matching, concurrent duplicate handling, and result retention.
- Using EXPLAIN · PostgreSQL 18 Read estimated versus actual query plans and diagnose expensive work.Focus: Use EXPLAIN basics, EXPLAIN ANALYZE, and caveats; compare estimates with actual rows. Apply: Annotate one slow plan with the first bad estimate, expensive operator, and a measured change.
- Authorization Cheat Sheet Deny by default and enforce authorization at every relevant request.Focus: Focus on least privilege, deny-by-default, per-request checks, and authorization regression tests. Apply: Create an access matrix with allowed and denied actions across users, tenants, resources, and exports.
- Zanzibar: Google’s Consistent, Global Authorization System PaperRelate ACL changes to content versions without serving newly forbidden data.Focus: Paper section 2: relation tuples, the “new enemy” problem, and consistency tokens. Apply: Specify revocation and content-update ordering for a retrieval service; demonstrate a forbidden read stays forbidden.
Do
- Document source schemas, access boundaries, refresh rates, and owners.
- Build a small ingestion path with duplicate and missing-data checks.
- Define what happens on stale data, access denial, or a failed write.
You are done when: A reproducible data sample and an integration contract another engineer can understand.