Forward deployed engineering case studies

Updated 1 November 2026

Firsthand accounts from people who did the job, and case studies of deployments that worked or did not.

Case studies

Read the numbers carefully

These are company reports and practitioner accounts, not independently audited benchmarks. Results describe the source’s deployment and period. A source being accessible verifies the citation, not the outcome or its causal explanation.

SPE

Cognition · Devin’s Playground

Internal app creation outpaced clear ownership of a protected deployment environment.

Nick Wong, Special Projects Engineer

Approach and result

Approach: Nick Wong describes building a shared environment, then exploring security products.
Reported result: A public account connects the internal project to Devin for Security; no quantified business effect is supplied.
Evidence limit: A company-hosted personal account, not an independent evaluation.
Discuss: Which ownership gap matters enough to justify a shared capability? Identify the user, baseline, intervention, measurement window, ongoing cost, and evidence you would need before repeating the claim.

Discovery

Ramp · Resolve the onboarding blocker

A requested feature appeared to require roughly three engineering days.

Forward Deployed Engineering

Approach and result

Approach: The team spoke directly with the customer and found a workaround during the call.
Reported result: The article reports avoiding the implementation.
Evidence limit: One anecdote; it does not quantify sustained adoption or maintenance cost.
Discuss: What did the customer need to accomplish, and which requirement was negotiable? Identify the user, baseline, intervention, measurement window, ongoing cost, and evidence you would need before repeating the claim.

Delivery

Palantir × Airbus · A350 workflow

Manufacturing work depended on fragmented information.

Reflections on Palantir

Approach and result

Approach: Qureshi describes connecting work orders, parts, quality issues, and schedules in one workflow.
Reported result: He reports the work contributed to a fourfold manufacturing ramp-up.
Evidence limit: A former practitioner’s retrospective. The software’s isolated causal contribution is not established.
Discuss: How would you validate that a workflow improvement reduces a production bottleneck? Identify the user, baseline, intervention, measurement window, ongoing cost, and evidence you would need before repeating the claim.

Delivery

OpenAI × semiconductor customer

Verification workflows needed useful AI under real engineering constraints.

What FDE means at OpenAI

Approach and result

Approach: FDEs learned the domain, built evaluations, and later consolidated custom implementations.
Reported result: OpenAI reports about 48% time savings for directed test generation and roughly 90% less custom code.
Evidence limit: An unnamed customer; independent methodology and cost accounting are not public.
Discuss: Which domain-specific evaluation would block an unsafe launch? Identify the user, baseline, intervention, measurement window, ongoing cost, and evidence you would need before repeating the claim.

Evaluation

OpenAI × telecom customer

Voice support exposed gaps in function calling and instruction following.

What FDE means at OpenAI

Approach and result

Approach: FDEs and researchers iterated against customer-defined evaluations.
Reported result: OpenAI reports key evaluation scores rising from about 60% to over 90% in six months.
Evidence limit: The evaluation set is not public; this is not an overall customer-satisfaction measure.
Discuss: When should you reduce scope, involve research, or stop the deployment? Identify the user, baseline, intervention, measurement window, ongoing cost, and evidence you would need before repeating the claim.

Evaluation

Thrive / Crete × OpenAI · Tax AI

Practitioner corrections were difficult to convert into durable system improvements.

Building self-improving tax agents with Codex

Approach and result

Approach: Capture corrections with production traces and turn failure patterns into evaluation targets.
Reported result: The article reports 7,000 pilot returns and about 50% higher throughput, with practitioner review.
Evidence limit: Provider-reported pilot results; field-completion accuracy is not the same as error-free returns.
Discuss: How would you preserve expert review while reducing total handling time? Identify the user, baseline, intervention, measurement window, ongoing cost, and evidence you would need before repeating the claim.

Adoption

Morgan Stanley · Knowledge retrieval

Advisors needed dependable access to a large internal knowledge base.

Morgan Stanley uses AI evals to shape the future of financial services

Approach and result

Approach: Expert-scored evaluations guided retrieval and response improvements.
Reported result: The source reports active use by more than 98% of advisor teams.
Evidence limit: Team-level adoption does not establish individual usage intensity or financial impact.
Discuss: What usage denominator and quality thresholds would make adoption meaningful? Identify the user, baseline, intervention, measurement window, ongoing cost, and evidence you would need before repeating the claim.

Delivery

Morgan Stanley · Debrief

Turning client conversations into usable notes and follow-ups required manual effort.

Morgan Stanley uses AI evals to shape the future of financial services

Approach and result

Approach: Meeting summaries enter the advisor workflow with consent and human review.
Reported result: The case describes CRM-integrated notes and draft follow-ups.
Evidence limit: A qualitative deployment account; no isolated ROI estimate is provided.
Discuss: How do you evaluate omissions, incorrect action items, and review workload? Identify the user, baseline, intervention, measurement window, ongoing cost, and evidence you would need before repeating the claim.

Adoption

Rakuten × Claude Code

Large development teams wanted shorter feature-delivery cycles.

Rakuten accelerates development with Claude Code

Approach and result

Approach: Teams adopted coding agents for development and refactoring work.
Reported result: The customer story reports feature time-to-market moving from 24 to 5 days.
Evidence limit: Vendor-hosted results; scope and counterfactual are limited. Do not generalize this improvement to every task.
Discuss: How would you compare similar work and account for review time? Identify the user, baseline, intervention, measurement window, ongoing cost, and evidence you would need before repeating the claim.

Discovery

Sonos × Sierra

Fast support interactions did not fully represent the customer’s goal: hearing music.

How Sonos elevates the listener experience with Sierra

Approach and result

Approach: The case connects AI support to time-to-music and broader customer-effort signals.
Reported result: A documented operating approach; no universal improvement percentage is claimed here.
Evidence limit: A customer story, with limited controlled measurement evidence.
Discuss: What is the end-to-end customer outcome beyond containment rate? Identify the user, baseline, intervention, measurement window, ongoing cost, and evidence you would need before repeating the claim.

Adoption

WeightWatchers × Sierra

Customer care needed domain-specific language and policy-aware assistance.

How WeightWatchers embraces AI to engage members with empathy, at scale

Approach and result

Approach: The team integrated business knowledge and systems, then started with a small set of use cases.
Reported result: The source displays approximately 70% resolution and a 4.6 CSAT.
Evidence limit: Publisher-reported measures; the visible headline does not establish sampling, scale, or causal lift.
Discuss: Which cases should always escalate, and how will you detect silent failures? Identify the user, baseline, intervention, measurement window, ongoing cost, and evidence you would need before repeating the claim.

Delivery

Nubank × Devin

A large monolith migration would consume substantial engineering capacity.

Building the Future of Software Engineering in Latin America

Approach and result

Approach: Cognition describes applying Devin to the migration.
Reported result: The article claims years-to-weeks delivery and more than 20× lower cost.
Evidence limit: An announcement with sparse methodology; no independent cost model or complete scope is disclosed.
Discuss: What would you request before using that claim in a business case? Identify the user, baseline, intervention, measurement window, ongoing cost, and evidence you would need before repeating the claim.

Adoption

Itaú × Devin

A bank needed to modernize and maintain a broad software estate.

Building the Future of Software Engineering in Latin America

Approach and result

Approach: The announcement describes repository documentation, migration, and vulnerability work.
Reported result: Cognition reports use by over 75% of technology teams and a 20–30% throughput increase.
Evidence limit: Vendor-reported adoption and throughput are different measures; neither proves causation alone.
Discuss: How would you define accepted output and prevent quality from falling? Identify the user, baseline, intervention, measurement window, ongoing cost, and evidence you would need before repeating the claim.

Architecture

Anthropic · Multi-agent research

Broad research tasks needed effective parallel exploration and synthesis.

How we built our multi-agent research system

Approach and result

Approach: The engineering team describes an orchestrator with specialist subagents, evaluation, and reliability work.
Reported result: A published production-system account with architecture and evaluation lessons.
Evidence limit: An internal product-engineering case, included as adjacent learning rather than evidence of an FDE engagement.
Discuss: When does parallel work justify coordination cost and extra token usage? Identify the user, baseline, intervention, measurement window, ongoing cost, and evidence you would need before repeating the claim.

Integration

Stripe · A live subscription migration

Subscriptions outgrew their original customer-record representation.

Online migrations at scale

Approach and result

Approach: Stage duplicate writes and backfill, validate the new representation, switch reads, then retire the old path.
Reported result: The engineering account describes a staged live migration and verification strategy.
Evidence limit: An adjacent software-engineering case, not an identified FDE engagement. The 2017 stack is historical.
Discuss: What must be true at every phase before you can advance or roll back? Identify the user, baseline, intervention, measurement window, ongoing cost, and evidence you would need before repeating the claim.

Reliability

GitHub · When failover made recovery harder

A short network partition triggered database failover into a topology the application could not tolerate.

October 21 post-incident analysis

Approach and result

Approach: The team preserved writes, degraded parts of the service, restored replication, and recovered queued work.
Reported result: GitHub reports 24 hours and 11 minutes of degradation after a 43-second connectivity loss.
Evidence limit: A company postmortem from 2018; included for recovery judgment, not as an FDE project or a current architecture description.
Discuss: Which application assumptions should a disaster-recovery exercise test beyond database health? Identify the user, baseline, intervention, measurement window, ongoing cost, and evidence you would need before repeating the claim.

Security

Google · Authorization that respects content changes

A stale access decision can expose newly changed content after a user’s permission is removed.

Zanzibar: Google’s Consistent, Global Authorization System

Approach and result

Approach: Zanzibar couples authorization reads to causal ordering and content-version consistency tokens.
Reported result: The paper describes a deployed authorization system and its production properties.
Evidence limit: A research paper about Google’s infrastructure, not an FDE customer engagement. Its scale is not a requirement for applying the invariant.
Discuss: How would the same invariant appear in an enterprise search cache or a generated summary? Identify the user, baseline, intervention, measurement window, ongoing cost, and evidence you would need before repeating the claim.