How Salesforce uses Claude Code
Scale coding agents by teaching engineers to plan and orchestrate work, then measure effective output and validate quality with evidence.
- Track coding ROI with effective output and cost per effective output, not lines of code.
- Move engineers from simple prompts to plans, sub-agents, parallel work, and automated workflows.
- Require agents to show evidence for quality checks and negative test cases before accepting changes.
The interview offers practical rollout and measurement ideas, which the notes capture. · Best for: Engineering leaders and teams adopting coding agents at scale
Salesforce describes using Claude Code across software development, scaling agent use through a maturity curve, shared skills, and workflow automation. They measure impact through effective output and validate agent work with benchmarks and quality checks.
Legacy applications and measuring ROI0:22
- Salesforce used Claude Code to make legacy applications more headless, reaching 80–90% API coverage across its clouds in weeks to a few months.
- The team worked with Stanford researchers on an effective-output score that distinguishes substantial code changes from minor edits or added comments.
- They track dollars per effective output and compare results before and after workflow or organizational changes.
Agent maturity and enablement3:13
- The maturity curve moves from simple instructions to planning, teams and sub-agents, parallel work, and finally autonomous workflows.
- Salesforce reported output doubling after moving engineers toward more agentic coding, without a matching increase in bugs or issues.
- Live demos and examples of shipped work helped engineers see how agents could help.
Skills, incident agents, and quality guardrails6:02
- Teams created incident-resolution agents; one example identified the responsible service and team and routed the issue instead of waking multiple teams.
- An engineering skill marketplace lets teams share skills and evaluate whether they improve outcomes or use fewer tokens.
- Salesforce said code volume nearly doubled while bugs per code generated were lower on average, alongside existing and new quality checks.
Benchmarks and proof of work9:51
- Consistent SDLC labels and links between tickets and pull requests provide data for evaluating changes.
- A benchmark set replays short- and long-horizon tasks to compare models, skills, and expected costs, including tasks models have repeatedly failed.
- Ask agents to show evidence for desired features and negative criteria, such as whether they changed unrelated schema fields.
Task sizing, model choice, and advice12:44
- Use quick interactive sessions for work that takes a few minutes; group larger work and let the model or harness decompose it.
- Choose a model based on task length and complexity; Salesforce described using Opus for a difficult, long-running bug investigation.
- The speakers recommend bottom-up innovation with appropriate guardrails and a learning environment where teams share what works.
Step by step
- Define a function-specific outcome metric and record a before-state; for coding, consider effective output and dollars per effective output.
- Teach engineers a progression from simple prompts to explicit plans, sub-agents, parallel work, and automated workflows.
- Encourage teams to demonstrate working examples and share useful skills across teams.
- Build a consistent dataset linking labeled tickets to pull requests, then replay benchmark tasks to compare models, skills, and costs.
- Set objective acceptance criteria, including negative checks, and require agents to provide evidence for their work.
- Review outcomes and quality after changes; choose models according to task length and complexity.
Tools & resources mentioned
- Effective-output score — measures the substance of generated code rather than raw line counts.
- Engineering skill marketplace — shares team skills and evaluates their effect on outcomes and token use.
- Benchmark eval set — replays SDLC tasks to compare models, skills, and costs.
- Ticket and pull-request labels — provide consistent data for evaluating engineering results.
Notes are AI-generated from the video's transcript and may miss nuance.