Stop Chatting With Spreadsheets — Francesco Galletta & Felippe Felisola Caso, Monzo
Build financial AI workflows around explicit assumptions, tested code for calculations, traceable evidence, human decision gates, and independent checks.
- Interview users until goals, terms, assumptions, and open decisions are explicit; organize work into reviewable steps.
- Run repeatable calculations in versioned, tested code—not in the model’s arithmetic—and surface test results.
- Label inputs as facts, estimates, or assumptions; trace outputs to their sources and route consequential choices through human review.
The notes capture the five principles; the workshop UI walkthrough adds some practical detail. · Best for: Teams building AI agents for financial analysis or other high-stakes decisions.
A workshop on designing trustworthy financial-analysis agents. The presenters demonstrate a wedding-budgeting harness and argue that trust comes from the workflow and its controls, not from a single model or prompt.
Trust as the design goal5:14
- For financial tasks, speed is not the main bottleneck; users need confidence in results.
- The proposed framework has five principles: shared domain, consistency, evidence, accountability, and verification.
Establish a shared domain and plan20:48
- Start by explaining the goal, then clarify terms and assumptions through questions rather than jumping straight to implementation.
- Break the work into decision points; distinguish user-provided facts from model inferences and unresolved questions.
- Allow corrections and discussion of individual steps without losing the wider plan.
Make calculations consistent36:36
- Use code for calculations that should give the same result every time, rather than relying on model arithmetic.
- Test and version the code; surface changes that could affect results and let the harness guide users through building and running tests.
Keep evidence traceable56:20
- Show where inputs and outputs came from, and how calculations were performed.
- Distinguish facts, estimates, interpretations, and assumptions; track changes to both source data and code.
Build useful accountability gates1:06:31
- People remain accountable for agent outputs, so expose meaningful decisions and assumptions instead of treating the agent as a black box.
- Use a few high-signal review gates for consequential choices, rather than approving every tiny action or reviewing everything line by line.
- Let users correct inputs and branch into side questions without unintentionally disrupting the main analysis.
Verify with an independent perspective1:20:07
- Use a separate review or outside reference to challenge assumptions and catch results that seem implausible.
- Feed material challenges back into the plan; allow reviewers or users to dismiss challenges that do not apply.
- Choose the harness’s scope based on risk, importance, and how many people will rely on it.
When to build a dedicated harness1:33:23
- A dedicated harness is most useful when a task needs reproducibility, has meaningful consequences, or repeatedly fails in a general-purpose agent setup.
- For lower-risk tasks, prompts or skills with human review may be sufficient; avoid building infrastructure that is more work than using it.
Step by step
- Describe the financial goal and provide the context you have.
- Have the agent ask clarifying questions; resolve ambiguous terms and label facts, estimates, assumptions, and open decisions.
- Review the proposed plan as discrete decision points, correct errors, and approve the plan before execution.
- Implement repeatable calculations in code, then run unit tests or golden examples and track code versions.
- Trace results back to their inputs, sources, assumptions, and calculation code.
- Put high-impact decisions behind human review gates and allow corrections without losing unrelated work.
- Run an independent check against external references or domain knowledge; incorporate relevant challenges into the plan.
Tools & resources mentioned
- Workshop repo and UI — example harness and materials for trying the wedding-budget workflow; the transcript mentions a QR code/repo but provides no URL.
- Build-your-own guide in the repo — instructions for creating a similar harness with the workshop’s principles.
- Claude and ChatGPT — mentioned as general-purpose assistants, not as the workshop’s core implementation tools.
- Google Data Studio / Looker Studio — mentioned in the speaker’s career introduction.
Notes are AI-generated from the video's transcript and may miss nuance.