Your Company Has No Model of Itself. Agents Can Build One - Joao Wenzel, Citadel Securities
Build company memory as a time-aware state estimator that reconciles documents, conversations, and code with human feedback.
- Keep dated, append-only records that distinguish reported status from evidence and explain each update.
- Use specialist agents to extract and review signals; test predictions against what later happened.
- Collect feedback where users work, and distinguish corrections to a single fact from corrections to a general rule.
The talk explains a practical agent workflow and shows how historical backtests and evidence checks improve it. · Best for: Leaders and engineers building internal agents or organizational knowledge systems.
Joao Wenzel describes building an institutional memory system that estimates a company’s changing state from formal reports and noisier sources such as Slack, meetings, code changes, and opted-in agent traces. Agents propose updates; code and people govern changes. The system supports leadership reviews and gives other agents shared context.
Company strategy needs institutional memory0:12
- A useful company model should help answer where the company stands, what may happen next, and how it got there.
- Relevant context includes project history, team relationships, and information that may be missing from polished reports.
- Formal decks and strategy documents provide a top-down view; conversations, code, and session traces provide noisy but useful bottom-up signals.
Treat company knowledge as a state estimate3:17
- Track how old each claim is, where its evidence came from, and whether other sources support it.
- Combine slide structure with visual interpretation: XML alone can miss meaning, while screenshots can be hard to read accurately.
- Keep a longitudinal, auditable record so updates can be traced over time.
Build the update and aging loop6:01
- Use an update-age-observe-grade-correct loop; agents propose changes while code and people decide how records change.
- RAG alone can return a plausible status while missing conflicting or newer context.
- Maintain monthly records rather than rewriting previous months; distinguish what a report says from what evidence suggests.
- Track claim dates because facts such as project dates become less reliable over time.
Observe with deliberate workflows and evaluate predictions9:51
- A single model reviewing the whole corpus missed details; specialist agents and review agents produced more insightful updates.
- Use dated snapshots to backtest predictions against later outcomes.
- In the presented example, projects marked green but flagged by the agent were twice as likely to later turn yellow or red; flags helped identify what to inspect, not exactly what would fail.
Add human feedback and bottom-up signals11:48
- Make feedback easy to give in the apps people already use, such as Slack or email.
- Ask whether feedback corrects one fact or a reusable rule; rule-level feedback can improve multiple outputs.
- Extract claims and evidence from meeting notes, Slack threads, pull requests, and opted-in agent traces.
Use the model in operations and extend it carefully14:42
- Do not treat missing sensor data as evidence of zero; track when a source was last available and account for conflicting signals.
- Use the model to focus leadership discussions on discrepancies between reports and evidence, and to summarize what changed or went quiet.
- For uncertain or high-stakes conclusions, carefully choose when and how to seek stakeholder input.
Three takeaways17:32
- Think of company memory as a state estimator, not just an index of documents.
- Build evaluations from the start and compare estimates with what happens later.
- Design a feedback loop that lets agents and stakeholders work together.
Step by step
- Create dated records for important projects, claims, and evidence; preserve prior records rather than rewriting history.
- Ingest formal sources and bottom-up signals, such as decks, Slack, meetings, code changes, and opted-in agent traces.
- Use specialist agents to extract claims, connect them to people and projects, review evidence, and propose updates.
- Record what was reported, what the evidence indicates, the evidence source, and the reason for each proposed update.
- Track claim dates and signal freshness; treat missing data as unknown rather than as a zero or negative finding.
- Provide a simple feedback path in the tools stakeholders already use; record whether feedback changes a fact or a general rule.
- Backtest estimates using historical snapshots and later outcomes, then use the results to improve workflows.
- Use discrepancies, stale claims, and uncertainty to guide leadership reviews and decisions about when human confirmation is needed.
Tools & resources mentioned
- Claude — vision/model example and agent analysis; the speaker also refers generally to other models.
- PowerPoint/XML — formal deck source; combine file structure with a vision model’s reading of slide images.
- Slack — conversational claims, attachments, participants, and feedback.
- Email — another possible channel for collecting stakeholder feedback.
- Pull requests and code changes — evidence of what changed or shipped.
- Agent session traces — opted-in records of how tasks, decisions, and test failures unfolded.
Notes are AI-generated from the video's transcript and may miss nuance.