Don't Vibe Code Your Agent Infrastructure - Akshat Bubna, Modal
Build agent infrastructure around isolated, persistent, elastic sandboxes with shared storage, fine-grained network controls, and observability.
- Separate the agent harness from tool execution in isolated environments; use shared storage when agents need to collaborate.
- Persist workspace state without keeping compute running, and allocate CPU or GPU resources only when needed.
- Use network allowlists or a custom proxy, and expose logs, metrics, utilization, and budget data to agents.
The talk is mainly an overview of infrastructure needs and trade-offs, without a hands-on walkthrough. · Best for: Teams building coding, finance, research, or other agents that run tools and compute.
Akshat Bubna describes infrastructure patterns for agents that need to run code, persist work, collaborate, and scale. He argues that reliability, security, and scale make building a sandbox platform from scratch a poor fit for vibe coding.
Spreadsheet agents and sandbox architecture0:12
- Heavy AI uses a separate agent for each spreadsheet cell.
- Separate the harness, which runs the agent loop, from the containers or micro VMs that execute tools.
- Use shared storage for collaboration and make environments quick to start and stop.
Workspaces, persistence, and elastic compute3:52
- Keep each agent's relevant files, dependencies, and services together in a workspace.
- Persist workspace state across long pauses while stopping compute when the agent is idle.
- Resource needs vary across tasks and over time; avoid provisioning every sandbox for peak CPU and memory use.
- Agents may also need GPUs and other compute for workloads such as scientific models, game development, and experiments.
Collaboration, access controls, and observability9:02
- Agents and people may share files, including files used for agent memory and background updates.
- Control network access and inject credentials so the agent cannot read and exfiltrate them.
- For tighter controls than domain allowlists, use a proxy with rules for specific paths or operations.
- Expose logs, metrics, GPU utilization, and budget information so agents and people can diagnose runs.
Why not vibe code the platform12:19
- Reliability requires reasoning about failure cases and tail events.
- Security boundaries need ongoing attention as systems and threats evolve.
- Scaling sandbox creation and keeping environments interactive require substantial systems engineering.
Dynamic environments as the next step14:34
- The harness can choose environments dynamically, snapshot them, and retain them as artifacts.
- Bubna argues that agents can move beyond homogeneous server templates by creating and working across environments as needed.
- His closing point: strong compute, storage, and networking primitives let agents do more.
Tools & resources mentioned
- Modal sandboxes — isolated environments for running agent tools and workloads.
- Modal volumes — shared file storage for agents and collaborators.
- Ramp Inspect — example of an internal coding agent workload discussed in the talk.
- Cloud Science Modal integration — offloads CPU preprocessing and GPU scientific-model work to sandboxes.
- Modal blog post on scaling to one million sandboxes — cited as an example of the engineering involved in scale.
- Modal blog post on agents hill-climbing model training — cited as an example of agents using sandboxes for parallel experiments.
Notes are AI-generated from the video's transcript and may miss nuance.