Building with GPT-Live and Image Generation | DevDay 2026
Build voice apps with GPT Live by letting it handle real-time conversation while a backend performs complex work.
- Prompt the voice model’s tone, speaking frequency, interruption behavior, and personality; use separate sessions to rehearse and evaluate conversations.
- Delegate complex work to a backend while the voice session continues; send updates as silent context, speakable content, or new instructions.
- Use turnless, continuous listening to handle interruptions and noisy settings, and let users change app controls by asking aloud.
Live demos show overlapping speech, delegation, background noise, and image generation in action. · Best for: Developers building voice-enabled apps or adding voice to existing agent workflows.
A live demonstration of GPT Live’s full-duplex voice interaction, configurable personality, backend delegation, and turnless architecture, with practical guidance for integrating these capabilities into apps.
Full-duplex conversation and model personality1:01
- GPT Live streams audio continuously, so it can listen and speak at the same time rather than relying on fixed speaker turns.
- The model handles turn-taking without a separate voice activity detector.
- Prompt its tone, accent, back-channeling, laughter, patience, eagerness, and whether it speaks.
- A separate voice session can act as a conversational partner or a simulated user for evaluations.
Delegation while conversation continues8:13
- The drawing game shows the voice model chatting while a backend draws and checks guesses.
- The app separates the UI, voice interface, and backend; the backend uses Astra 6 with code execution for drawing.
- Give the voice model relevant answers and hints so it can respond without waiting for a backend round trip on each guess.
- The backend can keep working and update state while the voice conversation and UI continue.
Send backend updates to the voice model11:42
- Use silent context updates to keep the model informed for later questions.
- Use speakable content when the model should announce information, such as scores.
- Use instruction updates to steer behavior, add guardrails, or change instructions.
- Choose update frequency based on what users need to know and what the UI already shows.
Choose a delegation mode13:58
- Client delegation sends events to your app, which can run its existing agent harness, business logic, or model.
- Return backend results as silent context, speakable content, or instructions.
- Responses delegation lets GPT Live supply conversation context to a configured model and tools through the Responses API.
- Responses delegation can use hosted tools such as web search or image generation.
Turnless interaction in noisy environments17:34
- The sound-control demo illustrates interaction while loud audio is playing, without waiting for silence.
- Voice can make app controls easier to access: users can describe what they want changed without finding the setting.
- Continuous listening lets the model decide whether to speak, keep listening, or invoke a tool amid background noise.
Image generation and getting started20:37
- The demo requests a 12-image grid while keeping the same voice session active.
- The presenters recommend starting with Codex, the OpenAI Developer plugin, and the GPT Live Starter app.
Step by step
- Decide which parts of your experience belong in the UI, the voice interface, and the backend.
- Prompt the voice model’s personality, speaking style, and interruption behavior; iterate until it fits the use case.
- Choose client delegation to use your own harness, or Responses delegation to have GPT Live manage requests through a configured model and tools.
- Send backend updates as silent context, speakable content, or instructions, based on what the user needs and what is already visible.
- Test conversations with separate voice sessions configured as simulated users, and check how the experience behaves during interruptions and background noise.
Tools & resources mentioned
- GPT Live — full-duplex voice model for continuous listening and speaking.
- Astra 6 with code execution — used by the demo backend to create drawings.
- OpenAI Responses API — delegation option for configured models and tools.
- Web search and image generation — hosted tools mentioned for Responses delegation.
- Codex — suggested starting point for building with the developer resources.
- OpenAI Developer plugin — resource to use with Codex.
- GPT Live Starter app — starter resource for building with GPT Live.
Notes are AI-generated from the video's transcript and may miss nuance.