OpenAI Releases Three More Voice Models: GPT-Realtime-2 Enters with GPT‑5‑Level Reasoning

Early this morning, OpenAI added three new voice models to its API in one go. The star is GPT-Realtime-2, officially described as “a voice model with GPT‑5‑level reasoning ability.” The other two fill in the ends of the audio pipeline—transcription (ASR) and speech synthesis (TTS). This is the first major update to OpenAI’s voice‑agent product line since the debut of GPT‑Realtime last August. In other words, OpenAI is now taking the voice space seriously to compete with ElevenLabs, Google’s Gemini Live, and a host of domestic teams developing Realtime solutions.

What’s Actually New This Time

Three released models:

• GPT‑Realtime‑2: end‑to‑end voice‑conversation model supporting speech‑in, speech‑out (S2S) with GPT‑5‑level contextual reasoning;
• gpt‑realtime‑transcribe‑2: new‑generation ASR replacing gpt‑4o‑transcribe, optimized for noisy environments and mixed languages;
• gpt‑realtime‑tts‑2: new‑generation TTS supporting fine‑grained emotion / pace / accent control with latency around 230 ms.

The key change isn’t “just more models,” but that OpenAI has re‑cut the voice stack. Previously, gpt‑4o‑realtime intertwined ASR, reasoning, and TTS in a single token stream; now these are split into three composable models while retaining GPT‑Realtime‑2’s end‑to‑end mode. You can now choose either extreme‑low‑latency end‑to‑end or separated modules for controllability and cost.

It’s a pragmatic decision. Developers who integrated the Realtime API last year know how “black‑boxy” the end‑to‑end model felt—no handle on when it interrupted, paused, or expressed emotion, leaving only prompt tuning. With the split, at least the TTS segment’s timing and SSML‑style can be directly controlled.

GPT‑Realtime‑2: Focus on Reasoning

The biggest complaint about the previous GPT‑Realtime wasn’t the voice—it was the brains. When conversations required multi‑step reasoning, tool calling, or long context, it fell into the classic trap of early voice models: off‑topic replies, random tool parameters, and lost state after interruptions.

GPT‑Realtime‑2 aims to fix this:

• Reasoning aligned to GPT‑5: official benchmark shows >20 pp improvement on Big Bench Audio, a new reasoning benchmark for voice models;
• More stable tool calls: supports parallel function‑calling with much better schema fidelity—critical for service or ordering agents;
• Stronger instruction following: OpenAI highlighted one metric—maintaining a polite tone in “angry customer” scenes improved from 61 % to 89 %, showing they truly validated with customer‑service PoCs;
• Context expanded to 32 k tokens: friendly for long calls and meetings, no need for manual rolling summaries.

Another small but useful upgrade: image input in realtime conversations. You can talk while tossing in pictures, letting the model see and answer at once. The official demo showed a field‑service case—an engineer sent a shot of a circuit board and asked “Is this capacitor burnt?” This will help teams doing industrial inspection or after‑sales support.

Latency and Pricing: No Surprise, But Fair

Official latency: “first‑byte audio < 320 ms (p50),” roughly equal to the previous generation. Given the increased parameter count, keeping that line means solid engineering work.

Pricing isn’t dramatically cheaper or pricier:

• GPT‑Realtime‑2: audio‑in $32 / M‑token, audio‑out $64 / M‑token (~20 % lower than before);
• gpt‑realtime‑transcribe‑2: $6 / M‑token;
• gpt‑realtime‑tts‑2: $15 / M‑token.

For comparison: ElevenLabs’ Conversational AI plan averages $0.08–0.15 per minute; GPT‑Realtime‑2 runs roughly $0.06–0.10 per minute at normal dialogue density. No pricing edge, but the model thinks better. For voice agents that must actually complete tasks, that trade‑off is worthwhile.

How to Call It: Fully Compatible with Realtime API

API protocol stays the same—still WebSocket‑based Realtime API; just change model names. A minimal example (all three models are live on OpenAI Hub; replace base_url and key to connect):

import asyncio, json, websockets

async def main():
    url = "wss://api.openai-hub.com/v1/realtime?model=gpt-realtime-2"
    headers = {
        "Authorization": "Bearer $OPENAI_HUB_KEY",
        "OpenAI-Beta": "realtime=v1",
    }
    async with websockets.connect(url, extra_headers=headers) as ws:
        await ws.send(json.dumps({
            "type": "session.update",
            "session": {
                "voice": "cedar",
                "modalities": ["audio", "text"],
                "turn_detection": {"type": "server_vad", "threshold": 0.5},
                "tools": [{
                    "type": "function",
                    "name": "lookup_order",
                    "parameters": {"type": "object", "properties": {"order_id": {"type": "string"}}}
                }]
            }
        }))
        # Then push audio frames via input_audio_buffer.append / commit
asyncio.run(main())

To use the separated TTS or ASR parts, just call the familiar REST APIs:

curl https://api.openai-hub.com/v1/audio/speech \
  -H "Authorization: Bearer $OPENAI_HUB_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-realtime-tts-2",
    "voice": "cedar",
    "input": "Read this sentence in a slightly apologetic and slower tone.",
    "instructions": "tone: apologetic; pace: slow"
  }' --output out.mp3

Note the new instructions field—TTS‑2’s natural‑language style control, far friendlier than SSML and instantly usable for prompt engineers.

Upgrades in Transcription and TTS

gpt‑realtime‑transcribe‑2 improvements:

1. Mixed Chinese‑English WER dropped from 9.2 % to 5.8 % (OpenAI’s own test set);
2. Auto‑detect 100+ languages—no explicit language parameter needed;
3. Word‑level timestamps opened for direct subtitle generation.

gpt‑realtime‑tts‑2 adds four voices (cedar, marin, juniper, willow) and retrains the previous ones. More notably, emotion control is visibly stronger—previously “make it happier” raised overall pitch awkwardly; now slight upward inflection occurs on key words, closer to real broadcasters. This is a substantial upgrade for audiobooks and podcasts.

Who’s Most Affected

In the short term, several groups will feel pressure:

1. Startups doing voice customer service / outbound calls: their ASR + LLM + TTS triple‑chain architectures usually had >1.5 s latency. GPT‑Realtime‑2’s end‑to‑end intelligence will become the new standard; lagging behind risks noticeable UX gaps.
2. ElevenLabs: emotional TTS is its core advantage, but GPT‑Realtime‑TTS‑2 now steps squarely into that territory.
3. Domestic Realtime‑solution teams: MiniMax, Step, Zhipu all released realtime voices earlier this year with varied metrics; GPT‑Realtime‑2 raises the reasoning + voice bar noticeably—pressure will carry into their next updates.

Still‑Unresolved Issues

To be fair, this release isn’t flawless:

• No voice cloning: OpenAI still limits users to preset voices for safety/compliance reasons, understandable but restricting to‑C use cases such as virtual companions or personalized audiobooks.
• Interrupt detection still via server VAD: Chinese has many pauses and filler words, so false triggers remain; for complex dialogue, better disable VAD and handle interruption manually.
• Few natural Chinese voices: of the four new voices, only one male voice sounds convincing; female voices remain from the previous generation.

Summary

GPT‑Realtime‑2 is OpenAI’s first revision that truly adds brains to real‑time voice. Earlier versions were demos—proof it could “talk.” This one finally aims to “get things done.” For teams already building voice agents, it’s worth migrating a PoC this week for comparison—the capability gap can shift with a single version. This update is worth half a day of testing.

All three models are already online at OpenAI Hub using the same Key; just switch base_url to try directly with the sample code above.

References

• OpenAI released three new voice models – linux.do: community first‑hand discussion and testing feedback
• OpenAI launched three new voice models – Zhihu: pricing and capability analysis for the voice API