Qwen Voice Trio Now Available on Bailian

The Full Qwen-Audio-3.0 Suite Is Now Officially Available

On the afternoon of August 17, Alibaba Cloud announced that the Qwen-Audio-3.0 family of speech models had officially launched on the Qwen AI platform and Alibaba Cloud Bailian. Developers can now access the models via API or subscribe to a Token Plan.

This release is not a single model designed to do everything, but three independent product lines that can also be used together:

• Qwen-Audio-3.0-ASR: converts speech into text;
• Qwen-Audio-3.0-TTS: synthesizes speech from text;
• Qwen-Audio-3.0-Realtime: provides end-to-end real-time speech understanding and conversation.

Alibaba emphasized that Qwen-Audio-3.0 ranked first in all three categories—ASR, TTS, and real-time interaction—on the speech leaderboards published by Artificial Analysis this July. Leaderboard results cannot, of course, be directly equated with production performance. However, pushing speech input, speech output, and full-duplex conversation into the top tier all at once shows that this update is more than simply giving an old model a new version number.

More noteworthy is the product structure: Alibaba has not forced developers to move entirely to an end-to-end speech model. Instead, it has retained both the traditional pipeline and real-time model approaches.

Traditional pipeline: Audio → ASR → Text model/business system → TTS → Audio

End-to-end approach: Microphone audio ⇄ Qwen-Audio-3.0-Realtime ⇄ Real-time spoken response

The former is controllable, easy to audit, and allows any component to be replaced independently. The latter offers lower latency while preserving information such as intonation, emotion, and pauses that is easily lost in text. For developers, this is far more practical than simply releasing a larger speech model.

ASR: It Is About More Than Getting the Words Right

Qwen-Audio-3.0-ASR is designed for speech-to-text use cases and is currently available in three forms:

| Model Type | Primary Use | Suitable Scenarios | | --- | --- | --- | | ASR-Flash | Short-audio recognition, with a maximum duration of 5 minutes per segment | Voice input, short recordings, form completion | | ASR-Filetrans | Offline file transcription | Meeting recordings, recorded courses, podcast archiving | | ASR-Streaming | Real-time recognition that outputs text as the user speaks | Live captions, customer-service assistance, voice assistants |

The main selling points of this generation of ASR are contextual consistency, industry-specific term recognition, and customizable hotwords—not squeezing another fraction of a percentage point out of the error rate on Mandarin test sets.

This is a problem much closer to real-world business needs. General-purpose recognition models can usually understand a term such as “database,” but may misrecognize product names, drug names, stock abbreviations, and programming terminology as more common words with similar pronunciations. The more specialized the meeting, the higher the subsequent manual proofreading cost. Alibaba previously disclosed that ASR-Flash had expanded its specialized vocabulary for fields including healthcare, IT and programming, stocks, and public figures. In an internal evaluation of medical terminology, its “recognition hit rate” reached 95.36%; its published results also include a character error rate of 1.7%.

The model also adds speech polishing and text organization capabilities, allowing it to directly output content that is easier to read. Rather than merely transcribing a recording word for word, it attempts to eliminate several post-processing steps, including removing verbal fillers, restoring punctuation, and applying basic structure.

This capability is useful, but it should be enabled cautiously in production environments. Legal evidence, medical records, and quality-assurance recordings place greater value on verbatim fidelity. Automatically deleting fillers such as “um” and “uh” or rearranging sentences may alter the original meaning. Meeting minutes, course subtitles, and customer-service summaries, by contrast, are better suited to polished text. Developers should ideally retain both the “raw transcript” and a “readable version” rather than keeping only the model-edited result.

Bailian’s documentation shows that the publicly listed base price for the real-time Streaming version in the Beijing region is RMB 0.00033 per second, or approximately RMB 1.19 per hour. In the Singapore region, the price is RMB 0.00066 per second. These prices do not include business-side storage, networking, post-processing models, or concurrency costs, but they are already low enough to make the cost of long-running use cases such as live captions and agent assistance reasonably predictable.

TTS: From “Reading Aloud” to “Performing as Instructed”

Qwen-Audio-3.0-TTS handles text-to-speech synthesis, supports multiple languages and dialects, and can control emotion, tone, and pacing.

Traditional TTS is more like broadcast software: it pronounces the words correctly and sounds sufficiently natural, but a comforting statement and a warning may use the same rhythm. Large-model TTS instead aims to make speech follow semantics and instructions. For example, the same sentence, “I understand,” can convey confirmation, impatience, hesitation, or reassurance.

This will directly affect several types of products:

1. Intelligent customer service: adjust speaking speed and emotion for different situations such as complaints, collections, and after-sales support;
2. Content production: generate more natural narration for short videos, podcasts, and audio content;
3. Accessibility and digital humans: avoid maintaining the same intonation throughout long passages;
4. In-vehicle and hardware assistants: deliver reminders with shorter, clearer speech and less of a mechanical feel.

However, the “controllability” of TTS cannot be judged solely from polished demos. The metrics that determine whether it is ready for production also include time to first audio packet, long-text stability, mixed reading of numbers and English, voice consistency across segments, and whether streaming input can be joined smoothly. Developers must also handle cancellation when text has already been sent but the audio has not finished playing. Otherwise, an old response may continue coming out of the speaker even after the user interrupts.

If a business must use a proprietary branded voice or clone a specific speaker, a separate ASR, LLM, and TTS pipeline remains easier to control. The advantage of an end-to-end model is faster response, not the automatic replacement of every specialized speech-synthesis system.

Realtime: The Real Challenge Is Being Interruptible

Qwen-Audio-3.0-Realtime is the most noteworthy part of this update. It supports end-to-end speech understanding and conversation, can listen and speak simultaneously, allows users to interrupt and ask follow-up questions at any time, and supports tool calling.

Bailian currently lists two versions, qwen-audio-3.0-realtime-plus and qwen-audio-3.0-realtime-flash, which can be accessed via WebSocket and are deployed in both the Beijing and Singapore regions. Alibaba also provides a WebRTC-based Realtime API and an AOQ access method for client-side scenarios.

The hardest part of real-time voice conversation is not connecting ASR, a language model, and TTS. A demo can be built in a few hours by chaining three APIs together. The real difficulty lies in turn management: determining when the user has finished speaking, when the model should begin responding, what to do when both sides speak at once, whether a background “um” counts as a new question, and how to cancel already-generated audio after the user interrupts.

The capabilities provided by Qwen-Audio-3.0-Realtime are beginning to address these engineering details. In smart_turn mode, for example, the system can distinguish meaningful conversation from ambient sound. If it detects “um,” “uh,” or sounds without clear semantic meaning, it can pass them to the client through an ambient-sound transcription event without adding them to the conversation context or prematurely triggering a new response.

It also supports speaker enhancement: developers can provide a prerecorded sample of the target user’s voice, allowing the model to focus on that speaker in open environments while ignoring bystanders and background noise as much as possible. For in-vehicle assistants, shopping-mall guides, home devices, and open-plan offices, this matters more than laboratory accuracy measured in a quiet room.

Tool calling also allows the real-time model to do more than simply chat. A user can say, “Find out why my order still hasn’t shipped.” After understanding the request, the model can call the order system and return the result by voice. If the tool takes a long time to execute, the product must also provide waiting prompts, timeout handling, and secondary confirmation rather than leaving the model silent for several seconds while the query runs in the background.

Why Retain Three Separate Model Families?

Given that Realtime can apparently handle the entire process from listening to speaking, will ASR and TTS become transitional products? Not in the short term.

An end-to-end speech model is more like a responsive human agent: it can perceive tone and reply naturally, but its intermediate process may not be fully transparent. A traditional pipeline is more like splitting listening, understanding, and speaking into three separate roles, with every step recorded, replaceable, and auditable.

The following scenarios are better suited to independent components:

• Meeting transcription and recording archives, where only ASR is needed;
• Audiobooks and content voice-overs, where only TTS is needed;
• Financial, medical, and other systems that must retain step-by-step processing records;
• Customer-service systems that need to integrate an enterprise’s own large model, knowledge base, or moderation model;
• Businesses with strict requirements for voice cloning, text review, and speech-delivery rules.

The following scenarios are better suited to Realtime:

• In-vehicle assistants, earbud assistants, and smart hardware;
• Voice-based customer service that allows users to interrupt at any time;
• Coaching, spoken-language instruction, and role-based interaction;
• Applications that must respond to intonation and emotion;
• Products that are highly sensitive to first-token and first-audio latency.

By opening all three approaches at once, Alibaba is effectively acknowledging that no single speech AI architecture can yet cover every requirement. This product decision is sensible and more mature than merely showcasing a low-latency demo.

Five Things Developers Should Consider Before Integration

1. Realtime Is Not an Ordinary Chat API with a Different Input Format

Real-time speech depends on persistent connections, audio buffering, incremental events, and playback-state management. Even if the request fields resemble those of mainstream Realtime APIs, developers cannot assume that a standard OpenAI-compatible text endpoint can proxy them seamlessly.

For aggregation platforms, including OpenAI Hub, supporting a particular text model is entirely different from fully supporting WebSocket, WebRTC, audio tracks, and interruption events. Before integration, developers should verify protocol-layer capabilities rather than merely checking whether the model name appears in a list.

2. Audio Formats Can Create Many “False Model Problems”

Bailian’s examples use 16-bit, 16 kHz, mono PCM audio streams. When the sample rate, bit depth, channel count, or encoding does not match, the result is often not a clear error message but altered playback speed, noise, or a sudden drop in recognition accuracy. Mobile clients must also handle system echo cancellation, Bluetooth device switching, and microphone permissions.

3. API Keys Must Not Be Distributed to Clients

Real-time voice connections are typically established from phones, browsers, or hardware devices, but long-lived API keys should still be stored only on the business server. The client should obtain short-lived credentials before each connection, while the server handles authentication, quotas, and abuse prevention. Embedding a Bailian key directly in an APK or front-end code is practically equivalent to making the account balance public.

4. Region Affects Both Latency and Configuration

The Beijing and Singapore deployments use region-specific keys and endpoints. Real-time conversation is far more sensitive to network jitter than text generation, so the region should not be selected based on price alone. For users in mainland China, the Beijing node is usually more direct. For overseas users, actual network conditions, data compliance, and service coverage must all be tested.

5. First Place on a Leaderboard Does Not Mean First Place in Your Use Case

Artificial Analysis provides a standardized basis for comparison, but meeting-room reverberation, regional accents, far-field microphones, industry-specific terminology, and children’s speech can all change the results. Before launch, developers should create at least one proprietary test set and separately measure recognition accuracy, time to first packet, interruption success rate, tool-calling success rate, and persistent-connection stability.

What Alibaba Has Added Is a Production Interface for Speech Applications

The significance of Qwen-Audio-3.0 is not that “speech now has a large model too,” but that Alibaba has organized recognition, synthesis, and real-time interaction into API products that can be purchased separately or deployed together.

ASR handles specialized terminology and live captions, TTS provides multilingual support and expressive control, and Realtime attempts to overcome the latency bottleneck of the traditional three-stage pipeline. With all three now available, developers can choose a transparent and controllable pipeline based on their business needs or adopt a more natural, lower-latency end-to-end solution.

Our assessment is that this product suite is genuinely useful for customer-service, meeting, in-vehicle, and smart-hardware teams, but the engineering cost of Realtime will remain significantly higher than that of an ordinary text API. Model capability is only the first step. Connection management, echo cancellation, interruption strategies, cost control, and error recovery are what determine whether a voice demo can become a product that runs reliably every day.

Qwen-Audio-3.0 is already used in Alibaba products such as the Qwen app, Qwen Office, and Qoder. Its official launch on Bailian today means these capabilities are no longer limited to Alibaba’s internal applications and will now be tested against real traffic from external developers.

References

• ITHome: Qwen-Audio-3.0 Speech Models Launch on the Qwen AI Platform and Alibaba Cloud Bailian — Information on this product launch, the model lineup, and the Artificial Analysis rankings.
• ITHome: Alibaba Releases Qwen-Audio-3.0-ASR-Flash — Information on ASR variants, specialized-term recognition, healthcare evaluations, and character error rates.