Alibaba Fun-ASR-Realtime Upgrade: Handles 30 languages and 16 dialects with sub-100 ms latency

Alibaba Takes Another Crack at the Tough Problem of Dialects

On July 6, Alibaba Qwen released an upgraded version of Fun-ASR-Realtime, with the API going live directly on the Bailian platform. The core selling points come down to two numbers: first-token latency pushed into the sub-100ms range, and a single model supporting 30 languages plus 16 dialects. This upgrade is not a major architectural overhaul, but it significantly strengthens the engineering side—especially for dialects and Southeast Asian low-resource languages.

For teams building real-time speech systems, this is one of the rare ASR iterations in China this year that reaches the level of “can directly replace the production model.”

What Does “Sub-100ms” Actually Mean?

In the streaming ASR world, first-token latency is the metric that most directly reflects user experience. You speak into the microphone and say, “Help me check this,” and the moment the first word appears on the screen—that’s what this measures.

Over the past two years, mainstream cloud streaming ASR systems typically landed in the 300–500ms range, with better systems getting slightly above 200ms. Fun-ASR-Realtime has now entered the sub-100ms category, meaning the perceived delay from speech to text is almost “moving along with your mouth.”

In practical scenarios, the difference is very noticeable:

• Simultaneous interpretation apps: subtitles used to visibly lag behind speech; now they are essentially synchronized
• Voice input methods: typing responsiveness approaches local on-device recognition
• Real-time captions: livestreams and meetings no longer suffer from the awkward “one sentence spoken before captions appear”

More importantly, Alibaba claims the streaming recognition accuracy is “close to offline-level.” Technically, that is difficult to achieve—streaming models cannot see future audio context, so they are usually 2–5 percentage points less accurate than offline models. Closing that gap typically requires lookahead mechanisms and smarter decoding strategies, usually at the cost of latency. This time Alibaba is aiming for both. Whether it delivers will depend on real-world testing.

In Dialects, Volcano Engine and Tencent Have Been Overtaken

At this point in China’s ASR race, Mandarin recognition is already commoditized. The real differentiator is dialect support. Alibaba’s numbers show an average character accuracy rate of 88.62% across 16 dialects, with 12 dialect categories outperforming competing products from Volcano Engine and Tencent.

Why are dialects so difficult?

1. Data scarcity: Cantonese and Hokkien are relatively manageable, but high-quality annotated corpora for Hakka, Gan, and Xiang dialects are almost nonexistent
2. Strong colloquial characteristics: dialects are used almost entirely in spoken settings, with grammar structures that differ significantly from Mandarin; punctuation and sentence segmentation rules are completely different
3. Unique vocabulary: Shanghainese “侬” and Suzhou dialect “倷” sound similar but use different characters, requiring the model to learn extremely fine-grained distinctions

Starting from the early Fun-ASR 1.5 version, Alibaba’s team adopted a “unified large model + MoE expert routing” approach—detecting the dialect being spoken and activating the corresponding expert module. This is far more economical than “training one model per dialect,” and it is also the underlying reason the Realtime version can support seamless multilingual switching. Users do not need to tell the model “I’m about to speak Cantonese”; even mixed Chinese-English-Cantonese sentences can be recognized automatically.

For customer service and meeting transcription teams, this eliminates a large amount of upfront language detection logic and removes an entire layer from the engineering pipeline.

Dedicated Optimization for Southeast Asian Scenarios

One particularly noteworthy signal in this upgrade: Fun-ASR-Realtime specifically optimized recognition for Thai and other East/Southeast Asian multilingual scenarios, improving accuracy by 20%.

This was not a random direction. Over the past two years, Chinese companies expanding overseas have heavily targeted Southeast Asia. Customer service centers, e-commerce livestreaming, and cross-border meetings all require multilingual ASR. Yet speech recognition for Southeast Asian languages has consistently been a weak point—Thai and Vietnamese phonetic systems differ significantly from Indo-European languages, and mainstream open-source models (the Whisper family, for example) only achieve mediocre performance on them.

If that 20% improvement means going from 70% to 84%, that is the difference between “unusable” and “usable.” Alibaba is clearly targeting Chinese overseas businesses here while also competing against Google Cloud Speech and AWS Transcribe in the Southeast Asian market.

The Offline Version, Fun-ASR-Flash, Ranked #1 Globally

One more update: Fun-ASR-Realtime’s offline counterpart, Fun-ASR-Flash, has also recently launched. On the third-party evaluation platform Artificial Analysis, Fun-ASR-Flash (formerly called Fun-realtime-ASR-preview) achieved a character error rate of 1.7%, ranking first globally.

What does a 1.7% CER mean? In practical terms, for a 100-character audio clip, the model makes an average of only 1.7 mistakes. For comparison, Whisper Large achieves roughly 4–5% WER in English ASR, and Chinese ASR is generally more difficult than English. A 1.7% CER is already close to the level of professional human transcriptionists.

The offline and real-time versions share the same technical foundation, meaning the Realtime version’s accuracy ceiling will continue to benefit from improvements in Flash. That is also why Alibaba is confident in saying “streaming approaches offline-level quality.”

Model Positioning and Selection Recommendations

Alibaba Cloud’s Bailian platform now has a fairly broad ASR lineup, so it is worth clarifying where Fun-ASR-Realtime fits.

Current major ASR options on Bailian:

| Model | Scenario | Protocol | Language Support | Features | |------|------|------|----------|------| | fun-asr-realtime | Real-time | WebSocket | 30 languages + 16 dialects | Hotwords, low latency | | fun-asr | Non-real-time | HTTP | Multilingual and dialects | Speaker diarization | | fun-asr-flash | Non-real-time | HTTP | Multilingual and dialects | Lowest character error rate globally | | qwen3-asr-flash-realtime | Real-time | WebSocket | Multilingual | Emotion recognition | | qwen3.5-omni-plus-realtime | Real-time | WebSocket | Multilingual | Prompt context injection |

Selection recommendations:

• If your use case involves stable terminology dictionaries + real-time transcription (meetings, livestream subtitles, voice input), choose fun-asr-realtime. Its hotword mechanism is more token-efficient than prompts and has lower latency.
• If you need domain adaptation with dynamic terminology (such as medical conversations where terms appear unpredictably), use qwen3.5-omni-plus-realtime; simply describe the context in the prompt.
• If you need emotion recognition (customer service QA, mental health applications), use qwen3-asr-flash-realtime
• For purely offline batch transcription with maximum accuracy, choose fun-asr-flash

The division between the Fun series and the Qwen-ASR series is actually quite clear: the Fun series consists of dedicated ASR systems deeply optimized for speech recognition, placing them in the top tier for latency and accuracy; the Qwen-ASR series comes from multimodal foundation models, offering greater flexibility (prompt support, emotion analysis) but with higher latency and cost.

Several Engineering Observations

Architecturally, Fun-ASR-Realtime appears to follow an end-to-end streaming Transformer + MoE approach. Several engineering details are worth noting:

1. Streaming decoding trade-offs

Streaming ASR typically uses either chunk-based methods or Streaming Transformers. The former offers stable latency but sacrifices accuracy, while the latter can achieve near-offline accuracy but requires sophisticated KV cache management. Combining sub-100ms latency with near-offline accuracy strongly suggests the latter approach.

2. Applying MoE to ASR

MoE in speech is not new, but fitting 30 languages plus 16 dialects into a single model while maintaining low latency makes routing efficiency critical. Alibaba has not disclosed routing strategy details, but judging from inference speed, the activated parameter count is likely kept relatively restrained.

3. Seamless switching implementation

The ability to “not tell the model which language is being spoken” relies on implicit language identification plus dynamic routing. This is especially important for cross-border scenarios—an audio stream may mix Chinese, English, and Cantonese together. Traditional systems require segmentation and language detection first; now it can be handled in a single pass.

Who Should Pay Attention to This Upgrade

To put it plainly, several types of teams should test this immediately:

• Companies expanding overseas: the 20% accuracy improvement in Southeast Asian scenarios directly affects the usability of customer service and meeting transcription
• Businesses handling dialect-heavy workloads: for localized services in Guangdong-Hong Kong-Macao, Jiangsu-Zhejiang-Shanghai, and Southern Fujian, Fun-ASR-Realtime is currently one of the strongest domestic options
• Real-time interactive products: voice assistants, AI companions, and real-time translation are highly latency-sensitive; sub-100ms delays significantly improve user experience
• Teams planning migration: if you currently use Whisper or another cloud vendor’s ASR, both accuracy and latency are worth benchmarking

Alibaba’s progress in speech over the past two years has actually been very steady—from Paraformer to Fun-ASR, and now the dual-track Realtime + Flash strategy, the product lineup has become increasingly clear. Unlike the broader Qwen foundation model race, Alibaba has maintained a fairly obvious technical lead in speech, and this upgrade further solidifies its position at the top of China’s ASR market.

The remaining questions are pricing and real-world testing. The API is already live on Bailian, and developers can directly benchmark it against production workloads—because no benchmark result is as convincing as running your own audio through the system.

References

• Single Model Supports 30 Languages + 16 Dialects: Alibaba Qwen Upgrades Fun-ASR-Realtime Speech Recognition Model - IT Home: IT Home’s report on this upgrade, including dialect benchmark data and Southeast Asia optimization details
• Supports Recognition for 16 Dialects; AI Recording Hardware Pocket Raises $11 Million - Zhihu: Zhihu column analyzing the end-to-end architecture of the Fun-ASR series