<think>**Translating headline phrase**</think> WeChat’s Open-Source Multimodal Embedding Model Tops the Rankings

<think>Preparing technical translation with preserved markdown</think>

WeChat Open-Sources Its Multimodal Embedding Model, Which Handles One Billion Calls a Day

On September 4, WeChat AI officially announced the open-sourcing of its general-purpose multimodal embedding model, WeMM-Embedding. This series is not designed for chat or content generation. Instead, it converts text, images, videos, and visual documents into vector representations that can be retrieved and compared.

WeMM-Embedding is available in 2B, 4B, and 9B versions. It supports text, images, videos, visual documents, and arbitrarily interleaved multimodal inputs. According to information disclosed by the WeChat team, the model series has already been deployed at scale in WeChat’s recommendation and search systems, covering services such as Channels, livestreaming, Official Accounts, e-commerce, and Moments, with daily usage reaching the billion-call range.

On the multimodal embedding benchmark MMEB-v2, WeMM-Embedding-9B scored 80.6, ranking first on the public leaderboard. The 2B version scored 77.9, surpassing the previously leading 8B open-source model. For embedding models, this result is more significant than simply taking first place on a leaderboard: it indicates that the WeChat team is converting the cross-modal data and engineering experience accumulated in industrial recommendation systems into general-purpose representation models that are smaller and easier to deploy.

It Does Not Just “Understand Images”—It Makes Different Types of Content Mutually Retrievable

The core task of multimodal embedding can be simply understood as placing different types of content into the same vector space.

A piece of text, a product image, a video, subtitles from a video, and a scanned document are all converted into vectors after being encoded by the model. The more relevant two pieces of content are, the closer their vectors will generally be; the less relevant they are, the farther apart they will be. In this way, search systems do not need to build completely separate retrieval pipelines for text, images, and videos.

For example, when a user enters “a natural wood-style living room suitable for a small apartment” in WeChat Search, the system can use the text vector to retrieve Official Account articles, Channels content, product images, and even livestream clips. When a user uploads a furniture photo, the system can also search for related content in reverse. A recommendation system can encode videos watched by a user, articles and images they have saved, products they have clicked, and search terms into the same space for candidate retrieval, ranking, and user-interest modeling.

This is also what distinguishes WeMM-Embedding from ordinary text vector models. Traditional text embeddings mainly address whether “one sentence is related to another,” whereas WeMM-Embedding must handle whether “text is related to an image,” whether “a video is related to a product description,” and whether “a visually formatted document containing both text and images is related to a search query.” It is not dealing with a single input type, but with a collection of content formats and business scenarios.

According to WeChat, WeMM-Embedding uses a unified backbone based on the Qwen3.5 multimodal architecture, allowing images, videos, and text to be processed jointly from the outset, rather than having separate models extract features first and then forcibly aligning them later. This choice is crucial. A late-concatenation approach is easier to build quickly, but fine-grained relationships across modalities are often lost along the way. A unified multimodal backbone, by contrast, offers greater potential for understanding the shared semantics among text, visual content, layout, and temporal order.

The 9B Model Takes the Top Spot, but the 2B Model Is the Real Highlight of This Release

Judging from the leaderboard results, the 9B version is the strongest: it achieved an overall MMEB-v2 score of 80.6, with scores of 81.9, 74.3, and 83.3 in the image, video, and visual-document categories, respectively.

For most developers and enterprises, however, the 2B version may be more practically attractive than the 9B model. WeMM-Embedding-2B scored 77.9 on MMEB-v2, significantly outperforming models of comparable size and even surpassing the previous 8B open-source baseline, Qwen3-VL-Embedding-8B, which scored 77.8. In other words, the model’s parameter count has been reduced without a proportional loss in performance.

This matters greatly for online systems. Embedding services are typically not invoked just once; they are called frequently across multiple stages, including retrieval, reranking, user-sequence updates, and offline index construction. Reducing a model from 8B to 2B affects not only the cost of individual inference calls, but also GPU memory usage, concurrency, batch throughput, and the number of machines required for deployment. For systems processing hundreds of millions or even larger volumes of content every day, the cost-effectiveness of a 2B model is often more important than the 9B model’s absolute score.

Internal evaluations disclosed by WeChat support this point. Across 26 internal tasks, WeMM-Embedding-2B raised the average score from 60.9 to 72.0 compared with a baseline of similar size. Improvements were observed in all five task categories: classification, search, cross-domain matching, article relevance, and video relevance.

Of course, leaderboard performance cannot be directly equated with performance in every business scenario. MMEB-v2 mainly covers 78 tasks in image, video, and visual-document domains, using different metrics to measure cross-modal retrieval capabilities. Online systems must also contend with data-distribution differences, content freshness, Chinese-language semantics, long-tail products, shifts in user preferences, and latency constraints. The fact that WeMM-Embedding has already been deployed at scale in WeChat’s businesses says more about its validation under real-world traffic than any public leaderboard, but developers still need to conduct offline evaluations and A/B tests on their own data.

From “Large-Scale Alignment” to “Business Relevance,” Its Training Approach Is Closer to Recommendation Systems

According to the technical report and publicly available materials, WeMM-Embedding uses a two-stage training process.

The first stage consists of large-scale multimodal alignment, with the goal of bringing content from different modalities into a relatively unified semantic space. The training data reaches the hundreds-of-millions scale, and the model must learn the basic correspondences among text, images, videos, and visual documents.

The second stage is more closely aligned with optimization for industrial retrieval and recommendation. It uses curated data, fine-grained relevance annotations, reranking supervision, and cross-scale knowledge distillation to further refine the model. In other words, the model is not trained merely to determine whether content is “relevant” or “irrelevant”; it must also learn “which content is more relevant” and “how much more relevant it is.”

This distinction is highly important in recommendation and search systems. For a user searching for “low-sugar breakfast,” a video that clearly explains a low-sugar recipe is generally more relevant than an image containing only the word “breakfast.” A long article that mentions a low-sugar diet but provides no specific recipe may rank below both. Binary labels can only tell the model what is right and wrong, whereas graded relevance allows it to learn ranking relationships.

WeMM-Embedding also supports nested representation learning, also known as Matryoshka Representation Learning (MRL). This allows the same model to output vectors of different dimensions while preserving as much semantic capability as possible and reducing vector storage and retrieval costs.

For vector databases, dimensionality is directly related to index size and retrieval efficiency. Taking a 4,096-dimensional vector as an example, each vector requires approximately 16 KB of raw storage when represented using 4-byte floating-point numbers. With one billion pieces of content, the raw vectors alone create enormous storage pressure. If a business does not require the highest possible precision, it can use shorter nested vectors for first-stage candidate retrieval, followed by full-dimensional vectors or a reranking model for precise ranking. This “low-dimensional coarse retrieval, high-dimensional fine ranking” approach is better suited to large-scale online systems.

WeChat’s Advantage Is That It Knows Which Types of “Relevance” Affect Clicks

A common problem with open-source models is that they perform well on public datasets, but their retrieval results may not align with user behavior after being integrated into real businesses. Search systems need more than semantic similarity; they also need to account for click intent, content quality, freshness, domain preferences, and platform ecosystems.

WeChat’s advantage lies precisely in its diverse content formats and high-frequency interaction scenarios. Channels, Official Accounts, Moments, livestreaming, and e-commerce content have complex cross-domain relationships. The same user may discover an article through a search query, click on a video, and eventually arrive at a product page. A model trained only on a single dataset to learn “text paired with images” would struggle to cover this kind of real-world content flow.

The WeChat team stated that the multimodal representations generated by WeMM-Embedding are already used in the following areas:

• Candidate content retrieval: Quickly finding semantically relevant candidates from massive collections of videos, articles, images, and products.
• Ranking feature construction: Providing cross-modal relevance features for downstream ranking models.
• User-sequence modeling: Representing different types of content that users have browsed, searched for, clicked on, or interacted with in a unified manner.
• Cross-domain content understanding: Connecting different content pools, such as Channels, Official Accounts, and e-commerce, to support cross-scenario matching.

This deployment path is worth considering for developers. An embedding model is typically not the entirety of a recommendation system; it is more like underlying “semantic infrastructure.” It converts content and user behavior into unified, computable representations, while the upper layers add rules, ranking models, business features, and real-time feedback.

But It Is Not a “Plug-and-Play” Universal Vector Model

The open-source value of WeMM-Embedding is clear: developers can access the complete code, model weights, and evaluation tools for multimodal search, recommendation, classification, deduplication, and cross-modal matching. However, it still has several practical limitations.

First, model size does not directly equal serving cost. The 2B model is better suited to high-concurrency online scenarios, while the 9B model is more appropriate for offline index construction where accuracy is prioritized, complex document retrieval, or high-quality encoding before reranking. Actual deployment must also account for video frame sampling, document-image resolution, batch size, quantization method, and vector-database index configuration.

Second, embedding performance depends heavily on the input protocol. Whether text queries and document content require different templates, whether a video is represented by keyframes or a complete frame sequence, and whether visual documents retain layout information can all affect the final vector quality. Developers cannot simply replace the model and expect search results to improve automatically.

Third, leaderboard coverage remains limited. Supplementary evaluations show that MMEB-v3 has expanded to 190 tasks, including text, agents, audio, and other more complex retrieval scenarios. However, WeMM-Embedding’s core strengths remain concentrated in images, videos, and visual documents; audio is not currently one of its primary capabilities. Systems that require speech, multi-turn tool use, or strong text-reasoning capabilities will still need to combine it with other models.

Fourth, developers need to carefully verify the model’s license, commercial-use scope, inference-framework compatibility, and GPU-memory requirements before putting it into production. In particular, projects with existing vector databases generally cannot directly reuse their old vectors after switching embedding models. They often need to re-encode the entire corpus, rebuild the index, and recalibrate similarity thresholds.

What It Means for Developers

The most direct impact of WeMM-Embedding’s open-source release is that the barrier to entry for multimodal retrieval models has been lowered further. In the past, teams wanting to build systems for “text-to-video search,” “image-to-product search,” or question answering over documents with mixed text and images typically had to combine a visual encoder, a text model, a cross-modal projection layer, and business-specific training data. Now they can start with a unified multimodal embedding model and adapt it to their specific needs.

Practical deployment approaches include:

1. Use 2B for online retrieval: Handle high-concurrency candidate generation while prioritizing latency and cost control.
2. Use 4B for medium-scale business retrieval: Strike a balance between performance and resource consumption for enterprise knowledge bases, content platforms, and e-commerce search.
3. Use 9B for offline encoding or high-quality reranking: Build high-quality indexes, analyze hard examples, and validate the upper bound of achievable performance.
4. Use MRL for hierarchical indexing: Use low-dimensional vectors for coarse filtering and high-dimensional vectors for fine ranking to reduce overall storage and computation costs.
5. Combine it with business-supervised fine-tuning: Convert clicks, dwell time, saves, conversions, and human relevance annotations into training signals that are more closely aligned with business objectives.

More importantly, what WeChat has open-sourced this time is a “representation model,” not a complete search product. The factors that truly determine system performance still include data cleaning, chunking strategies, index updates, retrieval fusion, reranking, caching, and evaluation loops. The model can place text and images in the same space, but it does not automatically know why your users click, nor can it solve the freshness problem for real-time content on your behalf.

Conclusion

Judging from the results, WeMM-Embedding is a substantial recent update among open-source multimodal embedding models: the 9B version ranks first on MMEB-v2, while the 2B version surpasses a previous 8B baseline at a much smaller scale. The model series has also already handled one billion daily calls in WeChat’s recommendation and search systems.

Its true competitiveness lies not merely in its score of 80.6, but in bringing together unified multimodal inputs, relatively small model sizes, industrial-scale validation, and complete open-source availability. For teams building multimodal search, recommendation, or knowledge-base systems, the 2B and 4B versions are particularly worth testing through an offline replacement experiment. For teams pursuing the highest possible performance, the 9B model can serve as an important baseline among currently available public models.

What remains to be seen is how it performs on more Chinese business data, long-video retrieval, complex visual documents, and agent memory retrieval. Ranking first on a leaderboard is only the starting point. The ultimate proof of value for this type of embedding model will be whether it can consistently reduce retrieval costs and improve clicks and conversions on the real-world data of different teams.

References

• ITHome: WeChat Open-Sources the General-Purpose Multimodal Embedding Model WeMM-Embedding — Covers the open-source announcement, model sizes, deployment in WeChat businesses, and MMEB-v2 results.
• WeMM-Embedding GitHub Repository — Entry point for the official code, evaluation tools, and usage instructions.
• WeMM-Embedding Technical Report on Hugging Face — Provides information on the model architecture, training process, and public evaluations.
• Zhihu Column: 2B/4B/9B Multimodal Vector Models, with 9B Ranking First on MMEB-v2 — Summarizes information on MMEB-v2, MMEB-v3, and the model versions.