Run Hybrid Retrieval in the Browser: Semantic Search Without a Backend

Over the past two years, semantic search has basically been tied to the server side — Pinecone, Weaviate, pgvector, pick any one of them and you still need to spin up a server and maintain an embedding pipeline. But recently, developer Bart de Goede published a fully browser-based hybrid retrieval solution on his blog, stuffing BM25, vector indexing, and embedding inference entirely into WebAssembly, end-to-end without touching a backend.

This is not a toy demo. He ran a full evaluation on a corpus of roughly 5,000 documents: cold start loading for both model and index took under 2 seconds, and single-query P95 latency stayed below 40ms on a standard MacBook Air. For developers building documentation sites, offline note-taking apps, or privacy-sensitive applications, this approach is worth serious attention.

Why Do This in the Browser?

First, let’s clarify the motivation. Moving retrieval into the browser sounds, at first glance, like making life unnecessarily difficult — servers have plenty of CPU and memory, so why fight against the constraints of the WebAssembly sandbox?

Three reasons:

First, the cost advantage of static hosting. A documentation site built with Docusaurus or VitePress can normally run entirely on Cloudflare Pages. The moment you add semantic search, you suddenly need Serverless Functions and an embedding service, multiplying both cost and operational complexity by an order of magnitude. If the entire retrieval pipeline can fit into the bundle, it remains a purely static site.

Second, privacy. What users search for and click on never leaves the browser. In legal, medical, or enterprise intranet documentation scenarios, this is not a nice-to-have — it’s a hard requirement. Even if you self-host pgvector, queries still have to hit a server.

Third, offline availability. Apps running inside PWA, Electron, or Tauri shells can continue searching while offline, which creates a noticeable UX advantage. Local-first note-taking tools like Obsidian and Logseq have already been pushing in this direction for years, though historically they mostly relied on keyword-only retrieval.

What Exactly Is “Hybrid” Retrieval Mixing?

A quick refresher before getting into the engineering details.

So-called hybrid retrieval combines two retrieval signals:

• Sparse retrieval: represented by BM25, essentially weighted keyword matching. It is extremely sensitive to exact terms, proper nouns, and code symbols. Search for useEffect, and it finds useEffect; it won’t return philosophical essays about “side effects.”
• Dense retrieval: converts both queries and documents into vectors, then uses cosine similarity to find semantically related content. Search for “how to avoid unnecessary component re-renders,” and it may return articles discussing memo, useMemo, or shouldComponentUpdate, even if the phrase “re-render” never appears.

Each has weaknesses. BM25 struggles with synonyms, hierarchical concepts, and conversational queries; vector retrieval often fails at exact matching. Search for an error code like E_INVAL_ARG, and it may return generic articles about “parameter validation” instead of the actual page containing that exact error.

Hybrid retrieval runs both approaches in parallel, then fuses the ranking lists. The two mainstream fusion methods in industry today are:

1. Weighted score fusion: normalize BM25 scores and vector similarities into the same scale, then combine them using weighted addition. Meilisearch’s semanticRatio follows this approach — 0 means pure keyword search, 1 means pure semantic search, with tunable values in between.
2. RRF (Reciprocal Rank Fusion): ignores raw scores and only considers ranking positions: score = Σ 1/(k + rank_i) where k is typically 60. The advantage is that you never have to reconcile absolute score distributions between the two systems; documents ranking highly in either branch score well overall.

Bart’s implementation uses RRF for a practical reason: browser-side score distributions are inherently unstable, making normalization error-prone, while RRF remains robust.

Tech Stack Breakdown

The core components of the solution are:

| Layer | Choice | Notes | |------|------|------| | BM25 Index | Custom WASM implementation | Inverted index, compressed to ~200KB | | Vector Index | HNSW (hnswlib-wasm) | Approximate nearest neighbor search with logarithmic retrieval speed | | Embedding Model | all-MiniLM-L6-v2 (quantized ONNX version) | 22MB, 384 dimensions | | Inference Engine | Transformers.js + ONNX Runtime Web | Supports WebGPU acceleration | | Fusion Strategy | RRF | k=60 |

Several points are worth discussing separately.

Choosing the Embedding Model

all-MiniLM-L6-v2 is an old veteran, but it remains one of the best choices for browser scenarios. Not because it has the best quality — bge-small and gte-small now outperform it on MTEB benchmarks — but because it is small, fast after quantization, and widely supported across the ecosystem.

Bart tested bge-small-en-v1.5 (roughly 33MB after quantization). Retrieval quality improved slightly, but initial loading time increased from 1.2s to 2.1s, which was not worth the tradeoff for web scenarios. The 384-dimensional embedding size also hits a sweet spot for both index size and retrieval speed.

For Chinese corpora, you could swap in bge-small-zh-v1.5 or text2vec-base-chinese; the overall architecture remains the same.

The Reality of HNSW in the Browser

HNSW is currently the most popular ANN algorithm. pgvector, Qdrant, and Weaviate all use it. The core idea is a hierarchical graph structure where upper layers are progressively sparser, and search proceeds greedily from top to bottom.

The problem is that HNSW indexes are not small. For 5,000 documents with 384-dimensional vectors and M=16 (connections per node), the index alone consumes several megabytes of memory. Bart’s solution is to serialize the index and ship it with the bundle, using the browser only for querying, not construction. Index building happens in CI, and the resulting artifacts are committed directly into the repository.

This workflow naturally aligns with static site generators (SSGs): generate indexes during build time, distribute them as static assets during deployment, and only load them at runtime.

The WebGPU Boost

After Chrome fully enabled WebGPU in stable releases during the second half of 2024, inference speeds in Transformers.js improved dramatically. Bart measured encoding a sentence with the same MiniLM model at roughly 45ms on the WASM backend versus only 8ms using WebGPU on an M2 chip.

WebGPU still requires flags in Safari, and Firefox only recently reached GA support. In production, feature detection is recommended: use WebGPU when available and fall back to WASM otherwise.

Engineering Pitfalls

The blog’s comment section reveals plenty of firsthand implementation details, confirming several pain points many developers have already encountered.

Cold start is the biggest enemy. On first visit, users must download the model (22MB), download the index (several to dozens of MB), initialize ONNX Runtime, and load HNSW. Any blocking on the main thread makes the page feel frozen. Bart moved everything into a Web Worker: the main thread handles only UI, the search box shows a loading state until initialization finishes, then seamlessly switches over.

What about index updates? Documentation sites may modify dozens or hundreds of pages per release. Forcing users to re-download tens of megabytes of indexes every time creates a terrible experience. Bart shards the indexes and uses content hashes as filenames, combined with HTTP caching so only modified shards are re-downloaded. This is essentially the same strategy as Webpack chunk hashing.

Chinese tokenization. BM25 depends on tokenization. English can split on spaces; Chinese requires tools like jieba. WASM versions of jieba exist, but they are fairly large (roughly a 3MB dictionary), so Chinese implementations involve additional tradeoffs. One compromise is bigram segmentation without dictionaries, which performs better than expected.

Memory usage. Once loaded, HNSW indexes remain resident in memory. For 5,000 documents, the index alone occupies roughly 20–40MB. Combined with model weights and runtime overhead, peak memory usage for the search module may reach 100MB. That is fine on desktop browsers, but iOS Safari’s memory limits require caution on mobile devices.

When Should You Use This Approach?

To be clear: this is not a universal solution.

Good fit for:

• Documentation sites, blogs, and knowledge bases with corpora ranging from thousands to tens of thousands of entries
• Privacy-sensitive or offline-capable applications
• Static-hosted projects looking to avoid backend costs
• Scenarios with relatively infrequent content updates where build-time indexing is acceptable

Poor fit for:

• Corpora with millions of entries — distributing indexes becomes a disaster
• Real-time content updates — rebuilding indexes for every change is impractical
• Multilingual or highly filtered search scenarios — server-side systems offer far more tuning flexibility
• Environments with weak client hardware — embedding inference on low-end phones causes noticeable lag

For most developers building documentation, note-taking, or content-focused applications, this sweet spot is actually fairly broad. Compared to continuing with Algolia (expensive, and requires sending your content elsewhere) or relying solely on keyword search (poor UX), browser-side hybrid retrieval represents a genuinely meaningful third option.

A Minimal Implementation You Can Copy Directly

Based on Bart’s repository structure, the core logic of a minimally viable implementation looks roughly like this:

import { pipeline } from '@xenova/transformers';
import { HierarchicalNSW } from 'hnswlib-wasm';
import BM25 from './bm25-wasm';

// Initialization (inside Worker)
const embedder = await pipeline(
  'feature-extraction',
  'Xenova/all-MiniLM-L6-v2',
  { quantized: true, device: 'webgpu' }
);

const hnsw = new HierarchicalNSW('cosine', 384);
await hnsw.readIndex('/search/vectors.bin');

const bm25 = await BM25.load('/search/bm25.bin');

// Query
async function hybridSearch(query, topK = 10) {
  // Run both retrieval paths in parallel
  const [denseResults, sparseResults] = await Promise.all([
    (async () => {
      const emb = await embedder(query, { pooling: 'mean', normalize: true });
      return hnsw.searchKnn(emb.data, topK * 2);
    })(),
    bm25.search(query, topK * 2),
  ]);

  // RRF fusion
  const k = 60;
  const scores = new Map();
  denseResults.neighbors.forEach((id, rank) => {
    scores.set(id, (scores.get(id) || 0) + 1 / (k + rank));
  });
  sparseResults.forEach(({ id }, rank) => {
    scores.set(id, (scores.get(id) || 0) + 1 / (k + rank));
  });

  return [...scores.entries()]
    .sort((a, b) => b[1] - a[1])
    .slice(0, topK)
    .map(([id, score]) => ({ id, score }));
}

Real-world projects still need to handle tokenization, highlighting, field weighting, deduplication, and other details, but the core structure is genuinely this straightforward. Bart has open-sourced the full implementation, and it is well worth studying directly.

A Broader Perspective

Browser-side AI has progressed much faster over the past year and a half than many people expected. Transformers.js has evolved from a toy into something production-capable, WebGPU has matured into stable deployment, and ONNX Runtime Web support continues improving. With these foundational pieces finally in place, many scenarios that previously required backend infrastructure now have viable alternatives.

Hybrid retrieval is simply one of the first to become practical. Looking ahead, browser-side rerankers, small-model RAG systems, and even parts of agent workflows are all foreseeable. The significance for developers is not merely eliminating a server — it changes the possible product experience itself. An app that can write code, search notes, and answer questions while offline on an airplane is fundamentally different from one that must call APIs over the network.

In the near term, server-side solutions will remain the default choice for large-scale hybrid retrieval systems. But if you are building documentation tools, note-taking apps, or personal knowledge bases, it is worth seriously considering implementing this approach yourself. Two weeks of engineering effort can produce tangible improvements in both product experience and operational simplicity.

References

• OceanBase Hybrid Retrieval Multimodal Search Practical Guide (Zhihu) — Discusses industrial-grade implementations combining vector, sparse, and full-text retrieval, useful for comparison with browser-side approaches.