Google Launches LiteRT.js: 3× Faster AI in the Browser, Is the TF.js Era Coming to an End?

On July 9, Google quietly released LiteRT.js in its developer blog, a new library built specifically for running AI inference in the browser. Its positioning is straightforward: replace the JavaScript kernel that TensorFlow.js has relied on for years. Google’s official numbers claim up to a 3× speed improvement over existing solutions on the 2024 MacBook Pro with the M4 chip.

At first glance this looks like a routine version update, but it’s actually a major overhaul of Google’s Web AI technology stack. TensorFlow.js has been around since 2018 and has effectively been the standard for browser-based machine learning, but developers have complained about its performance ceiling for years. LiteRT.js can basically be understood as Google tearing down that ceiling itself.

Why now? The TF.js JavaScript kernel can’t keep up anymore

First, some background. TensorFlow.js originally centered around running machine learning models in JavaScript while using WebGL for GPU acceleration. Back in 2018, that approach was cutting-edge—after all, running neural networks in the browser already felt miraculous. But there were two fundamental problems: JavaScript is still an interpreted language, and even with V8’s JIT compilation, there’s still a noticeable performance gap compared to native code; meanwhile, WebGL was essentially a graphics API awkwardly repurposed for computation, making matrix operations inefficient.

The result: slightly larger models such as Whisper or distilled versions of Stable Diffusion suffered from painfully high latency in the browser. Developers either abandoned client-side inference in favor of server-side deployment, or manually optimized critical operators with WebAssembly—a lot of extra work.

LiteRT.js directly addresses all three bottlenecks through its technical stack:

• WebAssembly as the core runtime: a binary instruction format that runs close to native speed, replacing the interpreted JavaScript kernel
• WebGPU for GPU acceleration: a modern GPU compute API designed for both general-purpose computing and graphics rendering, far more efficient than WebGL
• WebNN as the hardware backend: directly interfaces with operating system neural accelerators such as Apple Neural Engine and NPUs

Together, those three layers are what enable the reported 3× acceleration on M4 hardware.

Where does the 3× gain come from? Looking at Google’s benchmark details

Google’s own blog post is fairly cautious—the 3× figure was measured on an M4 MacBook Pro, and actual performance on older hardware or different browser engines “may vary.” Translated into plain language: you’ll likely only see that number on newer hardware with Chrome, while Safari and Firefox will probably lag behind.

Even so, the direction is clearly right. That’s because LiteRT.js runs .tflite models underneath—the mobile model format used by LiteRT (formerly TensorFlow Lite) for years, with a very mature ecosystem. In theory, any model that already runs on mobile can now run directly in the browser without additional format conversion or operator adaptation.

That point is arguably more important than the 3× speedup itself. One of the most painful parts of Web AI development used to be model conversion: taking a PyTorch model, converting it to ONNX, then to TF SavedModel, then finally to TF.js format, while dealing with incompatible operators at every stage. LiteRT.js reuses LiteRT’s existing conversion pipeline, effectively eliminating many of the pitfalls the mobile ecosystem has already solved.

Compared to ONNX Runtime Web, which one is stronger?

This is the question developers actually care about. Today’s major players in browser-based AI inference are roughly:

1. TensorFlow.js: the veteran option with the largest ecosystem, but weaker performance
2. ONNX Runtime Web: Microsoft-backed, strong cross-framework support, solid performance
3. Transformers.js: built by Hugging Face on top of ONNX Runtime Web, focused on LLM scenarios
4. LiteRT.js: the newcomer, backed by Google, aiming for near-native performance

Architecturally, LiteRT.js and ONNX Runtime Web are actually very similar—both rely on the WASM + WebGPU combination. The main difference lies in ecosystem alignment. LiteRT.js naturally integrates with TFLite/JAX/Keras models, while ONNX Runtime Web targets broader framework interoperability.

For developers, the decision logic is fairly simple: if you already use the TensorFlow ecosystem or work on mobile ML, LiteRT.js is the obvious choice; if you rely on PyTorch or need maximum cross-framework flexibility, ONNX Runtime Web is still the better option.

Who will actually use it? Three typical scenarios

Browser-based AI inference has long been one of those technologies that “looks promising,” but despite years of discussion, relatively few products have truly landed. The core reason has always been performance. Users can tolerate a one-second response time; at five seconds, they close the tab.

LiteRT.js lowers that latency threshold enough to unlock scenarios that were previously impractical:

• Real-time video processing: background blur, beauty filters, gesture recognition—products like Google Meet already use these features
• On-device LLMs: smaller models in the 1B–3B range can run directly in the browser, crucial for privacy-sensitive use cases
• Interactive ML education: platforms like Kaggle and Colab can deliver smoother demos and tutorials

The on-device LLM angle is especially interesting. Chrome’s built-in Gemini Nano follows a browser API model that users can’t control, whereas LiteRT.js gives developers full control—you decide which models to run, and quantized LLMs in .tflite format are becoming increasingly common.

A subtle but important point: this time Google got the timing right

To be fair, Google has always seemed somewhat indecisive about Web AI. deeplearn.js (2017) → TensorFlow.js (2018) → MediaPipe (2019) → now LiteRT.js: every few years there’s another reinvention, and developers end up paying the migration cost.

This time, though, LiteRT.js has a much clearer role. It’s not trying to replace the entire TensorFlow.js ecosystem—it’s replacing the inference kernel at the very bottom of the stack. Google also appears to be prioritizing API compatibility at the upper layers, so migration costs should remain manageable. Combined with the fact that WebGPU support has finally stabilized across major browsers this year, the timing feels much more appropriate.

Looking at the broader trend, on-device AI—whether on phones or in browsers—will likely become a major growth direction by 2026. Cloud-hosted large models are increasingly expensive, slow, and weak on privacy. More and more workloads that can run locally will move to the client side. LiteRT.js is an important missing piece for the Web platform in that shift.

For developers who still need server-side access to mainstream AI models, OpenAI Hub (openai-hub.com) remains a practical domestic solution that provides direct access to GPT, Claude, Gemini, and DeepSeek with a single API key. But in this discussion, the real focus is browser inference that removes the need for a server entirely.

References

• ITHome: Google releases LiteRT.js library, boosting Web AI inference performance by up to 3× - First domestic media report covering key information from Google’s official blog and architecture screenshots