Google Releases LiteRT.js: A JavaScript Binding for LiteRT Using .tflite Models in Browsers with WebGPU

Google is released LiteRT.js, JavaScript binding for LiteRT. LiteRT is Google’s cross-platform library, formerly known as TensorFlow Lite.

LiteRT.js works .tflite models directly within the browser. Because the thinking is always local, Google cites improved user privacy, zero server costs, and very low latency.

What is LiteRT.js?

It is not a new model format. Instead, Google integrated its existing native runtime into WebAssembly and exposed it to JavaScript.

Previous web AI solutions, including TensorFlow.js, relied on JavaScript-based kernels. Google describes those as slow. LiteRT.js instead ships a native cross-platform runtime with its own fixed configuration.

Thus, web applications inherit work done elsewhere. Performance improvements, benchmarking improvements, and hardware improvements for Android, iOS, and desktop are also coming to the web.

How It Works: One runtime, three backends

Under that operating time, LiteRT.js supports three backends:

• CPU uses XNNPACKGoogle’s advanced CPU library, with multi-threading support and free SIMD architecture.
• The GPU uses ML DriftGoogle’s on-device GPU solution, continues WebGPU.
• NPU uses i WebNN APIcurrently being tested on Chrome and Edge.

Two related laws govern shipping. First, LiteRT.js does not support component deployment. The graph cannot be separated across CPU and GPU.

Second, the transmission is empty or empty for each model. If the model cannot be fully exported to the selected accelerator, LiteRT falls back to it wasm to be killed. The CPU router has the widest range of functions.

Working

Given those basics, the Google team reports two different results.

Against other web runtimes, LiteRT.js is about to arrive 3x fast for every CPU and GPU index. That figure includes classical computer vision and sound processing models.

Against its CPU performance, GPU or NPU delivers 5–60x speed. That applies to real-time job searches like object tracking and audio transcription.

Both benchmarks run in a controlled browser environment on a 2024 MacBook Pro with M4 Apple Silicon. Google notes results vary by local GPU, thermal throttling, and driver optimization. The “10x” figure floating around the launch doesn’t appear in the announcement.

Finding the PyTorch In model

Litter Torch converts PyTorch models into .tflite in one step.

However, the requirements are strict. Your model should be shipped with torch.export.exportwhich means TorchDynamo-exports. It cannot contain Python conditional branches that depend on runtime tensor values. It also cannot have variable input or output sizes, including heap sizes.

In size, AI Edge Quantizer configures quantization schemes for all layers of different models. You are pre-trained .tflite models are also available on Kaggle and the LiteRT Hugging Face Community.

A small pipe

Once converted, the runtime code is short. This is the WebGPU method, validated against it @litertjs/core v2.5.2:

import {loadLiteRt, loadAndCompile, Tensor} from '@litertjs/core';

// Wasm files live in node_modules/@litertjs/core/wasm/ or on a CDN.
await loadLiteRt('path/to/wasm/directory/');

const model = await loadAndCompile('path/to/model.tflite', {
  accelerator: 'webgpu', // 'wasm' | 'webgpu' | 'webnn'
});

const input = new Tensor(new Float32Array(1 * 3 * 224 * 224), [1, 3, 224, 224]);
const results = await model.run(input);

// Accelerator results live off-heap. Move to CPU, then convert.
const cpuTensor = await results[0].moveTo('wasm');
const output = cpuTensor.toTypedArray();

// LiteRT.js uses manual memory management. Delete every tensor.
input.delete();
for (const t of results) t.delete();
cpuTensor.delete();

That last block needs attention. LiteRT.js does not garbage-collecting tensor. Always Tensor must be removed explicitly, or the application leaks device memory. The caption in Google’s announcement post omits this step.

WebNN requires one additional flag. LiteRT.js requires JSPI, which includes parallel kernel programming and parallel device polling:

await loadLiteRt('path/to/wasm/', {jspi: true});

const model = await loadAndCompile('model.tflite', {
  accelerator: 'webnn',
  webNNOptions: {devicePreference: 'npu'}, // 'cpu' | 'gpu' | 'npu'
});

Before writing preprocessing, check for false positives. Run npm i @litertjs/model-testerthen npx model-tester. Runs your model on WebNN, WebGPU, and CPU using random input. Use it model.getInputDetails() learning input words and shapes.

Use Cases with examples

Those APIs return four demos that Google shipped at launch:

LiteRT.js vs TensorFlow.js

Those demos raise an obvious question for existing ML web groups.

The important thing is that the two are not mutually exclusive. Google replaces TF.js with LiteRT.js Graph Models specifically, not the entire library.

TensorFlow.js remains the recommended tool for pre- and post-processing. I @litertjs/tfjs-interop the packet passes tensors between them with runWithTfjsTensors. Avoid tensor.dataSyncwhich carries a significant penalty on the WebGPU backend.

Interactive Descriptor

The lower embedment develops six sections of pipes across each rear area.