LiteRT
An open-source on-device AI framework runs models on mobile, desktop and web, with CPU, GPU and NPU acceleration and offline LLM support.
LiteRT is Google's framework for developers adding machine learning and generative AI to apps that run on users' devices. It succeeds TensorFlow Lite and supports local inference without a server round trip.
- Local LLMs: LiteRT-LM supports models such as Gemma, with conversation handling, streaming responses, multimodal input and tool calling.
- Model conversion: Convert PyTorch, TensorFlow and JAX models to .tflite and apply post-training quantization, or use preconverted models including Gemma, Llama 3.2, Qwen 3.5 and MobileNet.
- Vision: Run camera image segmentation on-device, or use FastVLM for vision-language tasks with NPU acceleration on Qualcomm devices.
- Speech recognition: Transcribe audio locally with Whisper, Moonshine or Parakeet using CPU, GPU or NPU acceleration.
- Offline speech synthesis: Generate speech without an internet connection and begin playback after the first sentence while the remaining output continues to generate.
LiteRT is open source under Apache 2.0. It supports Android, iOS, Linux, macOS, Windows, web and embedded devices, with CPU execution and GPU or NPU acceleration on supported hardware. LiteRT-LM can run LLMs fully offline and supports Apple Silicon GPU acceleration on iOS and macOS. Its browser runtime requires WebGPU.
