ExecuTorch

ExecuTorch is an on-device AI runtime for developers adding PyTorch models to apps and embedded devices. It handles inference on the user's hardware across Android, iOS, macOS, Linux and Windows, as well as embedded systems and microcontrollers.

• Local LLMs: Run quantized text models with long context, tool calling and speculative decoding for text generation and agent applications.
• Speech processing: Support streaming and offline speech recognition, speech synthesis, voice activity detection and speaker diarization.
• Vision and multimodal models: Run image classification, object detection and segmentation, plus models that handle text, images and audio.
• Hardware acceleration: Use CPU, GPU, NPU or DSP backends, including XNNPACK, Core ML, OpenVINO and Qualcomm accelerators, with portable CPU kernels for fallback.
• App integration: Embed inference through C++, Python, Swift/Objective-C or Kotlin/Java APIs, with profiling and numeric debugging tools to inspect performance and model output.

Developers can reuse a source PyTorch model across devices, but hardware-specific targets need separately exported model files. The runtime can include only the operators and backend components an application needs. ExecuTorch also runs AI features on Meta Quest and Ray-Ban Meta devices.