ONNX Runtime
An open source AI runtime runs ONNX models on desktop, mobile and web platforms, with CPU, GPU and NPU acceleration.
ONNX Runtime is an AI inference and training engine for developers adding models to apps and services. It supports deployment on a user's device or a server, including applications that need the same model format across platforms.
- Model compatibility. Run ONNX models originating in PyTorch, TensorFlow/Keras, scikit-learn, LightGBM and XGBoost, including workloads for text generation and image synthesis.
- Hardware acceleration. Use CPUs, GPUs or NPUs, with model optimizations that target latency, throughput and memory use.
- Browser and mobile inference. ONNX Runtime Web runs models in browsers, while ONNX Runtime Mobile brings inference to Android and iOS apps.
- Training. Train models on device for local personalization, or accelerate transformer training across multiple NVIDIA GPUs.
The runtime is open source under the MIT license. It runs on Windows, macOS, Linux, Android, iOS and web browsers, with interfaces for Python, C#, C++, Java, JavaScript and Rust. Deployment can be local or in the cloud. The project may collect usage data and send it to Microsoft.
