December 4, 2023:
Triton Inference Server is designed for flexibility and allows developers to create and deploy inferencing solutions in various ways. Triton Inference Server enables teams to deploy any AI model from multiple deep learning and machine learning frameworks, including TensorRT, TensorFlow, PyTorch, ONNX, OpenVINO, Python, RAPIDS FIL, and more.
NVIDIA has developed a Secure Deployment Considerations Guide to help our users make knowledgeable decisions that preemptively consider Secure Deployment. This guide provides best practices that users deploying Triton-based solutions should consider integrating to fortify the setup and deployment of their Model Repository.
Additionally, NVIDIA made the following items available in the Triton development branch on November 10, 2023, all of which are available in the release branch today, December 4, 2023.
- Updated software that behaves as follows:
- Provides the ability to restrict the HTTP endpoint of the model load API
- Prevents the model load API configuration option from accessing directories outside the model directory
Revision History
| Revision | Date | Description |
|---|---|---|
| 1.1 | December 18, 2023 | Release date updated to December 4, 2023 |
| 1.0 | November 30, 2023 | Initial release |
Disclaimer
ALL NVIDIA INFORMATION, DESIGN SPECIFICATIONS, REFERENCE BOARDS, FILES, DRAWINGS, DIAGNOSTICS, LISTS, AND OTHER DOCUMENTS (TOGETHER AND SEPARATELY, “MATERIALS”) ARE BEING PROVIDED “AS IS.” NVIDIA MAKES NO WARRANTIES, EXPRESS, IMPLIED, STATUTORY, OR OTHERWISE WITH RESPECT TO THE MATERIALS, AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OR CONDITION OF TITLE, MERCHANTABILITY, SATISFACTORY QUALITY, FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT, ARE HEREBY EXCLUDED TO THE MAXIMUM EXTENT PERMITTED BY LAW.
Information is believed to be accurate and reliable at the time it is furnished. However, NVIDIA Corporation assumes no responsibility for the consequences of use of such information or for any infringement of patents or other rights of third parties that may result from its use. No license is granted by implication or otherwise under any patent or patent rights of NVIDIA Corporation. Specifications mentioned in this publication are subject to change without notice. This publication supersedes and replaces all information previously supplied. NVIDIA Corporation products are not authorized for use as critical components in life support devices or systems without express written approval of NVIDIA Corporation.