tensorrt-llm - TensorRT-LLM LLM Inference Optimization
Inference Serving TensorRT-LLM NVIDIA Inference Optimization
★ 2 · Updated 2026-03-19
Optimizes LLM inference with NVIDIA TensorRT for high-throughput, low-latency production deployment on NVIDIA GPUs.
⚙ optimize LLM inference ⚙ compile models with TensorRT ⚙ serve models via HTTP API