tensorrt-llm - NVIDIA TensorRT-LLM LLM Inference Optimizer
Optimizes LLM inference on NVIDIA GPUs with quantization and multi-GPU scaling
Tags
Updated: 2026-03-19Capabilities
Typical Inputs
Typical Outputs
What this skill does
- optimize LLM inference
- generate text
- serve chat completions
- compile models
- quantize models
- scale across GPUs
Inputs
- LLM model identifier
- input prompts
- sampling parameters
- NVIDIA GPU hardware
Outputs
- generated text
- inference results
- model compilation output
- chat completion responses
Requirements
- NVIDIA GPU (A100/H100/GB200)
- CUDA 13.0.0
- TensorRT 10.13.2
- Python 3.10-3.12