serving-llms-vllm - High-throughput LLM serving framework
vLLM Inference Serving PagedAttention Continuous Batching
★ 0 · Updated 2026-09-22
High-throughput LLM serving engine supporting OpenAI compatible API, quantization, and tensor parallelism.
⚙ Deploy production LLM APIs ⚙ Run offline batch inference ⚙ Serve quantized LLM models