unsloth-inference - Deploy Fine-Tuned Models for Production Inference
Deploy fine-tuned models for production inference using optimized kernels, vLLM, or SGLang
Tags
Updated: 2026-02-15Capabilities
Typical Inputs
Typical Outputs
What this skill does
- load fine-tuned model
- enable optimized kernels
- merge LoRA weights
- deploy vLLM server
- create OpenAI-compatible API
- generate model responses
Inputs
- fine-tuned model
- tokenizer
- input prompts
- server configuration
- LoRA adapters
Outputs
- inference results
- OpenAI-compatible endpoint
- served model files
- API responses
Requirements
- unsloth library
- torch installed
- Python environment
- vLLM optional
- SGLang optional