Skills tagged "vllm"

vllm-metax-model-upgrade - Upgrade vLLM Model Support on MetaX MACA
vllm metax maca model-upgrade
★ 174 · Updated 2026-09-22
Review and upgrade MetaX model support against target vLLM revisions and installed MACA components, including model-dependent attention and kernels.
⚙ Build MACA upstream difference matrix ⚙ Inventory model dependencies recursively ⚙ Update affected dependencies and callers
huggingface-community-evals - Evaluate Hugging Face models with inspect-ai or lighteval
huggingface model-evaluation inspect-ai lighteval
★ 1 · Updated 2026-09-13
Runs evaluations against Hugging Face Hub models on local hardware using inspect-ai or lighteval.
⚙ Choose evaluation framework and backend ⚙ Run evaluation smoke tests ⚙ Execute local benchmark tasks
arxiv-2608-19889-write-once-run-everywhere-the-axon-dsl-for-shape-s - Axon DSL for Shape-Safe Framework-Agnostic LLMs
dsl llm pytorch jax
★ 3 · Updated 2026-09-11
Presents the Axon DSL for compiling shape-safe, framework-agnostic LLM architectures to PyTorch, JAX, MLX, and vLLM.
⚙ Compile LLMs to PyTorch implementations ⚙ Compile LLMs to JAX implementations ⚙ Compile LLMs to MLX implementations
vllm-ascend-model-adapter - Adapt Models for vLLM on Ascend NPU
vllm ascend npu model adaptation
★ 2 · Updated 2026-05-11
Adapt Hugging Face or local models to run on vllm-ascend with minimal changes
⚙ analyze model architecture ⚙ inspect model files ⚙ implement model adapter
llm-inference-scaling - Auto-scale LLM inference clusters on Kubernetes
kubernetes autoscaling LLM GPU
★ 602 · Updated 2026-03-26
Scale LLM inference horizontally on Kubernetes with GPU-aware autoscaling, request queuing, and spot instance strategies.
⚙ Deploy vLLM inference pods ⚙ Configure KEDA autoscaling ⚙ Set up queue-based scaling
Local Model Deployment - Deploy and integrate local AI models with VS Code Agent
local-models ollama vllm model-deployment
★ 0 · Updated 2026-02-23
Deploy local or open-source AI models with model selection, quantization, Ollama/vLLM setup, and integration with ModelSelector
⚙ deploy local models ⚙ configure Ollama runtime ⚙ configure vLLM runtime
unsloth-inference - Deploy fine-tuned models for production inference with Unsloth
model-inference model-deployment vllm sglang
★ 602 · Updated 2026-02-14
Deploy fine-tuned models for production inference with optimized kernels and serving engines.
⚙ Load fine-tuned model ⚙ Load tokenizer ⚙ Enable optimized kernels