llama-cpp - Local GGUF Inference & HF Discovery
Local GGUF inference, quant selection, and Hugging Face Hub model discovery for llama.cpp
Tags
Updated: 2026-05-11Capabilities
Typical Inputs
Typical Outputs
What this skill does
- Run local models
- Search Hugging Face Hub
- Find GGUF files
- Build llama-server command
- Build llama-cli command
- Enumerate GGUF files
- Select quant variants
- Load GGUF model
- Generate text
- Create chat completion
- Stream responses
- Generate embeddings
- Extract file metadata
Inputs
- Hugging Face repository
- GGUF file
- Quant variant
- Hardware profile
- Prompt text
- Chat messages
Outputs
- Generated text
- Chat completion
- Embedding vector
- llama-server command
- llama-cli command
- GGUF file list
- File metadata
Requirements
- llama-cpp-python >= 0.2.0
- Linux or macOS or Windows
- Optional CUDA/ROCm/Metal