llama-cpp - Local GGUF Model Inference
Run GGUF models locally and discover models from Hugging Face Hub
Tags
Updated: 2026-05-11Capabilities
Typical Inputs
Typical Outputs
What this skill does
- discover Hugging Face repos
- download GGUF files
- run GGUF models locally
- generate llama-server command
- generate llama-cli command
- generate text
- generate chat completions
- create embeddings
- select quantization
- list GGUF files
Inputs
- Hugging Face repo ID
- GGUF model file
- quantization label
- context window size
- model URL
Outputs
- text response
- chat completion
- embedding vector
- model file list
- server command
- HTTP endpoint
Requirements
- llama.cpp installation
- llama-cpp-python>=0.2.0
- Linux/macOS/Windows
- CPU or GPU