hugging-face-evaluation - Manage Hugging Face model evaluations
Extract, import, validate, and run model evaluations, then add structured results to Hugging Face model cards.
Tags
Updated: 2026-10-16Capabilities
Typical Inputs
Typical Outputs
What this skill does
- Inspect README tables
- Extract evaluation results
- Import Artificial Analysis scores
- Generate model-index YAML
- Merge model-index entries
- Validate model-index metadata
- Create evaluation pull requests
- Run inference-provider evaluations
- Run vLLM evaluations
- Run lighteval evaluations
Inputs
- Hugging Face model repository
- README content
- Model name
- Model identifier
- Evaluation table number
- Model column index
- Task type
- Artificial Analysis API key
- Evaluation task
- Hardware selection
- Hugging Face token
Outputs
- Evaluation YAML
- Updated model card
- Model-index metadata
- Evaluation pull request
- Evaluation job
- Evaluation results
Requirements
- Python environment
- Required Python dependencies
- Hugging Face write-access token
- Artificial Analysis API key for imports
- uv for vLLM scripts
- Sufficient GPU memory for vLLM evaluation
- NVIDIA GPU tools for GPU checks