hugging-face-evaluation - Manage Hugging Face model evaluations
Add and manage evaluation results in Hugging Face model cards by extracting README tables, importing Artificial Analysis scores, or running custom evaluations.
Tags
Updated: 2026-10-16Capabilities
Typical Inputs
What this skill does
- Check existing pull requests
- Inspect evaluation tables
- Extract README evaluation data
- Import Artificial Analysis scores
- Generate model-index YAML
- Merge evaluation metadata
- Validate model-index metadata
- Run Hugging Face Jobs evaluations
- Run GPU model evaluations
- Create evaluation pull requests
Inputs
- Hugging Face repository ID
- README content
- Table selection
- Model column hints
- Model and task identifiers
- Hugging Face write token
- Artificial Analysis API key
- Evaluation hardware
- GPU model resources
Outputs
- Generated evaluation YAML
- Updated model card metadata
- Evaluation scores
- Evaluation job
- Evaluation pull request
Requirements
- Python environment
- uv or installed dependencies
- Hugging Face write access
- Artificial Analysis API key for imports
- GPU and sufficient memory for vLLM
- uv installed for vLLM evaluation
- Hugging Face Jobs access for remote jobs
