promptfoo - LLM Evaluation Framework for Testing and Comparing Outputs
CLI tool for testing and comparing LLM outputs using evaluation configurations and assertions
Tags
Updated: 2026-03-09Capabilities
Typical Inputs
Typical Outputs
What this skill does
- run evaluation configurations
- create test cases
- debug evaluation runs
- work with assertions
Inputs
- evaluation config file
- test cases
- prompts
- assertion definitions
Outputs
- evaluation results file
- test comparison data
- assertion pass/fail status
Requirements
- Node.js environment
- promptfoo CLI
- YAML/JSON configuration files