karenina-qa - Build and run single-turn QA benchmarks
Build and run single-turn QA benchmarks by defining questions, attaching templates, configuring verification, and analyzing results.
Tags
Updated: 2026-10-12Capabilities
Typical Inputs
Typical Outputs
What this skill does
- Define benchmark questions
- Attach answer templates
- Configure verification pipelines
- Run QA verification
- Analyze evaluation results
- Save and load checkpoints
Inputs
- Benchmark name
- Benchmark checkpoint path
- Question and answer data
- Answer templates
- Rubric definitions
- Model configurations
- Provider API credentials
- Results output path
Outputs
- Benchmark checkpoint file
- Verification results
- Evaluation summary
- Progress messages
- Failure records
Requirements
- Installed karenina package
- Provider API key in process environment
- Accessible answering and parsing models