advanced-evaluation - Evaluate LLM outputs with reliable judging methods
Provides methods for scoring and comparing LLM outputs, designing rubrics, selecting metrics, and mitigating evaluation bias.
Tags
Updated: 2026-10-03LLM evaluationLLM-as-a-judgequality assessmentpairwise comparisonevaluation biasrubricsevaluation pipelines
Capabilities
Typical Inputs
Typical Outputs
What this skill does
- Score responses directly
- Compare responses pairwise
- Design evaluation rubrics
- Select evaluation metrics
- Mitigate position bias
- Calibrate confidence scores
- Analyze judge agreement
Inputs
- Original prompts
- LLM responses
- Evaluation criteria
- Rubrics
- Evaluation context
Outputs
- Scores
- Pairwise verdicts
- Justifications
- Confidence scores
- Evaluation summaries
- Evaluation rubrics
Requirements
- LLM judge support
- Structured JSON output support