ppa-benchmark - Set up defensible PPA comparison plans across arms
Produces a structured comparison plan and evidence-linked report across arms without scoring or naming a winner.
Tags
Updated: 2026-09-17Capabilities
Typical Inputs
Typical Outputs
What this skill does
- Define comparison arms and baseline
- Fix baseline design conditions
- Declare seed policy and runs
- Match metric scopes across arms
- Record host load and timing
- Document residual unfairness
- Generate evidence-linked comparison report
Inputs
- Arm definitions
- Held-constant conditions
- Metrics and scope definitions
- Run count per arm
- Host load information
Outputs
- Markdown PPA benchmark report
- Canonical metric JSON records
Requirements
- Python 3 environment
- _ppa/benchmark.py scorer program
- programs/_ppa/canonical_json.py serializer
- Compliance check script and configuration