curate-scanner-evaluation-corpus - Curate Scanner Evaluation Corpus
Select pinned repositories, define annotation scopes, prepare review packets, maintain ground truth, and adjudicate labels for deterministic scanner evaluation.
Tags
Updated: 2026-09-16Capabilities
Typical Inputs
Typical Outputs
What this skill does
- Select coverage candidate repositories
- Define exhaustive annotation scopes
- Prepare human review packets
- Adjudicate ground truth annotations
- Correct ground truth labels
- Protect evaluation metric integrity
Inputs
- Third-party code repositories
- Pinned commit hashes
- Taxonomy definitions
- Annotation scope rules
Outputs
- Candidate repository rationales
- Declared annotation scopes
- Human-review worksheets
- Accepted ground-truth data
- Corpus change notes
- Evaluation summaries