vlm-ocr-evaluation - Compare OCR systems before bulk processing
Evaluates and compares OCR systems on human-transcribed samples using CER and WER to select the optimal model before bulk execution.
Tags
Updated: 2026-09-24Capabilities
Typical Inputs
Typical Outputs
What this skill does
- Assemble OCR candidate model sets
- Build stratified ground truth samples
- Compute CER and WER edit distances
- Normalize text prior to scoring
- Aggregate accuracy metrics across strata
- Document model selection gate decisions
Inputs
- Candidate OCR model specifications
- Human-transcribed ground truth pages
- Corpus metadata and language tags
Outputs
- Model registry configuration
- Page-level normalized transcription outputs
- Stratified CER and WER reports
- Documented selection gate rationale
Requirements
- GPU memory for model execution
- vLLM or Ollama serving framework
- Tesseract OCR baseline installation
- Proprietary vision API access credentials