prompt-engineer-toolkit - LLM Prompt Engineering and Testing Toolkit
A/B test prompts, version them, and ensure regression safety for production LLM workflows
Tags
Updated: 2026-03-23Capabilities
Typical Inputs
Typical Outputs
What this skill does
- Run A/B prompt tests
- Score prompt outputs
- Track prompt versions
- Compare prompt versions
- Create prompt templates
- Run regression tests
- Audit prompt quality
- Calculate prompt metrics
Inputs
- Test cases JSON file
- Baseline prompt file
- Candidate prompt file
- LLM runner command
- Prompt name
- Change notes
Outputs
- A/B test results
- Prompt version history
- Version diff report
- Quality metrics report
- Prompt audit report
Requirements
- Python 3 runtime
- Local script execution
- LLM CLI access
- JSON configuration support
- Local filesystem access