fine-tuning-with-trl - Post-training and preference alignment for LLMs with TRL
Provides methods and workflows for LLM post-training, including SFT, DPO, PPO, GRPO, and reward modeling.
Tags
Updated: 2026-09-24Capabilities
Typical Inputs
Typical Outputs
What this skill does
- Execute supervised fine-tuning
- Train reward models
- Align models using DPO
- Optimize policies with PPO
- Train online RL with GRPO
- Evaluate aligned models
Inputs
- Instruction dataset
- Preference dataset
- Pre-trained language model
- Reward model
- Training configuration
Outputs
- Saved fine-tuned model checkpoints
- Trained reward model
- Generated text evaluation output
Requirements
- Python environment
- TRL package
- Transformers package
- Datasets package
- PEFT package
- Accelerate package
- PyTorch package