vision-transformers-registers - Vision Transformers Don't Need Trained Registers
Apply test-time register token injection to pretrained Vision Transformers to reduce high-norm outlier artifacts without retraining.
Tags
Updated: 2026-09-28Capabilities
What this skill does
- Detect register neurons
- Identify outlier tokens
- Inject test-time register tokens
- Shift neuron activations
- Analyze attention maps
- Compute attention metrics
Inputs
- Pretrained Vision Transformer
- Image dataloader
- Input images
- Target layer index
- Attention layer index
- Attention head index
Outputs
- Register neuron indices
- Injected-token inference outputs
- Attention maps
- Attention quality metrics
Requirements
- PyTorch environment
- Pretrained Vision Transformer checkpoint
- Access to transformer layer internals
- Inference without retraining