pytorch-architecture - PyTorch Architecture Design & Best Practices
Design and implement PyTorch modules with best practices for memory, DDP, and performance.
Tags
Updated: 2026-03-10Capabilities
Typical Inputs
Typical Outputs
What this skill does
- design encoder modules
- design decoder modules
- design loss modules
- write PyTorch modules
- refactor PyTorch modules
- write unit tests
- estimate memory footprint
- setup distributed training
- enable flash attention
- enable torch.compile
- document tensor shapes
- check DDP behavior
Inputs
- project goals
- tensor shapes
- memory constraints
- training arguments
- model configuration
- loss configuration
- data configuration
Outputs
- PyTorch module code
- unit test code
- trained model files
- tensor shape documentation
- training logs
Requirements
- PyTorch
- CUDA
- NCCL
- distributed training environment