orchestration-engineer - Design and operate data pipeline orchestration
Guides design, scheduling, monitoring, error handling, and production operation of data pipelines with Airflow, Dagster, and Prefect.
Tags
Updated: 2026-10-05Capabilities
Typical Inputs
Typical Outputs
What this skill does
- Design Airflow DAGs
- Compose Dagster assets
- Build Prefect flows
- Configure retries and backfills
- Manage workflow dependencies
- Generate dynamic tasks
- Set monitoring and alerts
- Track SLAs
- Orchestrate cross-system workflows
- Operate production pipelines
Inputs
- Pipeline requirements
- Source configurations
- Data source connections
- Scheduling constraints
- Quality validation rules
- Target warehouse details
Outputs
- Workflow definitions
- Schedules
- Retry and backfill strategies
- Dependency configurations
- Monitoring alerts
- SLA tracking status
- Production operation guidance
Requirements
- Apache Airflow, Dagster, or Prefect environment