ETL and Data Pipeline Engineer

Most etl and data pipeline work breaks on exactly the cases teams skip until production: schema and data-shape contracts across ingestion and warehouse boundaries, and idempotency, replay behavior, and duplicate prevention in reprocessing.

What Pipeline does:

• schema and data-shape contracts across ingestion and warehouse boundaries
• idempotency, replay behavior, and duplicate prevention in reprocessing
• batch/stream ordering, watermark, and late-arrival handling assumptions
• null/default handling and type coercion that can silently corrupt meaning
• data quality controls (completeness, uniqueness, referential integrity)
• observability and lineage signals for fast failure diagnosis
• backfill and migration safety for existing downstream consumers

What you get:

• SOUL.md — Pipeline's identity and working methodology
• etl-engineer.md — the full persona instruction file
• MEMORY.md — session-persistent context template

Install:

# 1. Place in your project
cp etl-engineer.md .claude/personas/PIPELINE.md

# 2. Add to CLAUDE.md
echo "## Active Persona\nPipeline handles etl and data pipeline work. See: .claude/personas/PIPELINE.md" >> CLAUDE.md

# 3. Call by name in Claude Code
# Pipeline, [your task here]

Pipeline maps the problem space before writing a line, validates success and failure paths, and reports residual risk honestly. Use it when production discipline matters more than speed.