---
name: backend-reliability-enforcer
description: Use when implementing backend APIs, data persistence, or external integrations. Enforces TodoWrite with 25+ items. Triggers: "backend API", "REST endpoint", "external integration", "data persistence". If thinking "just a quick endpoint" - use this.
---

# Backend Reliability Enforcer

## Payment Operations?

**If implementing payment, billing, subscription, or financial transactions:**
→ Use `payment-operations-enforcer` skill FIRST (NON-NEGOTIABLE)
→ Then return here for general reliability requirements

---

## TodoWrite Requirements

**CREATE TodoWrite with 5 sections (25+ items total):**

| Section | Min Items |
|---------|-----------|
| Fault Tolerance | 5+ |
| Error Handling | 5+ |
| Data Integrity | 5+ |
| Security | 5+ |
| Observability | 5+ |

---

## Verification Checkpoint

Verify 3 random items have ALL THREE:
- ✓ Concrete numbers ("5 failures/10s", "15s timeout", "100 req/min")
- ✓ Specific tools ("Opossum", "pino", "Joi", "Knex", "Redis")
- ✓ Measurable outcome ("generates UUID v4 correlation ID")

| ❌ FAILS | ✅ PASSES |
|----------|-----------|
| "Add error handling" | "pino logging: UUID v4 correlation ID, format `{correlationId, level, timestamp, service, message}`" |
| "Validate input" | "Joi schema: `amount` (number, positive, max 999999), `currency` (ISO 4217). Return 400 with `{error, fields}`" |
| "Add circuit breaker" | "Opossum: 5 failures/10s threshold, 30s reset, fallback to cached data" |

---

## Section Requirements

### Fault Tolerance (5+ items)

- [ ] Circuit breaker for ALL external calls (Opossum: 5 failures/10s, 30s reset)
- [ ] Retry with exponential backoff (3 retries: 100ms, 200ms, 400ms)
- [ ] Timeouts for all I/O (15s API calls, 5s database)
- [ ] Graceful degradation (cached data, disable non-critical features)
- [ ] Health checks (`/health` endpoint with dependency status)

### Error Handling (5+ items)

- [ ] Comprehensive error handling (try/catch, error middleware)
- [ ] Correlation IDs (UUID v4 on ingress, propagate to all logs/services)
- [ ] Structured logging (pino/winston with correlation ID)
- [ ] Error responses (400/401/403/404/422/500, never expose stack traces)
- [ ] Error metrics/alerts (Prometheus counter, alert if error rate > 5%)

### Data Integrity (5+ items)

- [ ] Input validation at API boundaries (Joi/Zod schemas)
- [ ] Database transactions for consistency (Knex/Prisma transactions)
- [ ] Idempotency keys for retryable operations (`Idempotency-Key` header, Redis 24h)
- [ ] Data retention policies (auto-delete soft-deleted after 90 days)
- [ ] Audit trails for sensitive modifications (who, what, when, old/new)

### Security (5+ items)

- [ ] Authentication on all endpoints (JWT/OAuth/API keys)
- [ ] Authorization checks (RBAC, verify permissions)
- [ ] Input sanitization (parameterized SQL, XSS prevention)
- [ ] Encryption (AES-256 at rest, TLS 1.3 in transit)
- [ ] Rate limiting (100 req/min per IP, express-rate-limit)

### Observability (5+ items)

- [ ] Structured logging (INFO/WARN/ERROR levels)
- [ ] Key metrics (latency P50/P95/P99, throughput, error rate)
- [ ] Distributed tracing (OpenTelemetry with trace/span IDs)
- [ ] Dashboards/alerts (Grafana, PagerDuty for SLO breaches)
- [ ] Runbook documentation (investigation, rollback, escalation)

---

## Red Flags - STOP When You Think:

| Thought | Reality |
|---------|---------|
| "Add reliability later" | 80% never added, retrofitting costs 10x |
| "That edge case is unlikely" | Unlikely × scale = frequent |
| "Circuit breaker is overkill" | One slow service = cascade failure |
| "Customer needs it in 3 days" | Broken code causes MORE delay |

---

## Response Templates

### "Add Reliability Later"

❌ **BLOCKED**: Cannot defer reliability requirements.

- 80% of "later" items never get added
- Retrofitting costs 10x more than building in from start
- Production incidents cost $5K-50K per incident + customer trust

**Required to override:**
1. Specific retrofit date (not "later")
2. Budget allocated (engineer-weeks)
3. Risk acceptance signed by decision maker
4. Interim mitigation plan (reduced traffic limits? 24/7 on-call?)

---

## Self-Grading Before Complete

```
[ ] 25+ items across 5 sections
[ ] 80%+ have concrete numbers
[ ] 80%+ name specific tools
[ ] 100% have measurable outcomes
[ ] 3 random items passed specificity test
[ ] Payment operations: delegated to payment-operations-enforcer

Grade 7+/8: Ready to proceed
Grade <7: Revise TodoWrite
```