ops-check

Ops Check Skill

Run a comprehensive production readiness checklist covering operational excellence pillars.

When to Use

This skill activates when:

• User requests "ops check", "production ready?", "is this deployable?"
• Before deploying to production for the first time
• After major refactoring that touches infrastructure code
• When setting up monitoring and alerting
• During incident post-mortem review

What It Does

Delegates to the ops-engineer agent (Opus model) for comprehensive operational assessment:

1. Structured Logging

   • JSON structured logging (not console.log)
   • Log levels (debug, info, warn, error, fatal)
   • Request correlation IDs for distributed tracing
   • Sensitive data redaction (passwords, tokens, PII)
   • Log rotation and retention configuration

2. Health Checks

   • Liveness probe endpoint (/health or /healthz)
   • Readiness probe endpoint (/ready)
   • Dependency health checks (DB, cache, external services)
   • Startup probe for slow-starting services

3. Graceful Shutdown

   • SIGTERM/SIGINT signal handlers
   • In-flight request drain period
   • Database connection cleanup
   • Message queue consumer shutdown
   • Background job completion or safe interruption

4. Error Handling

   • Global error/exception handler
   • Unhandled promise rejection handler
   • Error classification (retryable vs non-retryable)
   • User-facing error sanitization (no stack traces)
   • Error reporting integration (Sentry, Datadog, etc.)

5. Metrics & Monitoring

   • Request rate, latency, error rate (RED metrics)
   • Custom business metrics
   • Prometheus/StatsD/OpenTelemetry integration
   • Alert thresholds and SLO definitions
   • Dashboard configuration

6. Configuration Management

   • Environment variable usage (no hardcoded values)
   • Configuration validation at startup
   • Secrets management (no .env in production)
   • Feature flags for safe rollouts
   • Environment-specific configuration (dev/staging/prod)

Agent Delegation

Task(
  subagent_type="oh-my-claudecode:ops-engineer",
  model="opus",
  prompt="PRODUCTION READINESS CHECK

Assess the codebase for production readiness across all operational pillars.

Scope: [entire codebase or specific service]

Assessment Pillars:
1. Structured logging (not console.log)
2. Health check endpoints (liveness + readiness)
3. Graceful shutdown (SIGTERM handling, drain)
4. Error handling (global handler, sanitization)
5. Metrics & monitoring (RED metrics, alerting)
6. Configuration management (env vars, validation)

Output: Production readiness scorecard with:
- Per-pillar status (✅ Good / ⚠️ Needs Work / ❌ Missing)
- Specific file:line references for each finding
- Production impact of each issue
- Prioritized improvement roadmap
- Overall readiness grade (A-F)"
)

External Consultation (Optional)

Protocol

1. Form your OWN assessment FIRST - Complete the check independently
2. Consult for validation - Cross-check via a Claude Task agent
3. Critically evaluate - Never blindly adopt external findings
4. Graceful fallback - Never block if delegation is unavailable

When to Consult

• First production deployment of a new service
• Post-incident readiness re-assessment
• Multi-service deployment coordination
• Complex infrastructure requirements

When to Skip

• Routine pre-deployment checks
• Well-established services with existing monitoring
• Minor feature additions

Output Format

PRODUCTION READINESS SCORECARD
==============================

Overall Grade: C (Needs significant improvement)

PILLAR ASSESSMENT
-----------------

1. Logging ❌ MISSING
   - console.log used in 23 locations
   - No structured logging library configured
   - No request correlation IDs
   → Impact: Cannot debug production issues efficiently

2. Health Checks ⚠️ NEEDS WORK
   - /health endpoint exists (src/routes/health.ts:5)
   - No readiness probe (DB connectivity not checked)
   - No dependency health checks
   → Impact: Load balancer cannot detect unhealthy instances

3. Graceful Shutdown ❌ MISSING
   - No SIGTERM handler found
   - No connection drain logic
   → Impact: In-flight requests dropped during deployments

4. Error Handling ⚠️ NEEDS WORK
   - Global error handler exists (src/middleware/error.ts:10)
   - Stack traces leaked in error responses (line 25)
   - No unhandled promise rejection handler
   → Impact: Security risk + silent failures

5. Metrics ❌ MISSING
   - No metrics library configured
   - No request latency tracking
   → Impact: Cannot measure SLOs or detect degradation

6. Configuration ✅ GOOD
   - Environment variables used consistently
   - Validation at startup (src/config/index.ts:8)
   - No hardcoded secrets found

IMMEDIATE ACTIONS (P0)
---------------------
1. Add SIGTERM handler with 30s drain
2. Replace console.log with structured logger
3. Remove stack traces from error responses

IMPROVEMENT ROADMAP
-------------------
| Week | Action | Pillar |
|------|--------|--------|
| 1 | Add structured logging + correlation IDs | Logging |
| 1 | Add SIGTERM handler | Graceful Shutdown |
| 2 | Add readiness probe with DB check | Health Checks |
| 2 | Add unhandled rejection handler | Error Handling |
| 3 | Add Prometheus metrics | Metrics |
| 3 | Add alerting rules | Metrics |

Ops Checklist

Logging

• Structured logging library configured (winston, pino, bunyan)
• JSON format in production
• Request correlation IDs (X-Request-ID)
• Log levels used appropriately
• Sensitive data redacted
• No console.log in production code

Health Checks

• /health or /healthz endpoint exists
• /ready endpoint checks dependencies
• Returns appropriate status codes (200 healthy, 503 unhealthy)
• Response includes version and uptime

Graceful Shutdown

• SIGTERM handler registered
• HTTP server stops accepting new connections
• In-flight requests complete within drain period
• Database connections closed cleanly
• Background workers finish or checkpoint

Error Handling

• Global error handler catches all uncaught exceptions
• Unhandled promise rejections handled
• Error responses sanitized (no stack traces)
• Errors classified and logged with context
• Error reporting service integrated

Metrics

• Request rate metric (requests/sec)
• Request latency metric (p50, p95, p99)
• Error rate metric (errors/sec)
• Custom business metrics defined
• Metrics endpoint exposed (/metrics)

Configuration

• All config from environment variables
• Config validated at startup (fail fast)
• Secrets from secret manager (not .env in prod)
• Feature flags for risky changes
• Different configs per environment

Use with Other Skills

Full Backend Review:

/ops-check then /api-design then /db-review then /security-review

With Ralph:

/ralph ops-check then implement all P0 fixes

Assess, fix critical issues, re-assess until grade A.

With Team:

/team 3:executor "fix all ops-check findings"

Parallel implementation of operational improvements.

Best Practices

• Ops from day one — Don't bolt on logging and health checks after launch
• Fail fast — Validate configuration at startup, not first request
• Drain gracefully — 30s drain period matches typical k8s/ECS defaults
• Log context — Every log line should have enough context to debug without reproduction
• Alert on SLOs — Alert on user-facing impact, not just resource metrics