Know Before You Cut Over: Building a Migration Readiness Checker

Turning platform migration guidance into an automated risk report The problem Many organisations run their applications on an internal developer platform built on top of a public cloud and Kubernetes. The platform gives product teams a paved road: clusters, networking, ingress, monitoring, and security scanning out of the box. Sooner or later, that platform needs a new generation. When it arrives, every product has to move, and the old platform gets a fixed shutdown date. Teams have to migrate while still shipping features. A typical next-generation platform changes how teams own, configure, and secure their workloads: Area Old platform New platform Cloud account A resource group per team A dedicated cloud subscription per team, with more control Management A YAML configuration file in each repo A self-service portal Network & security One shared network Redesigned network; outgoing traffic can be restricted per namespace Ingress ingress-nginx Traefik, since ingress-nginx has been retired by the Kubernetes community Security tooling Trivy vulnerability reports Wiz vulnerability reports The platform team usually publishes a good migration guide. Teams read it. Then cutovers start, and things break anyway. Why cutovers fail The failures are rarely exotic. They are the same small problems, over and over: Ingress rules stop working. An ingress relies on an nginx-specific annotation, such as a URL rewrite. Traefik ignores it without complaint, and users start getting 404s. Outgoing calls get blocked. A service calls a payment provider or an external API. The new per-namespace egress policy blocks it because nobody listed that destination. Pods get evicted. Containers without resource limits are scheduled unpredictably on the new cluster and are first in line when nodes come under pressure. DNS names don't resolve. Hardcoded hostnames point at the old platform. Retries make things worse. Clients with no backoff turn a brief dependency blip into a retry storm. There's no way back. Nobody wrote down how to roll back. Every one of these is usually covered somewhere in the migration guide. The guide isn't the problem. The problem is expecting dozens of teams to audit their own config by hand against a long document, under deadline pressure. If the guidance can be written down, it can be checked automatically. The solution: a Migration Readiness Checker The Migration Readiness Checker reads a workload's configuration, checks it against the platform team's migration guidance, and returns a risk report before cutover. It works like a pre-flight checklist that runs itself. Teams don't each have to interpret the guide on their own. The checker audits the config and reports what will break, what might break, and what is still undecided. What it reads Kubernetes manifests and Helm charts: Deployments, StatefulSets, Services, Ingresses, HPAs, PodDisruptionBudgets Legacy platform config: the old YAML configuration file, so it can flag settings that need to move into the self-service portal Application config: environment variables and config maps holding endpoints, timeouts, and retry settings A migration plan file: a short, team-owned file describing the cutover window, owners, and rollback steps What it checks Every check maps to a section of the migration guide and to a real cutover risk. 1. Resource limits Every container should declare CPU and memory requests and limits. Without them, scheduling on the new cluster is unpredictable and pods risk eviction. 2. Pre-scaling New clusters and fresh nodes need time to warm up. The checker confirms there's a pre-scaling plan for cutover: either an HPA minimum high enough to absorb traffic, or a planned replica increase during the migration window. 3. DNS names It scans config for hardcoded hostnames, cluster-internal names, and endpoints tied to the old platform. Anything that won't resolve after the move, or will resolve somewhere unexpected, is flagged. 4. Retry settings Dependencies blink during cutover. The checker looks for sensible client timeouts, bounded retries with backoff, and idempotency where a retry could duplicate work. It flags both extremes: no retries, and unbounded retries. 5. Rollback plan A cutover without a rollback plan is a one-way door. The checker requires documented trigger conditions, steps, an owner, and an expected duration. 6. Platform-specific checks Ingress: detects ingress-nginx annotations and ingress classes, and lists what needs a Traefik equivalent. Egress: collects every outbound destination the workload talks to, so teams can set up namespace egress rules before traffic gets blocked. Config ownership: maps each setting in the legacy config file to its equivalent in the portal and flags anything without a clear home. Subscription scope: highlights references to resource-group-scoped cloud resources that need to move into the team's new subscription. Security scanning: checks that pipelines have Wiz reporting wired up and aren't still depending only on Trivy output. How it works flowchart LR A[Workload repomanifests, Helm, legacy config,app config, migration plan] --> B[Parsers] B --> C[Normalized workload model] D[Migration rulesfrom platform guidance] --> E[Rule engine] C --> E E --> F[Risk reportMarkdown + JSON] F --> G[PR comment / CI gate] F --> H[Platform dashboard] The design keeps three concerns separate: Parsers turn many config formats into one normalized model of the workload. Rules live as code and are versioned alongside the migration guide. When the guide changes, the rules change in the same pull request. Reporters produce output for humans (a Markdown report on the pull request) and machines (JSON for dashboards and CI gates). Rules are short and readable: - id: RES-001 title: Containers must declare resource limits severity: high applies_to: [Deployment, StatefulSet] check: every container has resources.requests and resources.limits for cpu and memory why: Pods without limits are scheduled unpredictably on new clusters and risk eviction. fix: Add requests and limits based on observed usage from the last 30 days. guide_ref: migration-guide#resources - id: ING-002 title: ingress-nginx annotations need a Traefik equivalent severity: high applies_to: [Ingress] check: no annotations with prefix nginx.ingress.kubernetes.io/ why: The new platform uses Traefik. These annotations will be silently ignored. fix: Replace with a Traefik middleware or IngressRoute. guide_ref: migration-guide#ingress Each rule carries a why, a fix, and a link back to the guide. A report that just says "failed" invites arguments. A report that explains the risk and the remedy gets acted on. Architecture and technology choices The checker has one firm rule: deterministic checks decide pass or fail, and AI only helps with the fuzzy parts. A readiness gate has to give the same answer twice and be easy to audit. An AI model is useful for explaining findings and handling messy input, but it shouldn't decide whether a cutover is safe. flowchart TB subgraph Trigger["Trigger"] PR["Pull request"] SCHED["Scheduled fleet scan"] end subgraph Collect["1. Collect and render"] GIT["Fetch workload repo"] RENDER["Render manifestshelm template / kustomize build"] REDACT["Redact secrets andsensitive values"] end subgraph Normalize["2. Normalize"] PARSE["ParsersK8s, legacy YAML,app config, migration plan"] MODEL["Normalized workload modelJSON"] end subgraph Check["3. Deterministic checks"] OPA["Rule engineOPA / Conftest"] RULES[("Rule catalogueRego + YAML metadataversioned with the guide")] end subgraph AI["4. AI assist (advisory only)"] RAG[("Migration guide indexretrieval")] LLM["LLMstructured JSON output"] VALIDATE["Schema validation"] end subgraph Report["5. Report"] MERGE["Merge findings"] MD["Markdown reportPR comment"] JSON["JSON report"] GATE{"Any HIGH findings?"} end subgraph Consume["Consumers"] CI["CI gateblock or allow cutover booking"] STORE[("Report store")] DASH["Fleet risk dashboard"] end PR --> GIT SCHED --> GIT GIT --> RENDER --> REDACT --> PARSE --> MODEL MODEL --> OPA RULES --> OPA OPA -->|findings| MERGE MODEL -->|unmapped config,free-text plans| LLM OPA -->|findings to explain| LLM RAG --> LLM LLM --> VALIDATE -->|suggestions| MERGE MERGE --> MD MERGE --> JSON JSON --> GATE --> CI JSON --> STORE --> DASH 1. Collect and render The checker runs as a container in the CI pipeline, triggered on every pull request and on a nightly scan of all workloads. It fetches the workload repo and renders the final manifests with helm template or kustomize build. Checking rendered output matters: the values teams actually deploy often differ from chart defaults. Before anything else, a redaction step strips secret values, tokens, and connection strings. Only keys and structure are passed on. This keeps sensitive data out of logs and out of anything sent to an AI model. 2. Normalize Small parsers, written in Python, read each input format and produce one normalized workload model in JSON: containers, resources, scaling, ingress routes, outbound endpoints, retry settings, and the rollback plan. Every downstream component works from this model, so adding a new input format never touches the rules. 3. Deterministic checks: OPA and Conftest Rules are written in Rego and run with Open Policy Agent through Conftest, a widely used policy-as-code approach for Kubernetes configuration. Each Rego rule is paired with YAML metadata (severity, why, fix, guide reference) like the examples above. package migration.ingress import rego.v1 deny contains msg if { input.kind == "Ingress" some key, _ in input.metadata.annotations startswith(key, "nginx.ingress.kubernetes.io/") msg := sprintf("ING-002: %s uses %s", [input.metadata.name, key]) } Rules are unit-tested with opa test and versioned in the same repository as the migration guide. A change to the guide and its rule ship in one pull request. 4. AI assist: advisory only A large language model handles the work that rules can't do well: Task Why rules struggle What the model does Explaining findings Generic fix text doesn't fit every workload Writes a short, workload-specific explanation and fix Mapping legacy config Old settings don't always map one-to-one to portal options Suggests the closest portal equivalent, with the relevant guide section Traefik suggestions nginx annotations vary widely, and some combine several behaviours Drafts a Traefik middleware or IngressRoute snippet Reading migration plans Rollback plans are often free text Extracts trigger, steps, owner, and duration into structured fields Guardrails keep it safe and predictable: Grounded in the guide. The migration guide is indexed for retrieval, and every prompt includes the relevant sections. Suggestions must cite a guide section. Structured output only. The model returns JSON that is validated against a schema. Anything invalid is dropped, and the deterministic finding still stands. Never decides pass or fail. AI output is labelled as a suggestion in the report and can't change a severity or clear a finding. Redacted input. The model only sees the redacted workload model, never raw secrets. Model-agnostic. Any hosted LLM with structured output works. On an Azure-based platform, a model hosted in the company's own Azure tenant keeps data inside existing boundaries. The checker still works with the AI step turned off. Teams just get the standard why and fix text instead of tailored suggestions. 5. Report and consume Deterministic findings and AI suggestions are merged into two outputs: A Markdown report, posted as a pull request comment. A JSON report, used by the CI gate (any high-severity finding blocks booking a cutover window) and stored for the fleet dashboard. Technology summary Layer Choice Runtime Container image run in CI and as a scheduled job Rendering helm template, kustomize build Parsing and orchestration Python Policy engine Open Policy Agent with Conftest, rules in Rego Rule tests opa test AI assist Hosted LLM with structured JSON output, retrieval over the migration guide Outputs Markdown (PR comment), JSON (CI gate, report store) Dashboard Any BI or observability tool reading the stored JSON reports What the risk report looks like A report for a typical billing service, a few weeks before cutover: Migration Readiness Report: billing-service Target: new platform Overall risk: HIGH (2 high, 3 medium, 1 low) HIGH ING-002 Ingress uses nginx.ingress.kubernetes.io/rewrite-target → Recreate as a Traefik middleware before cutover. RBK-001 No rollback plan found in migration-plan.yaml → Document trigger, steps, owner and expected duration. MEDIUM NET-003 Outbound calls to 3 external hosts not covered by an egress rule → Add them to the namespace egress allow-list in the portal. RTY-002 HTTP client for the ledger service retries without backoff (max 10) → Use exponential backoff and cap retries at 3. SCL-001 HPA minReplicas=2; peak traffic suggests at least 6 during cutover → Pre-scale for the migration window. LOW DNS-004 Config references an old-platform hostname in an unused block → Remove to avoid confusion. Passed: 14 checks (resource limits, PDB, liveness/readiness probes, ...) A team can read this in two minutes. It shows what blocks the cutover, what to fix this sprint, and what can wait. Where it fits in the workflow The checker is most useful early and often, not as a last-minute gate: Discovery: one run across all workloads gives the platform team a heat map of migration risk by team, so support goes where it's needed most. Preparation: the checker runs on every pull request, so teams see their score improve as they fix findings. Before cutover: a report with no high-severity findings is required before a migration window can be booked. After cutover: the final report is archived as part of the migration record. Design principles Guidance as code. Every rule traces back to a section of the migration guide. If it can't, it doesn't belong. Explain, don't just judge. Every finding says why it matters and how to fix it. Severity means cutover risk. "High" means "this will likely break during migration," not "this isn't best practice." Allow justified exceptions. Teams can suppress a finding with a reason and an owner. Exceptions stay visible in the report. Read-only by default. The checker reads config and reports. It never changes a workload. Fixes stay with the team that owns the code. What's next Natural extensions once the basic checker is in place: Runtime signals: combine static config with observed traffic to discover egress destinations and peak load automatically. Auto-generated fixes: propose Traefik equivalents for common nginx annotations as draft pull requests. Post-cutover verification: run the same checks against live deployments on the new platform to catch drift. Closing thought Platform migrations rarely fail because of one big problem. They fail because of a dozen small ones that each looked fine on their own. Platform teams already know what those problems are and have written them down. A Migration Readiness Checker turns that knowledge into something every team can run in seconds, long before the cutover window opens. If your platform has a shutdown date on the calendar, the best time to find out what will break is now.