Labs — Phase 12, GitOps & platform engineering¶
Eighteen exercises in the order they are meant to run, for the only phase that climbs up. Eleven phases removed abstraction until kubectl run nginx was a cited path from terminal to syscall; this one builds abstraction back — deliberately, as a platform someone else could use. Its real deliverable is not the platform but the critique of it: what your abstraction costs the developer who consumes it. Nothing here is new machinery — Flux is the P4 reconcile loop, a Composition is a CRD with a controller, a vcluster is an apiserver-plus-syncer whose pods are real host pods. What is new is judgement.
Every exercise is either a piece of the platform, or a measurement of what it costs. The build-artifact files (delivery, golden path, KRO, Crossplane, the platform capstone) each produce a working piece a developer can drive; the rest measure the cost — a kubectl top pod, a resources: {} QoS class, a time-to-leak, a syncer's real RSS. This phase's citations are the tools' own source at pinned tags (helm-controller's drift mode, Crossplane's mode enum and empty DeploymentRuntimeConfig, the vcluster syncer), held to the same archaeology standard as k/k — because platform tooling moves fast and a described mechanism is not a cited one.
Each file states one claim to test or one artifact to produce, links its topology rather than restating a footprint, and ends with a teardown that deletes what it created and says whether the topology stays or goes. The framing — why each module exists, the question to answer from each tool's source — stays in phases/12-gitops-platform.md. These files hold only what you type and what you should see.
| # | Exercise | Why it exists |
|---|---|---|
| 1 | commit → Source → Kustomization → applied, no kubectl apply |
The P4 reconcile loop with a git Source on the front — and the chart pulled from a git revision, so the isolated bridge needs no registry. Carries the empty-node baseline the whole budget rests on. |
| 2 | Drift detection is off by default | A hand-edit survives until you flip one field — the difference between converging on change and converging continuously, cited to helm-controller's GetMode(). |
| 3 | 12.C1 — a commit is a deploy to everything | One bad commit reconciles everywhere at once, and a rotated secret every consumer already cached. Git-as-truth is a double-edged guarantee. |
| 4 | Three ways to not commit a secret | SOPS, sealed-secrets, ESO compared on where decryption happens and what holds the key — SOPS shipped at zero in-cluster cost, proved by pod count. |
| 5 | A canary gated on a metric | Flagger promotes only while an SLO holds and rolls back on a breach — the P1 rollout from the other end. Installs Group A's heavy half (Istio + Prometheus). |
| 6 | 12.C2 — the metric dies mid-canary | Kill Prometheus mid-analysis and watch a metric-gated system fail closed, not open. The failure mode of a metric-gated system is the metric. |
| 7 | A golden path with no portal | Template repo + CRD + Flux: a PR becomes an app, no product installed. Platform-as-product made of three things you already have. |
| 8 | Backstage, read and never installed | Its scaffolder runs once; a reconciled CRD converges forever. Cited from Backstage's own source on forge — the build OOMs, so it is read, not run. |
| 9 | 12.C5 — the leak in the golden path | One deliberate leak, and a stopwatch on a fresh consumer hitting it. The failure-mode evidence the critique is built from. |
| 10 | The teardown that proves git | The group boundary is a test: if the cluster does not come back from git, Group A never made git the source of truth. Reclaims the RAM Group B needs. |
| 11 | A tenancy boundary and where it leaks | Four objects draw the namespace-per-team boundary; the deliverable is the inventory of exactly where it does not hold. |
| 12 | The same pod in two apiservers | One container, two views — real host pod and tenant shim — with the syncer's real RSS measured first and its 4Gi limit dropped to ~1Gi. |
| 13 | 12.C3 — the syncer dies, the host pods live | Delete the syncer and the cluster comes apart along the real/shim line: workloads up, tenant view dark. |
| 14 | KRO: custom resource in, children out | The bare mechanism — 1 pod, no packages, no functions, no providers — taught first so Crossplane's machinery arrives with a reason to exist. |
| 15 | Crossplane: the two lines that size your pod | mode has one value, the default DeploymentRuntimeConfig is empty, and every function pod ships resources: {} — the platform's own pod-sizing problem, cited. |
| 16 | Helm cannot publish an API | kubectl get webapp works under a Composition and fails under Helm — four consequences of publishing an API versus rendering a manifest once. |
| 17 | 12.C4 — the function pod with no limits | Remove the override and the mandatory function pod runs BestEffort until the node reclaims it — the phase's thesis, executed. |
| 18 | The platform, and its critique | A developer drives the whole thing with one create; then the critique — what it hides, where it leaks, which prior-phase skill it makes invisible. The deliverable, and the end. |
Order, topology, and the two groups — where the footprint fog patch resolves¶
This is the heaviest phase in the curriculum, and it is the second place the map warned a single 9.9GB window might not fit a whole phase. It resolves here, and the resolution is a topology change plus a split, not a softening.
One node, added for this phase. #14 costed the stack and found pair the wrong shape: nothing in P12 is a DaemonSet, so a second node would only pay the OS+kubelet tax twice and fragment 5GB into two buckets. The answer is platform — one node, 6144MB / 4c / 40G, real untainted kubeadm — added for P12 and not a replay of any earlier topology. It comes up at exercise 1 and is destroyed by the capstone.
Two groups, with a teardown between — because both at once is unmeasured. Per-exercise sequencing confirms the research arithmetic: against a ~1.0–1.3GB single-node baseline, Group A (delivery: exercises 1–9, ~1.9GB of requests, its heavy half being Istio + Prometheus at ~1.15GB) and Group B (platform API & tenancy: exercises 11–18, ~1.4GB) each sit near 2.5–3.2GB of the 6.0GB — enough headroom that observed RSS overshooting requests is survivable. Both co-resident would land near 3.9GB plus baseline; it probably fits, and it is still not done, because the one number the phase rests on — the vcluster syncer's real idle RSS — is unmeasured until exercise 12 measures it. Exercise 10 is the boundary: it tears Group A's stack down (reclaiming the ~1.15GB) and doubles as the strongest GitOps test in the phase.
Nothing in P12 was dropped for want of capacity, and nothing multi-cluster was added. vcluster is the single-node answer to multi-tenancy precisely because anything needing a second real cluster is out of scope — its syncer translates tenant pods into a host namespace, so "two views of one pod" is fully visible on one node. The one project that does not fit — Backstage — is cut for its build (yarn tsc OOMs at ~4GB), not its runtime, and turns out to be the one whose mechanism payload was thinnest.
The first lab step is a measurement¶
Exercise 1 opens with kubectl top pod before anything is installed — the empty-node baseline every later install is measured against, and the number that makes the two-group budget real rather than assumed. Exercise 12 opens with kubectl top pod -n vcluster-a — the single load-bearing measurement of the phase, because the two-tenant answer rests on the syncer's real RSS, not its 256Mi chart request (whose 4Gi default limit is a 16× tell that 256Mi is not the working set). If one tenant idles above ~600Mi, the exercise says to drop to one tenant plus a written walkthrough of the second.
Sizing is an exercise, not a footnote¶
resources: {} is a live hazard on a single 6.0GB node, and the build-track sizing standard — memory request from measurement, memory limit a written-down decision — applies to the platform components you install, not only to your own artifacts. It lands as three exercises with observable outcomes: the vcluster limit dropped 4Gi→1Gi, the Crossplane function pod sized by a mandatory DeploymentRuntimeConfig, and 12.C4, which removes that override and watches the mandatory pod run unbounded. The platform you build has the pod-sizing problem you spent P8 learning to see.
Exceptions and notes¶
- Maturity is stated out loud, not badged. KRO has no CNCF tier (a SIG subproject,
v1alpha1, its central CRD recently renamed), vcluster is vendor OSS (the chart's default image is the commercial build, soloft-sh/vcluster-ossis an explicit lab choice), and sealed-secrets is absent from the landscape entirely. The pattern is the lesson: platform tooling is markedly less mature than the layers below it. - Three stale levers to recognise, not try:
mode: Resourcesin a Crossplane Composition (removed from the schema, exercise 15); "use k3s in the vcluster, it's lighter" (the distro is gone); "ESO requires an external secret store" (itskubernetesprovider reads the local cluster, exercise 4). - Two exercises need no cluster. Backstage is a source clone on
forge; the capstone critique is a written document, though the platform it critiques must be running to critique honestly. - The word understand and the word know appear nowhere — in this directory or the phase. The whole phase turns on the difference between driving a tool and being able to say what it costs, which those words paper over.