Labs — Phase 11, synthesis¶
Twelve exercises in the order they are meant to run, for the phase where "exercise" changes meaning. Everywhere else a lab is a command with an observable outcome; here the central deliverable is a written artifact produced over days — one request, kubectl run nginx, traced from the terminal to a running container with a file:line citation at every seam that a hostile reader could clone k/k and check. No new subsystem, no new tool, no new area is opened. Every file below was read in an earlier phase; what is new is reading them in the order a single pod-create visits them, which is the order no single phase could show.
The unit is the seam, not the hop. A hop is one arrow — one file:line. A seam is a handoff between two areas, and it is the smallest thing that is a claim worth citing: the last frame in area N and the first frame in area N+1. The phase already decomposed the trace into three seams (A, B, C); this directory keeps that decomposition rather than exploding it into fifteen one-line files or collapsing it into one un-walkable page. Each seam is an artifact-to-produce — a cited sub-path — and the #35 template's build-artifact form fits it exactly: Artifact is the sub-path, Verify from outside is a stranger opening every cited line at your stated commit sha. A described seam fails the gate; a cited one passes.
This directory is mostly links. Because the trace crosses every earlier phase, each exercise's Rests on and Read point into the phases where those files were first read — P2's etcd read-back, P3's create.go chain, P5's from-scratch scheduler, P6's CRI calls, P7's CNI plugin, P0's syscalls. The mechanics live there and are not restated here; restating them would be the failure mode this phase exists to avoid. These files hold only the order, the citation discipline, and what you type to confirm each seam live.
Each file states one claim to test or one artifact to produce, links its topology rather than restating a footprint, and ends with a teardown that deletes what it created and says whether the topology stays or goes.
| # | Exercise | Why it exists |
|---|---|---|
| 1 | The map drawn from notes, before a single command | The prediction the live trace confirms. Where the map is wrong in red is where a phase did not land — found before any RAM is spent. |
| 2 | Seam A: kubectl to an etcd Txn, cited |
Two areas as one handoff, ending on the raw etcd key read back out directly. Admission strictly before durability. |
| 3 | 11.C1 — a create wedged at the admission line you cited | The ordering guarantee as an outage: a broken validator blocks durability, and no object reaches etcd. |
| 4 | Seam B: spec.nodeName empty to set, and the Binding |
The scheduler POSTs a Binding, not an in-place update — the second trip through Seam A, cited and watched live. |
| 5 | Seam C: kubelet to CRI to CNI to the syscalls | The loop closes on P0: the container the trace produces is the namespaces you once made by hand. |
| 6 | 11.C4 — kill one component, read the frame it owns | Pending when the bind can't happen, ContainerCreating when the netns can't be wired — the stall's name is the diagnostic. |
| 7 | Capstone 1 — the three sub-paths joined, terminal to container | One document, one sha, every seam a line a stranger can open. The gate the whole curriculum was built to reach. |
| 8 | The DNS talk, re-read for mechanism | The four-layer descent as a layer-to-phase table — the spine read backwards — ending on the one layer, if any, still unreadable. |
| 9 | 11.C3 — the conntrack table filling under a rollout | The DNS-talk mechanism on hardware you own: the direction of the table under churn, not the catastrophe. |
| 10 | 11.C2 — a botched rollout and the incident note | The failure mode most likely to bite during the upgrade, met first in isolation — recognised from the controller, rolled back, written up. |
| 11 | Capstone 2a — the upgrade that drops nothing | 0 5xx across a real minor-version upgrade, proven by the load generator's own log, draining one workhorse node at a time against a PDB. |
| 12 | Capstone 2b — scale under load, narrated by component | The trace you cited as source, now narrated as operation: metrics → HPA → Seam A → B → C → EndpointSlice, per-hop latency and the Pending frame when a node fills. |
Order, topology and the one switch¶
Exercises 1 and 8 need no cluster. The paper map is notes and source; the DNS postmortem is a talk and a table. Both are written artifacts, and both can be done with a cluster idle beside them or none at all.
One pair runs exercises 2 through 10. pair — 5.0GB, the daily-driver replay used by nearly every phase, nothing new — comes up at Seam A and carries the whole trace half plus the two conntrack/rollout drills. Two nodes are enough to watch the scheduler choose one, which is all the trace needs.
Then one switch, to workhorse, for the operations half. workhorse — 7.0GB, three nodes, already built once in P5 and torn down there, so this is a replay, not a new topology — comes up at the upgrade capstone and is destroyed by the scale capstone. Three nodes because a zero-downtime upgrade means draining one while the other two keep serving; two cannot demonstrate that honestly, and its 1-core workers are what make a Pending frame appear under real load. The switch is itself the point — the trace needs a pod, the upgrade needs a quorum-plus-spare — so it is folded into the upgrade capstone's setup rather than spent on a file of its own.
Exceptions and notes¶
- Capstone 1 sits mid-sequence, not last. This phase has two capstones — read the machine and operate it without stopping it — and the phase itself separates them. The trace capstone is placed the instant its three sub-paths are cited and fresh; the operations capstones close the directory.
- 11.C4 is one drill shown at two seams. The kill-a-component drill is referenced by three phase modules (11.1, 11.3, 11.4); rather than three near-identical files it is one exercise that stalls the pod at the scheduler frame and the CNI frame, because the single idea — the stall names the frame — is the same at both.
- Nothing new is installed. No ecosystem big rock, by design. The load generator and the metrics-server/HPA in the capstones are configured or are ordinary workloads, not new tools; this phase produces a trace and two write-ups, not a binary.
- No cross-phase lab links. Every reference to earlier work points at a phase file and anchor, never at another phase's
labs/— the mechanics are read where they were taught, and this directory only re-orders and cites them. - The three chaos drills and the botched-rollout rehearsal are by hand and stay that way. The manual-drills standard marks upgrade, rollout and drain permanently manual: the skill is recognising your own procedure's failure from the system's behaviour, and injecting the fault externally removes exactly that skill.