Then → Now: translating a kubernetes.io blog post to the pin¶
Resolves issue #54 (k3ii/k8s-academy), sub-issue of the MAP issue #52
(Blogwalk).
Research date: 2026-08-27. The pin: kubernetes/website@7c76070faf9b19e6a417c446043dbafd10a7aa1d
(2026-08-26). The newest Kubernetes at the pin is v1.37 "Garhwal", released 2026-08-26 —
the pin date itself.
What this is for¶
Every blogwalk/ exercise file has a section called "The diff, and why". This document is
its input. The blogwalk stance is that a stale post's staleness is the lesson: a post that
still works teaches you its own era, and a post that broke teaches you what changed and why.
That only holds if the why is right, so this is the table the authoring tickets read instead
of each re-deriving eleven years of release history and each getting it slightly differently.
Every row is shaped the same way:
| What the post says | What is true at the pin | Changed in | The decision behind it |
The fourth column is the one that matters. "It was replaced" is not an answer; the sentence has to name the pressure that moved it.
Read this before trusting any release number, including the ones here¶
Kubernetes' own CHANGELOGs and deprecation announcements misstate their own removals, in both directions. This was not a hypothesis going in — all three lines of research hit it independently, and it is the single most important methodological finding in this document:
CHANGELOG-1.28announces theazureFilein-tree plugin removal. That commit was reverted; the code is demonstrably present onrelease-1.28andrelease-1.29. Real removal was v1.30.kubectl run-container's removal was announced in v1.8. A bad conflict resolution in PR #50998 reinstated the alias two days later. It actually died in v1.13.- The dockershim removal was announced in the v1.20 blog as "currently planned for the 1.22 release". It landed in v1.24 (May 2022). Nearly every 2021 post carries 1.22 in good faith.
--exportwas announced for v1.18, removed in v1.19.kubectl convertwas announced for v1.17, removed in v1.20 — and was deprecated in v1.13, not the v1.14 the deprecation tables claim.
So a deprecation notice tells you the intent of a release, never the fact. Every removal
claim below was confirmed by reading the source tree at the release tag (gh api
repos/kubernetes/kubernetes/contents/...?ref=<tag>), not by trusting the note that announced
it. Authoring tickets should hold themselves to the same standard, and this is worth stating in
the exercise files themselves: the announcement is not the event.
This is the same failure mode strands/source-archaeology.md
describes for file paths — accurate when published, which is exactly why it is convincing now —
one level up, at the level of the changelog.
Confidence and gaps¶
198 rows across 14 tables. 192 fully verified against a primary source carrying a release
number. 6 rows carry an inline [UNVERIFIED] marker, and four further gaps are named in the
Traps prose of the part that found them. Nothing here is quoted from memory; where a release
number could not be confirmed it is marked rather than guessed, because a wrong version number
is worse than an admitted gap — it gets copied into an exercise file and read as fact.
Known gaps worth naming up front, so no authoring ticket rediscovers them:
- Bootstrap-token GA release — unconfirmed; CHANGELOGs 1.6→1.34 carry no promotion note. Last confirmed status is beta in v1.8.
- kubenet's removal release — no CHANGELOG names it. v1.24 is inferred structurally
(the package exists at
v1.23.0, 404s atv1.24.0). - FlexVolume has no removal release — it is still compiled in and registered at v1.37. That row asserts an absence, not a number.
resource.k8s.io/v1beta1's claimed v1.36 removal — no changelog confirms it.- The ScheduledJob → CronJob rename rationale — no primary source states it. The releases are verified; the why is not.
- v1.36 claims rest on
CHANGELOG-1.36plus arelease-1.35↔release-1.36tree diff; the v1.36 blog posts fetched truncated.
The three traps most likely to ruin a lab¶
Pulled forward out of the per-part Traps sections because each one silently succeeds:
--authorization-modestill defaults toAlwaysAllowat v1.37. The kube-apiserver reference says so verbatim. The 2015 posts are correct about the binary — only kubeadm saves you. Any exercise that starts a raw apiserver must pass--authorization-mode=Node,RBACexplicitly, or it teaches a cluster with no authorization at all and looks like it worked.- The API still accepts every dead volume type.
core/v1atrelease-1.37still declaresgcePersistentDisk,rbd,glusterfs,gitRepoand a dozen more. A 2015 manifest applies, the PV is created, the Pod is admitted and scheduled — and fails at mount time on a node, minutes later, somewhere the reader isn't looking. kubectl applyis still client-side by default at v1.37. Server-side apply is GA since v1.22 and its feature gate was deleted in v1.32, which reads like "it won" — but--server-sidedefaults tofalseand thelast-applied-configurationannotation a 2016 post describes is still being written to your objects today. Here the old post is more accurate than a 2021 one.
Part 1 — API surface, workload objects, kubectl¶
What a manifest and a command line looked like, and what they must become.
API surface¶
The v1beta1 / v1beta2 / v1beta3 era and the move to core v1¶
| What the post says | What is true at the pin | Changed in | The decision behind it |
|---|---|---|---|
apiVersion: v1beta1 (or v1beta2) with id, desiredState, currentState, /minions, labels= query param |
Nothing serves these. Core objects are apiVersion: v1; id→metadata.name, desiredState→spec, currentState→status, /minions→/nodes, labels=→labelSelector=, fields=→fieldSelector= |
v1beta1 and v1beta2 were deleted 2015-06-01 — PR #8917 "Delete deprecated apis", listed in the v0.19.0 release notes. Date stated in docs/api.md @ v1.0.0: "The v1beta1 and v1beta2 APIs were deleted on June 1, 2015." | v1beta1/2 had no spec/status split: desired and observed state were intermingled in one flat object, so the reconciliation contract that the whole control plane is built on was not expressible in the wire format. v1beta3 restructured every object around metadata/spec/status so that "what I asked for" and "what is" are separately readable and separately writable. |
apiVersion: v1beta3 — Pods/RCs/Services with spec/status, service.spec.portalIP, pod.spec.host, createExternalLoadBalancer |
apiVersion: v1. portalIP→clusterIP, pod.spec.host→pod.spec.nodeName, createExternalLoadBalancer: true→spec.type: LoadBalancer, publicIPs→spec.type: NodePort, containers[*].privileged/capabilities moved under containers[*].securityContext, state.termination→state.terminated, pod.status.Condition→pod.status.conditions |
v1beta3 was enabled and made the default API version in v0.15.0 (PR #6098, in the v0.15.0 notes). v1 was created in v0.16.0 by literally cloning v1beta3 (PR #7454 "Cloning v1beta3 as v1 and exposing it in the apiserver"). v1 enabled by default 2015-06-04, v0.19.0 (PR #9269 "Make v1 enabled by default"). v1.0.0 stopped exposing v1beta3 by default (PR #10525, in the v1.0.0 notes); the code was deleted in the v1.1 cycle (commit 1c3233a "Remove v1beta3", 2015-07-23, confirmed an ancestor of v1.1.1 by gh api .../compare/1c3233a1...v1.1.1 → ahead). Field-by-field list: docs/api.md @ v1.0.0 §"v1 conversion tips (from v1beta3)" |
v1 is v1beta3 with a rename pass, not a redesign — which is exactly the point. The pre-1.0 plan ("Kube 1.0 should have API v1 without v1beta* API versions") was to ship the final shape before 1.0 so users could migrate their stored objects off beta while still on a 0.x release, letting 1.0 drop the beta versions "with no effect". The beta window existed to absorb the renames; v1 froze them. |
"the Kubernetes API at /api/v1beta3" |
/api/v1 for the core group; every other group lives at /apis/<group>/<version> |
Group-versioned URL layout landed with API groups in v1.1 — see A3 | Per the multiple-API-groups proposal: "for /api/v1, we set the prefix=api, group="", version=v1, so the URL remains to be /api/v1." The core group kept the legacy path deliberately, "so existing v1 config files will remain valid" — which is why core objects to this day carry a bare apiVersion: v1 with no group name. |
Why the API is versioned at all, and the alpha/beta/GA conventions¶
| What the post says | What is true at the pin | Changed in | The decision behind it |
|---|---|---|---|
| "the API is in beta, expect changes" / no stability contract stated | A written, enforced policy: alpha = v1alpha1 (may vanish any release, no notice); beta = v1beta1 (deprecated ≤ 9 months or 3 minor releases after introduction, then served for a further 9 months or 3 minor releases, whichever is longer); GA = v1 (not removed within a major version) |
At v1.0 there was no policy: docs/api.md @ v1.0.0 says "The precise deprecation policy for eliminating features is TBD, but once we reach our 1.0 milestone, there will be a specific policy." The numbered-rule document was first published 2017-01-03 (kubernetes/website PR #1856 "Add deprecation policy", original path docs/deprecation-policy.md) — i.e. the v1.5-era docs. Current text: Deprecation Policy (Rules #1, #3, #4a, #4b) and api_changes.md §"Alpha, Beta, and Stable Versions" |
Rule #1 — "API elements may only be removed by incrementing the version of the API group" — is the whole reason versions exist. Objects are serialised in versioned form into etcd, so a field's meaning can never be changed in place; the only safe way to restructure is to stand up a new group-version, convert through an internal representation, and retire the old one on a published clock. |
A YAML uses a v1beta1 group-version and the post says "just enable it" / assumes it is on |
New beta APIs are off by default and must be turned on with --runtime-config. Beta APIs that existed before v1.24 kept their old default |
v1.24 — KEP-3136 "Beta APIs Are Off by Default", kep.yaml: status: implemented, stage: stable, milestone.stable: "v1.24". Also stated in api_changes.md: "beta APIs introduced before v1.24 were enabled by default, but this changed for new beta APIs" |
Beta-on-by-default made beta a de-facto GA: production users adopted v1beta1 because it was simply there, and then the 9-month removal clock became a breaking change for them. Turning it off forces an explicit opt-in, which is what "beta" was always supposed to mean. The companion KEP is 1635 Prevent PermaBeta. |
| Post issues a request against a deprecated version and nothing complains | The apiserver returns an RFC7234 Warning header, annotates the audit event "k8s.io/deprecated":"true", and increments apiserver_requested_deprecated_apis |
v1.19 — Deprecation Policy §"deprecated API versions" | Silence is why the v1.16 wave hurt. Making deprecated use observable — client-visible warning, cluster-visible metric, audit-visible annotation — turns a surprise at upgrade into a dashboard you can act on months earlier. |
Group-version removals, with the release that removed each¶
Authoritative source for every row: Deprecated API Migration Guide.
| What the post says | What is true at the pin | Changed in | The decision behind it |
|---|---|---|---|
apiVersion: extensions/v1beta1 with kind: Deployment / DaemonSet / ReplicaSet |
Not served. Use apps/v1 (available since v1.9) |
v1.16 — deprecation-guide §v1.16. Announced two releases earlier in the v1.14 CHANGELOG: "DaemonSet, Deployment, and ReplicaSet resources will no longer be served from extensions/v1beta1, apps/v1beta1, or apps/v1beta2 in v1.16." Removal PR trail via PR #74057 et al. |
extensions was a catch-all staging area ("the experimental group"), not a coherent group. Once each workload kind had a real home (apps, networking.k8s.io, policy, batch, autoscaling), keeping a second serving path for the same object meant two schemas, two defaulting tables and two sets of validation for one stored object — so the staging group was retired rather than maintained. |
apiVersion: apps/v1beta1 or apps/v1beta2 (Deployment, StatefulSet, ReplicaSet, DaemonSet, ControllerRevision) |
Not served. apps/v1 only. Verified at v1.37.0: pkg/registry/apps/rest/storage_apps.go registers exactly one version — appsapiv1.SchemeGroupVersion.Version (source @ v1.37.0) |
v1.16 — deprecation-guide §v1.16 | apps/v1beta1 and v1beta2 were the two iterations it took to settle the workload schema (required immutable selector, RollingUpdate default, bounded revision history). Rule #4b requires shipping the new and old version together for at least one release before advancing the storage version; once apps/v1 had done that in v1.9, the betas were pure conversion debt. |
apiVersion: extensions/v1beta1, kind: NetworkPolicy |
Not served. networking.k8s.io/v1 (available since v1.8) |
v1.16 — deprecation-guide §v1.16 | Same rehoming logic: NetworkPolicy belongs to the networking group, not to extensions. |
apiVersion: extensions/v1beta1, kind: PodSecurityPolicy |
Nothing serves PSP at all. Use Pod Security Admission or a third-party webhook | Two-step: v1.16 removed extensions/v1beta1 PSP (migrate to policy/v1beta1, available since v1.10); v1.25 removed policy/v1beta1 PSP entirely — deprecation-guide §psp-v116 and §v1.25 |
A 2015–2018 post's PSP example is doubly dead: first the group-version moved, then the object was deleted outright. PSP was replaced rather than re-versioned because its authorization model (which PSP a pod "uses" was resolved implicitly per-service-account at admission) was not fixable within a schema change. |
apiVersion: extensions/v1beta1, kind: Ingress (with spec.backend, serviceName, servicePort) |
Not served. networking.k8s.io/v1 with spec.defaultBackend, backend.service.name, backend.service.port.number/.name, and pathType is now required on every path |
v1.22 — deprecation-guide §ingress-v122. Confirmed in the v1.22 CHANGELOG: "A number of APIs are no longer serving specific Beta versions in favour of the GA version… includes beta versions of … Ingress, and IngressClass APIs." Note the timeline slipped: the v1.14 CHANGELOG said v1.18, the v1.16 CHANGELOG said v1.20, and it actually happened in v1.22 |
pathType becoming required is the load-bearing change: v1beta1 left path matching semantics undefined, so the same Ingress meant different things on nginx, GCE and Traefik. v1 forces the author to say Prefix, Exact, or ImplementationSpecific — the last of which is the only value that reproduces the old ambiguity. |
apiVersion: apiextensions.k8s.io/v1beta1 CRD (with spec.version, spec.validation, spec.subresources, no schema) |
apiextensions.k8s.io/v1: spec.versions[*], spec.versions[*].schema, spec.scope required, openAPIV3Schema required and must be a structural schema, spec.preserveUnknownFields: true disallowed, JSONPath→jsonPath |
v1.22 — deprecation-guide §customresourcedefinition-v122 | A schemaless CRD cannot be pruned, defaulted, converted, or server-side-applied — the apiserver has no idea which fields are real. Making the structural schema mandatory in v1 is what unlocked field pruning and managedFields for custom resources. |
admissionregistration.k8s.io/v1beta1 webhook config |
admissionregistration.k8s.io/v1, and the defaults inverted: failurePolicy Ignore→Fail, matchPolicy Exact→Equivalent, timeoutSeconds 30s→10s, sideEffects and admissionReviewVersions now required |
v1.22 — deprecation-guide §webhook-resources-v122 | failurePolicy: Ignore by default meant a security webhook silently failed open. matchPolicy: Exact meant a webhook registered for apps/v1 Deployments never saw the same object submitted as apps/v1beta1. Both defaults were footguns; v1 chose the safe side and made the ambiguous fields explicit. |
Other */v1beta1 in a 2016–2019 post: rbac.authorization.k8s.io/v1beta1, storage.k8s.io/v1beta1, certificates.k8s.io/v1beta1, coordination.k8s.io/v1beta1, scheduling.k8s.io/v1beta1, apiregistration.k8s.io/v1beta1, authentication.k8s.io/v1beta1, authorization.k8s.io/v1beta1 |
All …/v1. Notable field renames: SubjectAccessReview spec.group→spec.groups; CSR spec.signerName and spec.usages now required, and kubernetes.io/legacy-unknown may not be requested via v1 |
v1.22 — deprecation-guide §v1.22 | v1.22 was the "beta cleanup" release: every long-stable API that still had a vestigial beta alias lost it in one wave, so operators paid the migration cost once instead of eight times. |
apiVersion: batch/v1beta1, kind: CronJob |
batch/v1 (available since v1.21) |
v1.25 — deprecation-guide §v1.25 | CronJob sat at beta from v1.8 to v1.21 — the canonical "permabeta". It graduated in v1.21 with a rewritten controller and the beta alias was retired on the standard clock. |
apiVersion: policy/v1beta1, kind: PodDisruptionBudget |
policy/v1 (available since v1.21) |
v1.25 — deprecation-guide §v1.25 | Same wave; policy/v1beta1 was emptied out by v1.25 (PDB moved to policy/v1, PSP deleted). |
apiVersion: autoscaling/v2beta1 or v2beta2, kind: HorizontalPodAutoscaler |
autoscaling/v2 (available since v1.23). autoscaling/v1 is still served but only expresses CPU-target HPAs |
v2beta1 removed in v1.25, v2beta2 removed in v1.26 — deprecation-guide §v1.25 and §v1.26 |
HPA v2 took three betas to settle the metrics shape (object/pods/external metrics, then scaling behaviour/stabilisation windows). The betas were staggered a release apart on removal so a cluster never lost both migration targets at once — Rule #4b in practice. |
discovery.k8s.io/v1beta1 EndpointSlice, events.k8s.io/v1beta1 Event, node.k8s.io/v1beta1 RuntimeClass |
discovery.k8s.io/v1 (since v1.21), events.k8s.io/v1 (since v1.19), node.k8s.io/v1 (since v1.20) |
All v1.25 — deprecation-guide §v1.25 | Same v1.25 sweep. |
flowcontrol.apiserver.k8s.io/v1beta1 / v1beta2 / v1beta3 FlowSchema, PriorityLevelConfiguration |
flowcontrol.apiserver.k8s.io/v1 (available since v1.29) |
v1beta1 removed v1.26, v1beta2 removed v1.29, v1beta3 removed v1.32 — deprecation-guide §v1.26, §v1.29, §v1.32 |
Three betas in six releases: API Priority and Fairness kept changing how borrowing/lending of concurrency shares was expressed, and each incompatible reshape legally required a new beta rather than an in-place edit. |
storage.k8s.io/v1beta1 CSIStorageCapacity |
storage.k8s.io/v1 (since v1.24) |
v1.27 — deprecation-guide §v1.27 | Standard beta clock. |
| — (nothing an old post would say) | flowcontrol…/v1beta3 in v1.32 is the most recent removal the migration guide documents. Nothing in the guide is dated v1.33–v1.37. Two alpha-only removals happened later: storage.k8s.io/v1alpha1 VolumeAttributesClass (v1.35, PR #134625) and StorageVersionMigration v1alpha1 (v1.35, replaced by v1beta1); and kubeadm's own config API v1beta3 was removed in v1.37 (PR #136016), deprecated since v1.31, with a placeholder experimental v1 added |
v1.33–v1.37, verified by grepping CHANGELOG-1.33 … CHANGELOG-1.37 for "no longer served" / "removed … API", and by re-reading the migration guide source (website deprecation-guide.md, whose newest section heading is ### v1.32) |
Alpha versions are exempt from the support window (Rule #4a: "may be removed in any release without prior deprecation notice"), which is why they never appear in the migration guide even though they do disappear. |
resource.k8s.io/v1beta1 (DRA, in a 2024–2025 post) |
[UNVERIFIED] at the pin. The v1.33 CHANGELOG says "The resource.k8s.io/v1beta1 API is deprecated and will be removed in 1.36" (PR #129970), but I found no removal note in the v1.36 or v1.37 CHANGELOGs, and the Go type package staging/src/k8s.io/api/resource/v1beta1 still exists at ref=v1.37.0 (a package's presence does not prove it is served). Do not assert either way in an exercise. |
announced for v1.36; actual outcome unconfirmed | Out of scope for workload/kubectl exercises; recorded so nobody repeats the announcement as fact. |
What a 2015 post's YAML must literally become¶
apiVersion: v1beta1/v1beta2→apiVersion: v1, and restructure:id→metadata.name; labels/annotations intometadata;desiredState→spec;currentState→status;entrypoint→commandand oldcommand→args;hostDir→hostPath;restartPolicy: {always: {}}→restartPolicy: Always;imagePullPolicy: PullAlways→Always; resources asresources: {limits: {cpu: ...}}with suffixes instead of milli-scaled scalars; volume source inlined into the volume. (docs/api.md @ v1.0.0 §"v1beta3 conversion tips")apiVersion: v1beta3→apiVersion: v1, and rename:service.spec.portalIP→clusterIP;createExternalLoadBalancer: true→spec.type: LoadBalancer;publicIPs→spec.type: NodePort;pod.spec.host→pod.spec.nodeName;privileged/capabilities→ undersecurityContext. (same doc, §"v1 conversion tips (from v1beta3)")apiVersion: extensions/v1beta1+kind: Deployment|ReplicaSet|DaemonSet→apiVersion: apps/v1, and addspec.selector(required and immutable), and accept the new defaults:maxSurge/maxUnavailable1→25%,revisionHistoryLimitunbounded→10,progressDeadlineSecondsnone→600, DaemonSetupdateStrategy.typeOnDelete→RollingUpdate. Deletespec.rollbackToandspec.templateGeneration— they do not exist inapps/v1. (deprecation-guide §v1.16)apiVersion: extensions/v1beta1+kind: Ingress→apiVersion: networking.k8s.io/v1, restructure the backend, and addpathTypeto every path.
Traps — API surface¶
- Old YAML that still parses but is rejected for a different reason.
apiVersion: extensions/v1beta1does not fail with "unknown field"; it fails at the discovery layer —no matches for kind "Deployment" in version "extensions/v1beta1". Readers conclude their kubectl is broken or their cluster is missing a CRD. Theextensionsgroup has no registry directory at all at v1.37.0 (gh api repos/kubernetes/kubernetes/contents/pkg/registry?ref=v1.37.0lists noextensions), so there is no--runtime-configflag that brings it back — unlike the v1.14–v1.15 era, when PR #72249 explicitly allowed per-resource enablement insideextensions/v1beta1and the v1.16 CHANGELOG documented--runtime-config=extensions/v1beta1/deployments=trueas a temporary escape hatch. A post from that window tells you the escape hatch exists. It doesn't any more. - The Go packages still exist.
staging/src/k8s.io/api/extensions/v1beta1,apps/v1beta1andapps/v1beta2are all still present in the tree atref=v1.37.0. Code that importsk8s.io/api/apps/v1beta1compiles fine and only fails at runtime against a real apiserver. A reader who "verifies" an old client-go snippet by building it learns nothing. - YAML that validates and means something different. An
extensions/v1beta1Deployment with nospec.selectorwas legal — the selector was defaulted from the pod template labels. Inapps/v1the selector is required and immutable. A mechanicalapiVersionbump therefore either fails validation, or, if the author "fixes" it by pasting in a selector that differs from the template labels, produces a Deployment whose ReplicaSets adopt nothing. - Defaults changed silently under the same field names. A post that says "rolling update replaces one pod at a time" was describing
extensions/v1beta1'smaxUnavailable: 1/maxSurge: 1. The identical YAML underapps/v1gets25%/25%. On a 100-replica Deployment that is a 25-pod-wide rollout, not a 1-pod-wide one. Same forrevisionHistoryLimit(all → 10) and DaemonSetupdateStrategy(OnDelete→RollingUpdate— an old DaemonSet post that says "pods are only replaced when you delete them" is now describing an automatic rollout). - Announced removal releases slipped. Ingress
extensions/v1beta1was announced for v1.18 (in the v1.14 CHANGELOG), then v1.20 (v1.16 CHANGELOG), and actually went in v1.22. A post from 2019 quoting "removed in 1.18" is citing a plan, not the outcome. Only the migration guide records what happened. /api/v1vs/apis/<group>/<version>is not a typo. Posts that construct URLs by hand break on the core group, which has an empty group name and the legacy/apiprefix by deliberate design (api-group.md).- A beta version being "available" is not the same as it being on. From v1.24 (KEP-3136), a newly introduced beta API is not served unless enabled. A post that says "just apply this
v1beta1manifest" may be correct about the API existing and wrong about your cluster serving it — and the error looks identical to the removed-API error above. - The migration guide is silent about alpha. Nothing in it covers
v1alpha*, yet alpha versions do get deleted (storage.k8s.io/v1alpha1VolumeAttributesClass in v1.35). A reader who checks the guide and finds nothing concludes their alpha manifest is still fine.
Workload objects¶
ReplicationController → ReplicaSet → Deployment¶
| What the post says | What is true at the pin | Changed in | The decision behind it |
|---|---|---|---|
kind: ReplicationController is how you run N copies of a pod |
ReplicationController still exists and is still served as apiVersion: v1 — it is not deprecated in any machine-readable sense. Verified at v1.37.0: pkg/registry/core/replicationcontroller exists (tree); RC's type carries only +k8s:prerelease-lifecycle-gen:introduced=1.0 with no deprecated=/removed= marker; it appears nowhere in the migration guide. Docs call it "Legacy API… Superseded by the Deployment and ReplicaSet APIs" and note "A Deployment that configures a ReplicaSet is now the recommended way to set up replication" — but never the word deprecated |
Nothing. RC has survived every removal wave. Discouraged from v1.2 onward (when ReplicaSet + Deployment arrived), and RC was still being actively maintained in v1.37 (its ObjectMeta gained declarative-validation markers +k8s:beta(since: "1.37")). Docs: ReplicationController concept page |
RC is a correct object with a weak selector. Removing it would break stored objects and buy nothing (Rule #1 forbids changing v1 in place anyway), so it was left served and editorially demoted instead. Say "legacy, still works, don't write new ones" — not "deprecated". |
spec.selector: {app: nginx} — a flat map — is how you select pods |
apps/v1 uses a LabelSelector: spec.selector.matchLabels and/or spec.selector.matchExpressions (In, NotIn, Exists, DoesNotExist). RC still uses the flat map |
v1.2 introduced ReplicaSet at extensions/v1beta1 (PR #19042 "Introducing ReplicaSet, a new incarnation of Replication Controller", issue #3024) |
Verbatim, CHANGELOG-1.2: "The new ReplicaSet API (Beta) in the Extensions API group is similar to ReplicationController, but its selector is more general (supports set-based selector; whereas ReplicationController only supports equality-based selector)." A rollout needs to say "all pods of this app except the current revision" — that is a NotIn expression, and it is simply not expressible in map[string]string. |
A rolling update is something you run: kubectl rolling-update foo foo-v2 --image=… |
Rollout is a server-side property of a Deployment. You edit the Deployment; the deployment controller creates a new ReplicaSet, scales the old one down, and records the revision. You observe it with kubectl rollout status and reverse it with kubectl rollout undo |
Deployment introduced v1.1 (alpha, extensions/v1beta1), beta v1.2; kubectl rolling-update removed in v1.18 (PR #88057, CHANGELOG-1.18: "The deprecated command kubectl rolling-update has been removed") |
The old rollout lived in the client. From docs/design/simple-rolling-update.md: "Whenever the rolling update command executes, the kubectl client looks for replication controllers called foo and foo-next"; crash recovery was smuggled into annotations desired-replicas and update-partner; and the doc concedes "Obviously there is a race here, where if you kill the client between delete foo, and creating the new version of 'foo' you might be surprised about what is there." A rollout that only exists while a laptop process is alive cannot be audited, resumed by anyone else, or driven by a CI job — so it was moved into a controller. The Deployment proposal states the goal ("declarative config updates") and the property that makes it work: "DeploymentController is stateless so that it can recover in case it crashes during a deployment." |
| (a 2015 post) "a Deployment spins up a ReplicationController" | A Deployment owns ReplicaSets, one per pod-template revision, distinguished by the pod-template-hash label |
v1.2 — PR #19840 "Move Deployments to ReplicaSets and switch the Deployment selector to the new LabelSelector". The original proposal genuinely said RC: "Users will be able to create a Deployment, which will spin up a ReplicationController to bring up the desired pods" (proposal @ v1.2.0) | Deployment needs matchExpressions to talk about "every revision of this app", so its child had to be the object with the expressive selector. This is the actual controller split: ReplicaSet owns replica count; Deployment owns revision history and rollout policy. Two controllers, two levels, one stored template hash joining them. |
apiVersion: extensions/v1beta1, kind: Deployment |
apiVersion: apps/v1 |
Group-version walk: v1.1 extensions/v1beta1 (alpha) → v1.2 beta → v1.6 apps/v1beta1 (PR #39683, "Introduce apps/v1beta1.Deployments resource with modified defaults") → v1.8 apps/v1beta2 (PR #48746) → v1.9 GA apps/v1 (PR #53679, CHANGELOG-1.9: "The core workloads API, which is composed of the DaemonSet, Deployment, ReplicaSet, and StatefulSet kinds, has been promoted to GA stability in the apps/v1 group version"). Old versions stopped being served in v1.16 |
Each hop existed to change a default or a constraint, which Rule #1 forbids doing in place. apps/v1beta1 was created specifically to carry different defaults from extensions/v1beta1; v1beta2 made the selector required and non-defaulted; v1 froze it. |
| — | v1.9 also flipped cascading delete at the REST level: "The default garbage collection policy for Deployment, DaemonSet, StatefulSet, and ReplicaSet has changed from OrphanDependents to DeleteDependents when the deletion is requested through an apps/v1 endpoint" |
v1.9 — PR #55148 | Orphaning by default meant deleting a Deployment left its ReplicaSets and Pods running — surprising to everyone and a common source of "ghost" pods. |
spec.rollbackTo: {revision: 2} / POST to the rollback subresource |
Neither exists. apps/v1 has no RollbackTo field and no DeploymentRollback kind. kubectl rollout undo is now a client-side PATCH: it reads the target revision's ReplicaSet, strips pod-template-hash, and patches the Deployment's pod template (polymorphichelpers/rollback.go) |
Deprecated in v1.8 (PR #49340); never carried into apps/v1, so effectively removed in v1.9 and unserveable from v1.16. Listed under v1.16 notable changes: "spec.rollbackTo is removed" (deprecation-guide §deployment-v116) |
A write-only "do a thing" field inside a spec is not declarative — kubectl apply of the same manifest would re-trigger the rollback forever, and the field's value was meaningless after the fact. Rollback was re-expressed as "set the spec back to what it was", which apply and GitOps can both represent. |
PetSet → StatefulSet, DaemonSet, Job/CronJob, sidecars¶
| What the post says | What is true at the pin | Changed in | The decision behind it |
|---|---|---|---|
apiVersion: apps/v1alpha1, kind: PetSet |
apiVersion: apps/v1, kind: StatefulSet |
Alpha as PetSet in v1.3 (apps/v1alpha1; verified in pkg/apis/apps/v1alpha1/register.go @ v1.3.0); renamed and promoted to beta in v1.5 (PR #35663 rename, PR #35731 "Move Statefulset (previously PetSet) to v1beta1"); GA apps/v1 in v1.9 (PR #53679) |
Rationale is in issue #27430 "Please consider changing the name of PetSet before General Availability". The name was always a placeholder — thockin: "We went with PetSet as a working name in part because we knew it could never be the 'real' name… Names hold power." The pets-vs-cattle metaphor drew two independent objections: it could read as offensive (erictune: "I don't think Kubernetes should offend even a small minority if it can reasonably be avoided") and it mis-sold the feature (pwittrock: it "implies these things are going to be a lot of work to maintain"). "Set" is technically wrong — bgrant0607: "This is not a Set. The members are ordered and indexed… it's an Array" — but consistency won (alex-mohr: "everything else is a Set, so I think departing from that convention adds more friction"). Settled by a ranked-choice community vote; erictune's postmortem: "I don't think voting turned out to be a good way to do that." |
| "upgrade your cluster and your PetSets become StatefulSets" | They don't. apps/v1alpha1 was deleted outright; the v1.5 notes say "If you have existing PetSets, you must perform extra migration steps both before and after upgrading" |
v1.5 — CHANGELOG-1.5 §Action Required | Alpha versions carry no upgrade guarantee (Deprecation Policy Rule #4a: alpha "may be removed in any release without prior deprecation notice"). erictune, on #27430: "A key purpose of alpha is so that developers do not have to do headstands when they make a change that is difficult to make backward compatible." |
| A DaemonSet's pods are replaced "only when you delete them" / you edit the template and delete pods by hand | DaemonSet rolls itself out. spec.updateStrategy.type defaults to RollingUpdate in apps/v1 |
DaemonSet: alpha v1.1, beta v1.2 (both extensions/v1beta1); rolling update added in v1.6 (CHANGELOG-1.6: "[beta] Introduce the rolling update feature for DaemonSet"); apps/v1beta2 in v1.8; GA apps/v1 in v1.9. Default flip verified in code: extensions/v1beta1/defaults.go sets OnDeleteDaemonSetStrategyType; apps/v1/defaults.go sets RollingUpdateDaemonSetStrategyType |
Before v1.6 there was no server-side DaemonSet update mechanism, so OnDelete was the only honest default. Once one existed, the new group-version took the useful default. The v1.8 notes are explicit that this is a version-boundary change with no fixup: "The default spec.updateStrategy for the StatefulSet and DaemonSet kinds is RollingUpdate for the apps/v1beta2 group version… and no strategy auto-conversion is applied to replace default values." |
apiVersion: extensions/v1beta1, kind: Job, with an explicit spec.selector |
apiVersion: batch/v1, and you should omit spec.selector — a unique one is generated |
Job: beta v1.1 (extensions/v1beta1), GA batch/v1 in v1.2. CHANGELOG-1.2: "Job was Beta in 1.1 and is GA in 1.2. apiVersion: batch/v1 is now available. You now do not need to specify the .spec.selector field — a unique selector is automatically generated for you." extensions/v1beta1 Job removed in v1.6 (PR #38614, deprecated by PR #36355) |
A hand-written Job selector is a footgun: two Jobs from the same template share a selector, so each adopts the other's pods and neither ever completes. Auto-generating a UID-based selector removed the class of bug. This is the one workload object where writing less YAML is the correct migration. |
apiVersion: batch/v2alpha1, kind: ScheduledJob |
apiVersion: batch/v1, kind: CronJob |
ScheduledJob alpha v1.4 (PR #25816); renamed to CronJob in v1.5 (PR #36021; CHANGELOG-1.5: "batch/v2alpha1.ScheduledJob has been renamed, use batch/v2alpha1.CronJob instead"); beta batch/v1beta1 v1.8 (PR #51465), and batch/v2alpha1.ScheduledJob removed the same release (PR #49930); GA batch/v1 in v1.21 (PR #99423); batch/v1beta1 removed v1.25 (deprecation-guide §v1.25) |
The rename itself: [UNVERIFIED] rationale. PR #36021 is a mechanical rename with no stated reason, and neither the v1.4 nor v1.5 CHANGELOG explains it. (Tried: PR #36021 body, both CHANGELOGs, docs/proposals/scheduledjob.md.) Do not put a reason in an exercise. What is verifiable is the shape of the fix: "Cron" names the syntax in the schedule field, so the kind and its field agree; and CronJob spent v1.8 → v1.21 at beta, the longest permabeta in the workloads API, graduating only after the controller was rewritten. |
| Sidecars are "a second container in the pod" (with an initContainer for setup) | Sidecars are a first-class API: an entry in initContainers with restartPolicy: Always. It starts before app containers, keeps running for the pod's life, and does not block pod termination or Job completion |
Alpha v1.28 (SidecarContainers gate, PR #116429); beta and on by default v1.29 (PR #121579); GA v1.33 (PR #129731 — "The SidecarContainers feature has graduated to GA"); the escape-hatch gate LegacySidecarContainers was removed in v1.34 (PR #131463); the SidecarContainers gate itself was removed from the codebase in v1.37 (PR #137755, CHANGELOG-1.37: "unconditionally enabled since v1.33, is removed"). KEP-753 kep.yaml: status: implemented, stage: stable, milestone: {alpha: v1.28, beta: v1.29, stable: v1.33} |
A plain second container has no defined ordering relative to the app container and no defined shutdown ordering. That broke two things concretely: a service-mesh proxy that starts after the app misses the app's first outbound calls, and a log-shipping sidecar that never exits keeps a Job's pod Running forever. Encoding "runs for the whole pod lifetime, starts first, terminates last, doesn't count for completion" as a real API field is what makes both cases correct without an in-pod supervisor. Status at the pin: GA, unconditional, no gate to enable. |
Traps — workload objects¶
- "ReplicationController is deprecated" is wrong at the pin, and the correction matters. RC is served, unmarked in the prerelease-lifecycle generator, absent from the migration guide, and was still receiving code changes in v1.37. An exercise that tells a learner "this API is deprecated, migrate off it" is teaching a false fact; the true statement is "legacy, superseded, still fully supported, no removal scheduled". Meanwhile the thing that is gone —
extensions/v1beta1Deployment — is the object the 2015 post probably reached for as the modern alternative. - The 2015 Deployment posts describe a Deployment managing ReplicationControllers. That was genuinely true in the original proposal and stopped being true inside the same release cycle (PR #19840). A reader who follows the post will look for RCs under
kubectl get rcafter creating a Deployment, find nothing, and conclude the Deployment failed. kubectl rolling-updatestill appears to exist in muscle memory but is gone since v1.18 — and its replacement is not a command that does the rollout, it is a command that watches one (kubectl rollout status). The conceptual move — from "run the rollout" to "declare the target and observe convergence" — is the actual lesson, and a post that predates it teaches the opposite mental model.- Defaults changed under identical field names, with no conversion.
maxSurge/maxUnavailable1→25%,revisionHistoryLimitunbounded→10,progressDeadlineSecondsnever→600, DaemonSetupdateStrategyOnDelete→RollingUpdate. A post that says "one pod at a time" or "DaemonSet pods are only replaced when deleted" is factually describingextensions/v1beta1behaviour while the reader's YAML now saysapps/v1. Nothing errors; the rollout is just wider and faster than described. (Driving issues: #23597 unbounded ReplicaSet accumulation, #23304 single-replica downtime.) spec.selectorflipped from "helpfully defaulted" to "required and immutable".extensions/v1beta1defaulted it from the pod template labels;apps/v1does not, andValidateDeploymentUpdate/ ReplicaSet / DaemonSet / StatefulSet validation all callValidateImmutableFieldon it. Reason from CHANGELOG-1.8: "aspec.selectordefault value is no longer available, because it's incompatible withkubectl applyand strategic merge patch." A learner who "fixes" the validation error by inventing a selector that doesn't match the template gets a Deployment stuck at 0 available replicas with no error message.- PetSet→StatefulSet is not a rename you can apply to a live object. The v1.5 notes require migration steps before and after upgrading; there was no in-place conversion because
apps/v1alpha1was deleted. A post that says "PetSet is now called StatefulSet, just update the kind" understates it. - Job's migration is to delete YAML, not add it. Every other workload object gained a required
spec.selector; Job lost the recommendation to write one. Copying aspec.selectorout of a 2015 Job post into abatch/v1Job produces two Jobs that fight over each other's pods. - The sidecar feature gate is gone, which cuts both ways. From v1.34 the
LegacySidecarContainersescape hatch no longer exists (PR #131463) and from v1.37 theSidecarContainersgate itself is gone (PR #137755). A 2023–2024 post that says "enable theSidecarContainersfeature gate" describes a flag that will now cause the component to fail to start on an unknown flag, not a no-op. Also note v1.33's own changelog predicted the gate's removal in v1.36; it actually happened in v1.37 — cite the removal, not the prediction. kubernetes.io/created-byis gone (v1.9, PR #54445). Posts that teach "find the controller that made this pod" via that annotation now find nothing; usemetadata.ownerReferenceswithcontroller: true.
kubectl¶
kubectl run, the generator era, and imperative → declarative¶
| What the post says | What is true at the pin | Changed in | The decision behind it |
|---|---|---|---|
kubectl run-container nginx --image=nginx |
kubectl run nginx --image=nginx |
Renamed in v0.18.0 (PR #8596, "Kubectl command renaming (run-container to run and resize to scale)", in the v0.18.0 release notes); the hidden run-container alias was removed in v1.13 (PR #70728: "The deprecated command run-container has been removed. Invocations should use kubectl run instead.") |
The command never only ran containers — it created a controller — so the noun in the name was wrong and got shortened to the verb. Removal rationale from the alias-cleanup PR (#49935): "These command aliases have been deprecated for more than two years. It's time to remove them." |
kubectl run nginx --image=nginx --replicas=3 gives you a replicated app (an RC, or later a Deployment) |
kubectl run creates exactly one Pod. There is no --replicas. To get a Deployment you use kubectl create deployment or kubectl apply -f |
Three distinct eras: run created a ReplicationController v0.18–v1.1; a Deployment from v1.2 (CHANGELOG-1.2: "'kubectl run' now produces Deployments (instead of ReplicationControllers) and Jobs (instead of Pods) by default"), moving to apps/v1 Deployments in v1.13 (PR #71006); a bare Pod from v1.18 (PR #87077, "kubectl run has removed the previously deprecated generators, along with flags unrelated to creating pods. kubectl run now only creates pods."). --replicas was made inert in v1.18 (same PR) and removed in v1.21 (PR #99732: "Remove deprecated --generator, --replicas, --service-generator, --service-overrides, --schedule from kubectl run") |
kubectl run had become a second, undocumented object model: its output kind depended on --restart, on --generator, and on API discovery against your cluster, so the same command produced different objects on different clusters. Collapsing it to "one image, one Pod" made it a debugging tool with one meaning, and pushed durable objects into create/apply where they are reviewable text. Stated intent, from the v1.12 deprecation (PR #68132): "This is part of a move to make kubectl run simpler, enabling it create only pods; if additional resources are needed, you should use kubectl create instead." |
--generator=deployment/v1beta1 (or run/v1, job/v1, cronjob/v2alpha1, run-pod/v1) |
The flag does not exist on any command. Generators are gone from the codebase | All run generators except run-pod/v1 deprecated in v1.12 (PR #68132); removed from run in v1.18 (PR #87077); flag deleted from run in v1.21 (PR #99732); --generator deprecated on kubectl create subcommands in v1.18 (PR #88655) and removed from create deployment / autoscale in v1.22 (PR #99915, PR #99900); removed from kubectl expose (with --container-port) in v1.24 (PR #106824) |
A generator was a client-side, versioned template for an API object — meaning kubectl had to carry, and keep in sync, a hand-written copy of every workload API's shape for every supported server version. That is the same coupling that later killed kubectl convert (C3). Deleting generators moved object construction back to the server-defined schema. Trap for post-hunting: the --generator help text pointed at kubernetes.io/docs/user-guide/kubectl-conventions/#generators; that anchor no longer exists on the kubectl conventions page |
kubectl rolling-update frontend-v1 frontend-v2 --image=... |
The command does not exist. You change the Deployment's pod template (kubectl apply, kubectl set image, kubectl edit) and watch with kubectl rollout status; you undo with kubectl rollout undo |
rollingupdate introduced v0.9.0, hyphenated to rolling-update in v0.15.0 (PR #6118); kubectl rollout introduced v1.2.0 (PR #19882); rolling-update deprecated and hidden in v1.11 (PR #61285); removed in v1.18 (PR #88057: "This removes the deprecated and hidden command rolling-update… This command has been deprecated and hidden since April 2018.") |
rolling-update ran the rollout in the client: per its design doc, "the kubectl client looks for replication controllers called foo and foo-next", writes desired-replicas / update-partner annotations purely so a crashed client can resume, and the doc itself admits "if you kill the client between delete foo, and creating the new version of 'foo' you might be surprised about what is there." Moving the loop into the Deployment controller made the rollout a piece of cluster state that survives the laptop closing — the Deployment docs state the replacement contract: "You describe a desired state in a Deployment, and the Deployment controller changes the actual state to the desired state at a controlled rate." |
kubectl create -f app.yaml then kubectl replace -f app.yaml to update, and kubectl stop rc foo to tear down |
kubectl apply -f app.yaml for both create and update; kubectl delete to tear down |
kubectl apply arrived in the v1.1 cycle (PR #14621, "Add the kubectl apply command", merged 2015-10-08), together with the kubectl.kubernetes.io/last-applied-configuration annotation (PR #14626). Verified by tree state: pkg/kubectl/cmd/apply.go is absent at v1.0.0 and present at v1.1.1. kubectl stop was already deprecated in code at v1.1.1 ("The stop command is deprecated, all its functionalities are covered by delete command") and was removed in v1.8 (PR #46927; pkg/kubectl/cmd/stop.go present at v1.7.0, absent at v1.8.0) |
create fails if the object exists and replace fails if it does not, so every 2015-era update script was a create-or-replace conditional that also blew away fields it did not know about. apply made the file the source of truth and the diff the server's problem — one command that is correct on first run and on the hundredth, which is the precondition for keeping manifests in git. |
Apply: client-side merge → server-side apply¶
| What the post says | What is true at the pin | Changed in | The decision behind it |
|---|---|---|---|
kubectl apply merges your file against the kubectl.kubernetes.io/last-applied-configuration annotation |
That is still the default at the pin. Server-side apply exists, is GA, and is opt-in per invocation via kubectl apply --server-side |
SSA: alpha v1.14, beta v1.16 (PR #81956, CHANGELOG-1.16: "The server-side apply feature is now beta"), a widened second beta in v1.18 (CHANGELOG-1.18 §"Serverside Apply - Beta 2": "This new version will track and manage changes to fields of all new Kubernetes objects"), GA v1.22 (CHANGELOG-1.22 §"Server-side Apply graduates to GA"); KEP-555 kep.yaml records status: implemented, stage: stable, milestone: {alpha: v1.14, beta: v1.16, stable: v1.22}. The ServerSideApply gate was removed in v1.32 (PR #127058). Default-ness verified at v1.37.0: apply.go registers Bool("server-side", false, ...) and the kubectl apply reference documents --field-manager default "kubectl-client-side-apply" |
The annotation is a client-side reconstruction of "what I last said", which fails whenever more than one actor writes an object: kubectl cannot tell a field it should remove from a field an HPA or a mutating webhook legitimately set. SSA replaces the guess with server-recorded ownership — KEP-555: "Instead of using a kubectl.kubernetes.io/last-applied-configuration annotation, the control plane will track a 'manager' for every field", and the docs: it "tracks a user's (or client's) field management, rather than a user's last applied state." |
kubectl apply --experimental-server-side / --experimental-force-conflicts |
--server-side / --force-conflicts, plus --field-manager |
Renamed in v1.16 as part of PR #81956 (commit a3f4e6e "Rename --experimental- flags to -- for server-side apply"); verified an ancestor of v1.16.0 and not of v1.15.0 |
The --experimental- prefix was the deprecation mechanism for alpha CLI surface: promoting to beta meant dropping the prefix. Because the rename rode inside the beta-graduation PR there is no separate "added --server-side" release note to find — worth knowing if you try to date the flag from the changelog. |
managedFields clutters kubectl get -o yaml, so tools should strip it |
kubectl hides it for you, incrementally: kubectl edit v1.19 (PR #91946), kubectl get v1.21 with an opt-in --show-managed-fields (PR #96878), kubectl diff v1.25 (PR #111319), kubectl describe for custom resources v1.27 (PR #114584) |
metadata.managedFields itself landed in v1.14 with SSA alpha (PR #72947) |
Field ownership is machine bookkeeping, so it belongs in the object but not in the human's terminal. The staggered dates matter for exercises: a 2020-era post showing a huge managedFields block in kubectl get -o yaml output was accurate for exactly the v1.14–v1.20 window. |
kubectl apply --prune is how you delete removed objects |
Still alpha at the pin, and it still refuses to work with SSA: apply.go at v1.37.0 errors "--prune is in alpha and doesn't currently work on objects created by server-side apply". --prune-whitelist was renamed --prune-allowlist in v1.30 (PR #120246) |
Reworked under KEP-3659 kubectl apply --prune redesign, alpha since v1.27 |
Prune has to answer "which objects did I previously own and no longer declare?" — the same ownership question SSA answers server-side, which is why the two cannot be composed until the redesign lands. Any tutorial presenting --prune as production GitOps is presenting an alpha feature. |
| — (a v1.36-era pitfall worth knowing) | SSA had been treating empty arrays and maps as absent and duplicating atomic elements from associative lists; fixed in v1.36 (PR #135391), which changed what an apply request means for empty collections, and caused a follow-on 422 required regression fixed in v1.37 (PR #140294, backported as #140296) |
v1.36 / v1.37 | Included because it is the one apply change inside the pin window that alters semantics rather than surface: foo: [] used to be indistinguishable from omitting foo, so "apply an empty list to clear the field" silently did nothing. |
Removed and renamed commands and flags a 2015–2018 post will use¶
| What the post says | What is true at the pin | Changed in | The decision behind it |
|---|---|---|---|
kubectl convert -f old.yaml --output-version=apps/v1 |
Not a kubectl subcommand. It is a separate binary/plugin, kubectl-convert, that you install yourself |
Deprecated in v1.13 (PR #70820, milestone v1.13: "kubectl convert: add deprecation warning for 1.13"); removed from the kubectl binary in v1.20 (PR #96190), shipped as kubectl-convert; added to the released client binaries in v1.21 (PR #99155) and statically linked in v1.27 (PR #114228). Verified at v1.37.0: no convert registration in pkg/cmd/cmd.go, no pkg/cmd/convert directory, but cmd/kubectl-convert/ still exists |
convert needs compiled-in conversion functions for every API version it might be asked about, which makes a general-purpose client permanently coupled to the full history of the API. Splitting it out let the converter be versioned and installed independently of the client you upgrade weekly. Note the paper trail is wrong: the v1.14–v1.16 deprecation tables say "deprecated since v1.14, will be removed in v1.17" — both halves are off; the deprecation is v1.13 and the removal v1.20. |
kubectl get svc myapp --export -o yaml > svc.yaml to get a clean manifest |
The flag does not exist and there is no replacement. You strip status, metadata.uid, resourceVersion, creationTimestamp, managedFields yourself |
Deprecated in v1.14 (PR #73787, announced for v1.18), actually removed in v1.19 (PR #88649, milestone v1.19: "Remove --export flag from kubectl get command") |
Verbatim from PR #73787: "kubectl get --export was broken almost from its inception, multiple bugs… were reported wrt its functionality but not much was addressed. Additionally, server-side export… was brought for discussion and most of the time sig-apimachinery decided not to do it." The deeper reason it could not be fixed: "which fields are cluster-assigned" is not a property any single flag can know for arbitrary resources including CRDs. The announced v1.18 deadline slipped one release — the flag is provably still in get.go at v1.18.0 — so cite v1.19. |
kubectl get pods -a / --show-all to see Completed and Error pods |
Terminated pods are shown always; -a/--show-all is an unknown-flag error |
Default flipped false→true and flag deprecated in v1.10 (PR #60210); made fully inert in v1.11 (PR #60793); removed in v1.14 (PR #69255) |
The flag "only affected pods and only for human readable/non-API printers" (CHANGELOG-1.10) — a printer-local special case that made kubectl get pods disagree with kubectl get pods -o yaml about what existed. Hiding objects by default is the wrong default for a tool people debug with. |
kubectl log mypod |
kubectl logs mypod |
Renamed in v0.18.0 (PR #8634, v0.18.0 release notes: "Rename 'kubectl log' to 'kubectl logs'"); the singular alias survived until v1.16 (PR #78098: "Remove kubectl log, use kubectl logs instead") | Plural to match the other collection-returning verbs. Notable as the longest-lived deprecated alias in kubectl — deprecated before v1.0, removed in v1.16; per PR #78098: "kubectl log was deprecated back in commit cb872e9 which is even before first release." |
kubectl resize rc foo --replicas=3 |
kubectl scale |
Renamed in v0.18.0 (PR #8596); alias removed in v1.8 (PR #49935, which also removed apiversions, clusterinfo, rollingupdate, update) |
"Resize" reads like changing a machine's size; the operation changes a replica count. Sibling flags survived intact: --current-replicas and --resource-version are still present at v1.37 as optimistic-concurrency preconditions ("Requires that the current size of the resource match this value in order to scale") — one of the oldest unchanged pieces of kubectl surface. |
kubectl get minions |
kubectl get nodes (no) |
The minions/mi alias was rejected in v0.19.0 (PR #9027, v0.19.0 release notes: "Stop supporting the minion alias for nodes in kubectl"), initially replaced by an explicit error; the error shim was dropped in the v1.1 cycle (PR #13958); the cluster scripts' NUM_MINIONS/MINION_SIZE became NUM_NODES/NODE_SIZE in v1.2 (CHANGELOG-1.2) |
The API kind was always Node; "minion" was Borg-inherited jargon that made docs and API disagree. Worth flagging because 2015 posts use the two words interchangeably in prose while the URL path had already moved (/minions → /nodes, see A1). |
kubectl set image deployment/x c=img:2 --record to populate CHANGE-CAUSE |
--record still works at the pin but is deprecated and hidden from help output — so it is absent from the generated reference pages while remaining functional |
Deprecated in v1.22 (PR #102873: "The --record flag is being replaced with the mechanism which annotates HTTP requests with kubectl command details."). Verified at v1.37.0: cli-runtime/pkg/genericclioptions/record_flags.go registers record then calls MarkDeprecated("record", "--record will be removed in the future") |
An annotation written by the client records only what kubectl did, and only when the user remembered the flag — useless as an audit trail. KEP-859 moves the provenance into request headers, where the server records it for every request unconditionally. Trap: because MarkDeprecated hides a flag, checking the docs suggests --record is gone; only the source shows it still working. |
kubectl exec mypod ls / |
Hard error. kubectl exec mypod -- ls / is mandatory |
Warned from v1.18; removed in v1.31 (PR #125437, milestone v1.31; CHANGELOG-1.31: "The -- separator has been recommended since the Kubernetes v1.18 release, which also deprecated the legacy way of invoking kubectl exec.") — the code now returns "exec [POD] [COMMAND] is not supported anymore. Use exec [POD] -- [COMMAND] instead" |
Without the separator kubectl cannot tell its own flags from the remote command's: kubectl exec pod ls -l had to guess whether -l was kubectl's. The separator removes the ambiguity instead of heuristically resolving it. Caution: CHANGELOG-1.35 carries a note that restates this removal, but its PR (#133841) is titled "[failing-test] fix kubectl exec command in cmd test" — a test fix with a misleading release note. The removal release is v1.31. |
kubectl alpha diff / kubectl alpha debug |
kubectl diff, kubectl debug, kubectl events are top-level. kubectl alpha still exists but is empty and hidden — invoking it prints "No alpha commands are available in this version of kubectl" |
The alpha group was created in v1.7 (PR #46151). diff: alpha v1.9 (PR #53920) → top level v1.13 (PR #69167, "Move diff command from alpha sub-command to root cmd"), alpha form gone the same release. debug: alpha v1.18 (PR #88004) → kubectl debug v1.20 (PR #96138; CHANGELOG-1.20: "Invocations using kubectl alpha debug are now deprecated and will be removed in a subsequent release") → alpha form removed v1.21 (PR #98111). events: alpha v1.23 (PR #99557) → kubectl events v1.26 (PR #113819), same release. auth whoami: alpha v1.25 (PR #110564) → top level v1.27 (PR #116510). kuberc view/set: alpha v1.35 (PR #135003) → top level v1.36 (PR #136643) |
kubectl alpha is a staging area, not a deprecation surface: the API deprecation policy's alpha rule (Rule #4a, no notice required) applies, so promotion usually deletes the alpha form in the same release. debug is the only one that got a grace period. Consequence for exercises: a command name under alpha has a shelf life of two to three releases, so any post older than that is citing a path that simply 404s. |
kubectl top node reads metrics from heapster |
Reads the metrics API served by metrics-server; v1.37 added metrics.k8s.io/v1 support (PR #139726) |
kubectl top added v1.4 (PR #28844); metrics-API support added v1.10 with heapster fallback (PR #56206, whose body notes "If the metrics.k8s.io API is not served by the apiserver, then this still falls back to the previous behavior"); heapster support dropped in v1.19 (PR #87498, body: "Heapster is already retired… Having kubectl top mention Heapster is confusing for users") |
Reading a specific add-on's HTTP endpoint hard-codes one implementation into the client. Going through an aggregated API (metrics.k8s.io) means any conformant provider works and kubectl needs no knowledge of it — the same "define an API, not a client integration" move as A3's group split. |
Traps — kubectl¶
kubectl runis the single most dangerous command in an old post, because it still exists and still succeeds. A 2016 tutorial'skubectl run nginx --image=nginxprinteddeployment "nginx" created; at the pin the identical command printspod/nginx created. Nothing warns you. The follow-up steps in the post —kubectl get deployments,kubectl scale deployment nginx,kubectl rollout status,kubectl delete deployment nginx— then all fail on a missing object, and a learner reasonably concludes their cluster is broken. If a post's narrative depends onrunproducing a controller, the exercise must replace the command, not just the flags.--replicas=3onrunfailed loudly for three releases and now fails differently. v1.18–v1.20 accepted it and silently ignored it (MarkDeprecated("replicas", "has no effect…")) — so a copy-pasted command on a v1.19 cluster created one pod and reported success. From v1.21 it is an unknown-flag error. The "accepted but inert" window is the nastiest state to reproduce in a lab and worth calling out explicitly.kubectl applyis still client-side by default at the pin. Posts from 2021 onward often assert that "apply is now server-side"; it is not, unless you pass--server-side. Verified two ways at v1.37.0 (the flag defaults tofalse;--field-managerdefaults tokubectl-client-side-apply). So thelast-applied-configurationannotation a 2016 post describes is still being written to your objects — the old post is more accurate here than the newer one.- CHANGELOG entries lie about their own removals, in both directions. Three confirmed cases: CHANGELOG-1.8 announced removal of the
run-containeralias, which a bad conflict resolution in PR #50998 reinstated two days later — it actually died in v1.13;--exportwas announced for v1.18 and removed in v1.19;kubectl convertwas announced for v1.17 and removed in v1.20. Never date a kubectl removal from the announcement — check the source tree at the release tag. (Corollary: CHANGELOG-1.20's "Generators for services are removed from kubectl" overclaims;kubectl expose --generatorsurvived to v1.24.) - A deprecated flag disappears from the docs before it disappears from the binary.
MarkDeprecatedhides a flag from--helpand therefore from the generated reference pages.--recordis the live example: absent from the docs, present and functional at v1.37.0. Absence from the reference page is not evidence of removal. kubectl get podsoutput shape has drifted independently of any flag.--show-allis gone because terminated pods are now always shown;kubectl describe service/ingressread EndpointSlices instead of Endpoints from v1.31 (PR #124598);kubectl get/describestopped reporting counts for referenced tokens and secrets in v1.35 (PR #117160);kubectl describeshows events only for single-object describes from v1.36 unless you pass--show-events(PR #137145). Any exercise that greps or diffs kubectl's human-readable output against a transcript from an old post will mismatch on whitespace and on content.- Flag renames that a post will hit outside
run:kubectl drain --delete-local-data→--delete-emptydir-data(old form dropped v1.31, PR #125842);kubectl apply --prune-whitelist→--prune-allowlist(v1.30, PR #120246);kubectl autoscale --cpu-percentbegan deprecation in favour of--cpu/--memory(v1.34, PR #129373). - The commands a 2015 post uses that no longer exist at all:
run-container,resize,rollingupdate,rolling-update,log,clusterinfo,apiversions,update,stop,convert,minions. Every one of these produces "unknown command", which is at least a loud failure — unlikerun,apply, and--record, which fail quietly. - [UNVERIFIED]
kubectl expose --generatormay never have had a formal deprecation release. NoMarkDeprecated("generator", …)appears inexpose.goat any tag v1.19.0–v1.23.0 (only--container-portcarried one), and the v1.18 blanket deprecation (PR #88655) patched thecmdutil.AddGeneratorFlagshelper thatexpose.godoes not use. Tried: CHANGELOG-1.16→1.24 greps forexpose+generator, PR search, and the full commit history ofexpose.go. Removal in v1.24 (PR #106824) is solid; the deprecation date is not — do not put one in an exercise. - [UNVERIFIED] the "kubectl died mid-rollout so the rollout stalled" framing. The mechanism is fully documented (
simple-rolling-update.md: client-driven loop,desired-replicas/update-partnercrash-recovery annotations, "Recovery is achieved by issuing the same command again"), but no primary source states that sentence. Teach the mechanism, not the slogan.
Part 2 — Security, authorization, policy, and the container runtime¶
Who is allowed to do what, and what is actually running the container.
Security, authorization, policy¶
| What the post says | What is true at the pin | Changed in | The decision behind it |
|---|---|---|---|
| "The API server allows everything by default; you don't need to grant anything." | Still literally true of the flag: --authorization-mode "Defaults to AlwaysAllow if --authorization-config is not used." What changed is that no real installer leaves it there — kubeadm builds Node,RBAC clusters — and the docs now carry a warning that AlwaysAllow "bypasses authorization; do not use this on a cluster where you do not trust all potential API clients, including the workloads that you run." |
Flag default unchanged through v1.37; the installer default flipped in v1.6 ("kubeadm creates RBAC-enabled clusters") | The project never changed the binary's default because changing it would break every hand-rolled cluster on upgrade; instead the decision was to move the safe default into the thing that builds clusters (kubeadm), so that "insecure by default" survives only for someone who assembles an apiserver by hand. |
"Set --authorization-mode=ABAC and write a policy file." |
ABAC still exists and still works, unchanged: one JSON object per line, apiVersion: abac.authorization.kubernetes.io/v1beta1, kind: Policy, fed by --authorization-policy-file. It was never deprecated or removed. It is also still true that "the apiserver will need to be restarted to pick up the new policy lines." |
Never removed (present at v1.37); superseded in practice by RBAC from v1.6/v1.8 | ABAC's policy lives in a file on the control-plane disk, so it cannot be granted or revoked through the API, cannot be RBAC-audited, and needs an apiserver restart per change — fatal for multi-tenant and for anything GitOps-shaped, which is why authorization moved to API objects rather than files. |
| "kube-up gives you AllowAll." | Historical: GCE kube-up switched from AllowAll to ABAC in v1.3; by v1.6 the gce provider enabled RBAC plus a permissive legacy ABAC policy "that makes all service accounts superusers", opt-out via ENABLE_LEGACY_ABAC=false. kube-up itself is long gone from the supported path. |
v1.3 (PR #24210), v1.6 (PR #43544) | The legacy-ABAC-superuser escape hatch existed precisely because turning RBAC on would have broken every add-on's service account overnight; it is the historical reason so many 2016–2017 tutorials "work" without any RoleBinding. |
"RBAC is a new alpha thing" / a manifest using rbac.authorization.k8s.io/v1alpha1 |
rbac.authorization.k8s.io/v1 only. v1alpha1 was deprecated in v1.8 and disabled by default in v1.8 (--runtime-config needed to re-enable). |
alpha API group v1.3; v1beta1 + beta + default roles v1.6; v1 GA v1.8 ("The rbac.authorization.k8s.io/v1beta1 API has been promoted to rbac.authorization.k8s.io/v1 with no changes") |
RBAC replaced ABAC because permissions became first-class API objects: grantable at runtime, namespaced, aggregatable, and themselves subject to authorization — which is what made escalation-prevention (you may only grant permissions you hold) expressible at all. |
--authorization-rbac-super-user=admin |
Gone. Deprecated in v1.6 in favour of the system:masters group, which has privileged access. |
deprecated v1.6 (PR #38121) | A single named super-user is a flag, not a credential you can rotate or revoke; moving it to a group means the escape hatch is expressed in the same certificate/group vocabulary as everything else. |
| "Authorization is configured with flags." | Still supported, but the modern form is a file: --authorization-config (Structured Authorization Configuration) defines an ordered authorizer chain and is mutually exclusive with --authorization-mode and the --authorization-webhook-* flags. |
KEP-3221: alpha v1.29, beta v1.30, stable v1.32 | Flags could express only one webhook and no ordering/failure semantics; a config file lets an operator chain multiple webhooks with per-authorizer timeouts, failure policy and CEL match conditions, and reload it without restarting. |
| — (nothing a 2015 post could say) | Authorizers can now return conditional decisions (not just Allow/Deny/NoOpinion) — core machinery landed as alpha. | KEP-5681 alpha v1.37 (CHANGELOG-1.37, PR #137513) | Allow/Deny alone forces authorizers to over-grant when a decision depends on request content; conditional decisions let the authorizer say "yes, but only for objects matching X", closing the gap that previously required an admission webhook. |
| — | Authorizers can restrict by label/field selector (Authorize with Selectors), and anonymous auth can be limited to named endpoints instead of being all-or-nothing. |
KEP-4601 alpha v1.31, beta v1.32, stable v1.34; KEP-4633 alpha v1.31, beta v1.32, stable v1.34 | Both close the same class of hole: RBAC verbs were too coarse to say "watch only your own node's pods", and --anonymous-auth=true was needed just for /healthz, which accidentally exposed everything else. |
| "Every ServiceAccount has a Secret with a token in it; mount it and you're authenticated forever." | No Secret is created. Secret API objects containing service account tokens "are no longer auto-generated for every ServiceAccount." Pods get a projected token: audience-scoped, time-limited, rotated by the kubelet before expiry, and bound to the Pod (and since the node-binding work, to the Node). Long-lived Secret tokens can still be created deliberately, and existing ones still authenticate. | Bound tokens KEP-1205: alpha v1.13, beta v1.21, stable v1.22. Auto-generation stopped: LegacyServiceAccountTokenNoAutoGeneration beta and on by default in v1.24. KEP-2799 (tracking + clean-up of legacy tokens): beta v1.24, stable v1.30 |
The legacy token was a non-expiring, non-audience-scoped bearer credential sitting in etcd and mounted into every pod — one leaked pod filesystem was permanent cluster credential theft, and there was no way to tell whether a token was still in use. Projected tokens made the credential expire, name its audience, and be attributable to a pod/node so it can be both rotated and revoked by deleting the pod. |
"Get a token with kubectl get secret <sa>-token-xxxxx" |
That Secret usually does not exist. Tokens come from the TokenRequest API (kubectl create token <sa>) or a projected volume. |
TokenRequest gate is part of KEP-1205 (stable v1.22); auto-created Secrets stopped v1.24 | Same decision as above: issuing a token became an explicit, audited, expiring API call rather than a side effect of creating a ServiceAccount. |
| — | Tokens carry a JTI, pod/node info, and can be node-bound. | KEP-4193: alpha v1.29, beta v1.30, stable v1.32 | Without an identifier and node/pod claims a stolen token was indistinguishable from a legitimate one in the audit log; embedding them makes token misuse detectable and lets the apiserver reject a token used off its node. |
| — | Pod Certificates (X.509 identity for pods via podCertificate projected volume + PodCertificateRequest) reached GA. |
v1.37 (CHANGELOG-1.37, PR #139579); NodeRestriction hardening PR #140006 | The natural end point of the bound-token work: mTLS identity for workloads issued by the cluster CA rather than a bearer token that anything holding it can replay. |
| "Write a PodSecurityPolicy to restrict privileged pods." | PodSecurityPolicy does not exist. The API and admission plugin are removed; a PSP manifest is rejected by the API server. |
Types present in extensions/v1beta1 at v1.2.0; enforcing admission plugin + beta in v1.3 (PR #24600); moved to policy/v1beta1 in v1.10 (PR #54933); deprecated v1.21 (PR #97171); removed v1.25 (PR #109798) |
PSP's authorization was inverted: which policy applied depended on which service account created the pod, and if several PSPs matched, the mutating one that won was effectively arbitrary. That made it impossible to reason about what a pod would get, impossible to test before enforcing, and impossible to roll out namespace-by-namespace — "the updates required to improve its usability would have introduced breaking changes, so it became necessary to remove it in favor of a more friendly replacement." |
| "PSP is the way to enforce pod security." | Pod Security Admission: a built-in, non-mutating admission plugin enabled by default. Three levels — privileged, baseline, restricted — applied per namespace via labels: pod-security.kubernetes.io/<MODE>: <LEVEL> and optional pod-security.kubernetes.io/<MODE>-version: <VERSION> where MODE ∈ {enforce,audit,warn}. Cluster-wide defaults and exemptions come from --admission-control-config-file with pod-security.admission.config.k8s.io/v1. |
KEP-2579: alpha v1.22, beta v1.23, stable v1.25 (CHANGELOG-1.25, PR #110459) | Three fixed profiles instead of arbitrary policy objects means the outcome is knowable from a namespace label alone; three modes means you can turn on warn/audit first and see what would break before you enforce; version-pinning the profile means a cluster upgrade cannot silently tighten policy under a running workload. PSA is deliberately non-mutating — PSP's mutation was the part nobody could reason about. |
| "PSP blocks privileged pods, so my probes are safe." | PSA now also blocks host fields in probes and lifecycle handlers under the restricted profile. |
KEP-4940 stable v1.34 | A pod barred from host networking could still aim a probe at a host address, so the profile's stated guarantee did not hold; the fix closed the gap rather than adding a new profile. |
| "Migrate PSPs to PSA with the migration tool." | There is no official tool. The documented path is manual and staged: simplify/standardise existing PSPs, label namespaces, use "a combination of dry-run and audit and warn modes", review namespace-creation processes, then disable PSP. Anything PSP did that PSA cannot (mutation, per-service-account policy) needs a third-party webhook. |
guide exists from v1.22–v1.25 era | There is no mechanical mapping because PSP could mutate and PSA cannot — a tool would have had to invent the policy decisions, so the project shipped a decision procedure instead of a converter. |
"Use NetworkPolicy from extensions/v1beta1 and annotate the namespace with net.beta.kubernetes.io/network-policy to turn on isolation." |
networking.k8s.io/v1. The namespace annotation is gone; isolation is per-Pod — a Pod is isolated iff some NetworkPolicy's spec.podSelector selects it, and a Pod selected by nothing accepts all traffic. |
v1beta1 API v1.3 (PR #25638); GA networking.k8s.io/v1 in v1.7 (PR #39164) |
Namespace-level "DefaultDeny" was a single boolean for a whole namespace, so you could not incrementally isolate workloads; per-pod isolation makes the policy object itself the unit of opt-in. Trap: the v1.7 notes explicitly warn that v1beta1 policies that "had no effect" become load-bearing under v1 semantics and can silently block traffic. |
| "Secrets are encrypted." | Secrets are base64 in etcd unless you configure encryption at rest. Config file: apiVersion: apiserver.config.k8s.io/v1, kind: EncryptionConfiguration, via --encryption-provider-config. |
encryption at rest alpha v1.7; config format stable + flag renamed from --experimental-encryption-provider-config in v1.13 (PR #67383, #71206) |
Encryption was made opt-in and pluggable rather than default because the key has to live somewhere, and the project would not choose a key-management story for every operator. |
| "Use a KMS plugin." | KMS v2 is the current provider. KMS v1 (gRPC envelope service) still exists but v2 is what you want. | KMS v1 gRPC envelope v1.10 (PR #55684); KMS v2 KEP-3299: alpha v1.25, beta v1.27, stable v1.29 | KMS v1 called the external KMS once per object read, which made the KMS a latency and availability dependency of every API read; v2 introduced a cached local KEK with key-ID observability so rotation is verifiable and reads do not hit the KMS. |
"Turn on audit logging with --audit-log-path." |
Still the mechanism, but the policy/event schema is audit.k8s.io/v1. |
advanced audit alpha v1.7, beta v1.8; audit.k8s.io/v1 available v1.12, became the default for --audit-log-version/--audit-webhook-version in v1.13 (PR #70476) |
Basic audit logged one line per request with no way to say which requests matter; the advanced/policy form let operators choose per-rule levels (Metadata/Request/RequestResponse) so that audit volume became a policy decision rather than an on/off switch. |
| "Enforce custom policy with a ValidatingWebhookConfiguration." | Webhooks still work and are still default-enabled, but in-tree CEL policy is now the first choice: ValidatingAdmissionPolicy (GA, default on) and MutatingAdmissionPolicy (GA). Default-enabled plugins at v1.37 include PodSecurity, ValidatingAdmissionPolicy, MutatingAdmissionWebhook, ValidatingAdmissionWebhook. |
VAP KEP-3488: alpha v1.26, beta v1.28, stable v1.30 (PR #123405, "enabled by default"). MAP KEP-3962: alpha v1.32, beta v1.34, GA v1.36 (PR #136039, "The feature is now enabled by default") | A webhook is a separate deployment on the critical path of every write: if it is down, either the cluster stops accepting writes (failurePolicy: Fail) or policy silently stops applying (Ignore). Embedding declarative CEL "within the API, without relying on any external HTTP callouts" removes that availability coupling and the certificate rotation burden for the large class of policies that are pure validation. |
"Gate images with ImagePolicyWebhook." |
Still present and documented at v1.37, configured via --admission-control-config-file. [UNVERIFIED] its current stability level (alpha vs beta) — the admission-controllers doc fetch confirmed the section exists but the fetched excerpt was truncated before the feature-state banner; I did not find a KEP for it in keps/sig-auth or keps/sig-node. |
present since the 1.4-era image-policy work (CHANGELOG-1.4 lists "Container Image Policies" among 1.4 themes) | It is the one admission controller that was never modernised, which is itself the signal: image admission moved to the ecosystem (sigstore policy controllers, third-party admission) rather than into the API server. |
"Seccomp is off / use the seccomp.security.alpha.kubernetes.io/pod annotation." |
securityContext.seccompProfile is a first-class field. The annotations were deprecated at GA and removed in v1.22. SeccompDefault (kubelet applies RuntimeDefault to pods that specify nothing) is stable and opt-in per kubelet. |
seccomp GA v1.19 (annotations removed v1.22); SeccompDefault KEP-2413: alpha v1.22, beta v1.25, stable v1.27 |
Annotations bypass API validation and versioned conversion, so a typo produced an unconfined container silently; promoting it to a field made it validatable. SeccompDefault stayed opt-in because turning on a syscall filter cluster-wide can break workloads that the cluster operator, not the API, has to vouch for. |
"AppArmor via the container.apparmor.security.beta.kubernetes.io/<container> annotation." |
GA field. The AppArmor feature gate can no longer be disabled, and from v1.31 the kubelet hard-rejects pods requesting AppArmor on a node that does not have it. |
KEP-24: alpha v1.4, beta v1.4, stable v1.31 (PR #125257; rejection PR #125776) | AppArmor sat in beta for 27 releases; the reason GA mattered is the rejection behaviour — previously a pod asking for a profile on a node without AppArmor ran unconfined, so the security control failed open. |
| — (a 2015 post has nothing at all here) | Kubernetes release artifacts are signed keyless with cosign/sigstore: container images, client/server/source tarballs, standalone binaries, the SPDX SBOM, and provenance.json. Verified with cosign verify registry.k8s.io/kube-apiserver-amd64:v1.37.0 --certificate-identity krel-trust@k8s-releng-prod.iam.gserviceaccount.com --certificate-oidc-issuer https://accounts.google.com. |
KEP-3031: image signing alpha v1.24 ("Image signing was alpha for v1.24 and v1.25"), beta v1.26 with binaries + SBOM + provenance added | Signing is keyless and identity-based (an OIDC identity for the release-engineering service account, logged in a transparency log) rather than a long-lived project key, because a project key held by a rotating group of release managers is the thing you cannot secure. |
| — | SLSA work is tracked by KEP-3027, "SLSA Level 3 Compliance in the Kubernetes Release Process", status: provisional, latest-milestone: v1.23. [UNVERIFIED]: the specific claim "Kubernetes attests SLSA Build Level 3 at the pin". The KEP is still provisional and names Levels 1–4 as staged goals without recording an achieved level in kep.yaml; a web search surfaced "SLSA Level 1 from 1.23" but only from summary text, not a primary statement I could quote. Provenance files are definitely produced and signed (see the row above). |
KEP-3027 opened against v1.23 | Provenance answers a different question from signing: a signature says "the release team published this", provenance says "this artifact came out of this build, from this source, by this process" — needed because the 2015 model was curl \| tar from a bucket with an optional SHA1. |
Traps — security, authorization, policy¶
--authorization-modestill defaults toAlwaysAllow. This is the most dangerous "nothing changed" in the whole area. A reader who follows a 2015 post to stand up an apiserver by hand — or a lab exercise that starts one directly — gets a cluster with no authorization at all, at v1.37, with no error. Only kubeadm/managed distros save you. Any exercise that spawns a raw apiserver must pass--authorization-mode=Node,RBACexplicitly.- A PSP manifest is now a hard error, but an RBAC manifest referencing PSPs applies cleanly and grants nothing.
kubectl applyof aPodSecurityPolicyfails outright (good — visible). But the accompanying ClusterRole withapiGroups: ["policy"], resources: ["podsecuritypolicies"], verbs: ["use"]applies successfully at v1.37 and is inert. RBAC does not validate that the resource exists. The reader ends up believing pod security is enforced when nothing is enforcing it. extensions/v1beta1vspolicy/v1beta1PSP. Posts between v1.3 and v1.10 useextensions/v1beta1; posts between v1.10 and v1.21 usepolicy/v1beta1. Both are equally dead — but a reader who sees "the apiVersion is wrong" may try to "fix" it forward rather than concluding the whole object is gone.- Namespace labels are the silent failure mode of PSA. A namespace with no
pod-security.kubernetes.io/enforcelabel enforcesprivileged— i.e. nothing. Applying a "restricted" workload proves nothing about the namespace's policy. Conversely, an exercise that labels the namespaceenforce: restrictedwill break most 2015-era pod manifests (norunAsNonRoot, noseccompProfile, noallowPrivilegeEscalation: false, no dropped capabilities) — that breakage is the lesson, but it needs to be expected. kubectl get secretfor a ServiceAccount token returns nothing. Every 2015–2023 post that says "find the token in the<sa>-token-xxxxxSecret" is broken since v1.24, and it fails by absence, not by error:kubectl get secrets -n defaultjust lists nothing. The replacement iskubectl create token <sa>. Note the second-order trap: the token you get is short-lived, so a copied-out token in an exercise script will pass once and fail an hour later.- Legacy tokens still authenticate. Auto-creation stopped in v1.24; tokens that already exist, or that you deliberately create as a Secret, still work. So "it worked when I tried it on an old cluster" is not evidence the mechanism is current.
- A v1beta1 NetworkPolicy that was inert becomes enforcing. Explicitly called out in the v1.7 upgrade notes: policies that had no effect under the namespace-annotation model start isolating pods under v1 semantics. A 2016 post's example policy can block traffic the post says it permits.
rbac.authorization.k8s.io/v1alpha1fails as "no matches for kind", not as "deprecated". Disabled by default since v1.8; the error looks like a typo rather than a version problem.- Webhook-based policy examples still work, which hides that they are no longer the recommended shape. A 2019 post's ValidatingWebhookConfiguration applies and functions at v1.37. Nothing tells the reader that a ValidatingAdmissionPolicy would do it without a deployment, a Service, a CA bundle, and a failure-mode decision.
Container runtime¶
| What the post says | What is true at the pin | Changed in | The decision behind it |
|---|---|---|---|
| "Docker is the container runtime. The kubelet talks to the Docker daemon." | The kubelet speaks CRI over gRPC to a runtime socket; there is no Docker-specific code path. Documented runtimes at v1.37: containerd, CRI-O, Docker Engine via the third-party cri-dockerd adapter, and Mirantis Container Runtime. kubeadm defaults to unix:///var/run/containerd/containerd.sock. |
dockershim deprecated v1.20 (announced 2020-12-02, PR #94624); removed v1.24 (May 2022, PR #97252); kubeadm defaulted to the containerd socket in v1.24 (PR #107317) | dockershim was "always intended to be a temporary solution (hence the name: shim)": Docker Engine does not implement CRI, so the Kubernetes project carried in-tree adapter code for one vendor's daemon, and "it has seen maintenance issues in the Kubernetes community". Removing it means every runtime is on equal footing behind one interface instead of one runtime being privileged in the kubelet's source tree. |
"You can also run rkt with --container-runtime=rkt." |
The flag is gone, the code is gone, the project is gone. At v1.2.0 the kubelet really did carry --container-runtime with "Possible values: 'docker', 'rkt'. Default: 'docker'." plus --rkt-path and --rkt-stage1-image. |
rktnetes deprecated v1.10 in favour of rktlet (PR #58418); rktnetes code removed v1.11 (PR #61432); the --container-runtime flag itself removed v1.27 (PR #114017). The rkt/rkt repo is archived, described "[Project ended]", last release v1.30.0 (2018-04-16), last push 2020-02-24 |
The in-tree, per-runtime integration model is exactly what CRI was created to end: rkt support meant a second vendor-specific code path in the kubelet, so the plan was to move rkt behind CRI (rktlet) rather than keep it in tree — and when rkt itself wound down, nothing in Kubernetes had to be removed a second time. That is the payoff of the interface. |
| — | CRI is the only way in. The API is runtime.v1; support for v1alpha2 was dropped, "which means that container runtimes just have to implement v1." |
CRI v1alpha1 introduced v1.5 (alpha; design doc in kubernetes/community); KEP-2040 records alpha v1.5, beta v1.23; v1alpha2 support dropped in v1.26 (PR #110618) |
"The CRI standard was created to enable interoperability between orchestrators and many different container runtimes." Dropping v1alpha2 in v1.26 is the moment CRI stopped being a compatibility layer and became a contract: the kubelet no longer negotiates down, so a runtime either implements the current API or does not run pods. |
| "Docker images won't work any more" (the 2020 panic reading) | Wrong then and wrong now: "the images produced from docker build will work with all CRI implementations. All your existing images will still work exactly the same." Docker remains a fine image build tool. |
clarified at deprecation, v1.20 (2020-12-02) | Image format and runtime are separate concerns: docker build emits an OCI-compatible image, and every CRI runtime consumes OCI images. What was removed was the kubelet↔dockerd control-plane adapter, not the image format. |
| "The removal is coming in v1.22." | It slipped. Deprecation in v1.20 announced removal "currently planned for the 1.22 release in late 2021"; the actual removal was v1.24. | announced v1.20, executed v1.24 | Worth teaching as itself: the two extra releases were bought by ecosystem readiness, and any post written in 2021 states the wrong removal version in good faith. |
"SSH to the node and run docker ps to see your containers." |
The command may still exist (if Docker is installed on the node for other reasons) and will show nothing that Kubernetes runs. Use crictl, "a command-line interface for CRI-compatible container runtimes… to inspect and debug container runtimes and applications on a Kubernetes node", pointed at the runtime endpoint (e.g. unix:///var/run/containerd/containerd.sock). |
v1.24 | This is the single worst trap in the area because the failure is empty output, not an error. kubeadm itself made the switch in v1.24: "Use crictl for all communication with CRI sockets for actions like pulling images and obtaining a list of running containers instead of using the docker CLI." |
"docker exec -it <id> sh to debug." |
crictl exec on the node, or — the intended path — kubectl debug with an ephemeral container, which needs no node access at all. |
Ephemeral containers KEP-277: alpha v1.16, beta v1.23, stable v1.25 | Node-level exec requires SSH plus root on the host, which is a much larger grant than "debug this pod"; ephemeral containers moved debugging into the API where it is RBAC-controlled and audit-logged. |
"Images and container state live in /var/lib/docker." |
Runtime-dependent and not Docker's: containerd uses /var/lib/containerd, CRI-O uses /var/lib/containers. Nothing in Kubernetes documents a path here, because the layout is the runtime's business. [UNVERIFIED] exact default paths — I did not find these asserted in kubernetes.io primary docs (they are runtime-project documentation, outside this scope's source rules); treat any specific path in an exercise as runtime-specific and verify against the runtime in use. |
v1.24 | Same reason as docker ps: the kubelet never owned that layout, so when the adapter left there was no path to preserve. Node-level disk troubleshooting moved to crictl and kubelet stats/eviction signals. |
Kubelet flags --docker-endpoint, --network-plugin, --cni-conf-dir, --cni-bin-dir |
All removed. Along with dockershim the kubelet dropped --experimental-dockershim-root-directory, --docker-endpoint, --image-pull-progress-deadline, --network-plugin, --cni-conf-dir, --cni-bin-dir, --cni-cache-dir, --network-plugin-mtu. --container-runtime-endpoint moved into the kubelet config file. |
v1.24 (PR #106907); endpoint flags migrated to config in v1.27 (PR #112136); --container-runtime removed v1.27 (PR #114017) |
CNI configuration was only ever the kubelet's business because dockershim did pod networking; once the runtime owns the sandbox, the runtime owns CNI, so the flags had no owner left in the kubelet. |
| "Kubernetes runs on cgroup v1" (unstated assumption in every pre-2021 post) | cgroup v1 is deprecated and the kubelet will not start on a cgroup v1 node by default — failCgroupV1 defaults to true from v1.35; an admin must set failCgroupV1: false in the kubelet config to run on cgroup v1. Removal is still pending: KEP-5573 "Remove cgroup v1 support" is status: implementable, stage: beta, latest-milestone: v1.35. |
cgroup v2 KEP-2254: alpha v1.18, beta v1.22, stable v1.25. cgroup v1 to maintenance mode: KEP-4569 alpha/beta/stable all in v1.31 (warning log + event + version metric, PR #125328). cgroup v1 deprecated v1.35 (PR #134298; kubeadm escape hatch PR #134744) | "With some distributions now defaulting to this API, Kubernetes must support it to continue operating on those distributions" — the move was forced by the distros, not chosen. The two-step (maintenance mode in v1.31 with a metric and an event, then fail-to-start in v1.35 with a documented opt-out) exists so operators could discover cgroup v1 nodes from a metric before an upgrade started refusing to run them. Features like MemoryQoS and memory.low/memory.min protection only exist on v2, which is why v1 could not stay indefinitely. |
"Set --cgroup-driver=cgroupfs" (or nothing, and it works) |
cgroupfs is still the kubelet's default but is "not recommended when systemd is the init system, because systemd expects a single cgroup manager on the system"; systemd must be set on both the kubelet (cgroupDriver: systemd) and the container runtime. At v1.37, with KubeletCgroupDriverFromCRI and a runtime supporting the RuntimeConfig CRI RPC, the kubelet auto-detects the driver from the runtime and ignores the cgroupDriver setting. |
KEP-4033 "Discover cgroup driver from CRI": alpha v1.28, beta v1.31, stable v1.34 | Two components independently configured to manage the same cgroup tree is a mismatch that manifests as unstable resource accounting rather than a startup error — the classic silent kubeadm misconfiguration. Reading the driver from the runtime over CRI removes the possibility of disagreeing, because there is now one answer instead of two settings. |
| — | User namespaces for pods (spec.hostUsers: false) are GA. Rootless kubelet is separate and still beta. |
UserNamespaces KEP-127: alpha v1.25, beta v1.35, stable v1.36 (PR #136792, "Feature gate UserNamespacesSupport is now GA"). Rootless mode KEP-2033 (KubeletInUserNamespace): alpha v1.22, beta v1.37. UserNamespacesHostNetworkSupport added as a disabled-by-default gate in v1.35 (PR #134893) |
Every pre-2025 post's mental model is "container root is host root unless you set runAsNonRoot". User namespaces break that identity mapping, so a container can be uid 0 inside and unprivileged outside — which is what makes several whole classes of container escape non-exploitable rather than merely discouraged. Note the two are distinct: pods in user namespaces (GA) vs the kubelet itself running unprivileged (beta). |
| "The kubelet garbage-collects images when disk fills up." | Still true, plus an age-based axis: ImageMaximumGCAge lets the kubelet evict images unused for longer than a configured age regardless of disk pressure. |
KEP-4210: alpha v1.29, beta v1.30, stable v1.35 | Disk-pressure-only GC means a node accumulates stale images indefinitely until it is nearly full, then evicts in a burst; an age bound turns image retention into a policy instead of a side effect of free space — and shrinks the window in which a long-cached image with a known CVE is still runnable. |
| — | The CRI keeps growing: v1.37 added CheckpointPod/RestorePod RPCs to the CRI v1 RuntimeService. |
v1.37 (PR #140366) | Evidence for the thesis: because runtime capability now arrives through one versioned interface, new node features land as CRI RPCs rather than as kubelet code special-cased per runtime. |
Traps — container runtime¶
docker psreturns success and an empty (or unrelated) list. No error, no warning, no hint that the containers exist under containerd. This is the highest-value trap in area B: any 2015–2021 troubleshooting walkthrough that starts with "SSH in and list the containers" leads the reader to conclude their pods never started. The equivalent iscrictl ps— and it needs the runtime endpoint set, or it "attempts to connect to a list of known endpoints, which might result in an impact to performance."- A
--container-runtime=rkt(or=docker) flag makes the kubelet fail to start with an unknown-flag error, not a deprecation warning. Removed in v1.27. An exercise that copies a 2015 kubelet unit file gets a dead kubelet with a message that does not mention CRI. - The kubelet refusing to start on a cgroup v1 node looks like a broken install. From v1.35 this is the intended behaviour, and the fix (
failCgroupV1: false) is a kubelet config field, not a flag — so it is invisible to anyone grepping--help. Any lab built on an old base image (or an old CI runner) can hit this. - The cgroup driver mismatch produces no error at all. kubelet on
cgroupfsand runtime onsystemdstarts fine and runs pods; the symptom is wrong/unstable resource accounting under load. Since v1.34KubeletCgroupDriverFromCRImay silently ignore thecgroupDriveryou set — so a post telling you to set it, and an exercise verifying that it took effect, can both be "right" and disagree. - "Removal is planned for v1.22" is in every 2021-vintage post. The reader who checks "am I affected?" against v1.22 concludes wrongly. The real boundary is v1.24.
- Docker being installed on a node proves nothing. Many node images still ship the Docker CLI/daemon; its presence makes the old commands run, which is worse than them being absent. The question is never "is Docker there", it is "what socket does the kubelet's
containerRuntimeEndpointpoint at". cri-dockerdmeans "you can still use Docker Engine" — but it is a third-party component outside the Kubernetes project. A post saying "Docker still works" and a post saying "Docker was removed" are both accurate about different things; the distinction is in-tree adapter (gone) vs out-of-tree adapter (exists, separately maintained).- Old
securityContextexamples get quietly stronger guarantees than they claim, and older ones get weaker ones. A pre-v1.31 post describing AppArmor via annotation would have run unconfined on a node without AppArmor; at the pin the pod is rejected. So a 2018 exercise "proving" a profile was applied may never have proven anything. hostUsers/user namespaces change whatrunAsUser: 0means. Posts explaining "never run as root in a container" are still good advice but their stated reason (container uid 0 == host uid 0) is no longer unconditionally true at v1.37.
Part 3 — Networking, storage, cluster lifecycle, observability¶
The subsystems whose posts are most likely to still apply — and still be wrong.
Networking¶
Independent verification anchors used in this section: v1.37.0 was published 2026-08-26T16:29:12Z; the newest stable kubernetes-sigs/gateway-api release at/before the pin is v1.6.1 (2026-07-16); kubernetes.io still states the default proxy mode is iptables and that "a future version of Kubernetes will change the default to nftables".
| What the post says | What is true at the pin | Changed in | The decision behind it |
|---|---|---|---|
| kube-proxy proxies Service traffic in userspace — it opens a local port per Service and relays connections. | Gone. Not deprecated-but-working: the mode is removed and --proxy-mode=userspace is an unrecognised value. |
Deprecated v1.25, removed v1.26 | Every packet had to leave the kernel, cross into a user process and come back, doubling the syscall/copy cost per connection and making kube-proxy itself a data-path failure point; iptables did the same DNAT entirely in-kernel. |
--proxy-mode=iptables is the new/optional fast path. |
iptables is still the default on Linux at v1.37 — kubernetes.io: "In Kubernetes 1.37, this is iptables, but a future version of Kubernetes will change the default to nftables." |
Became the default in v1.2 | Moving DNAT into netfilter removed the userspace hop entirely; kube-proxy became a control-plane process whose only job is writing rules. |
| (post predates ipvs) | ipvs mode exists but is deprecated and logs a warning. kubernetes.io: "Support for ipvs mode will be disabled by default from Kubernetes v1.40 (you can re-enable it with the KubeProxyIPVS feature gate)"; the gate itself was added in v1.37. kubeadm emits a deprecation warning. |
Alpha v1.8, beta v1.9, GA v1.11; deprecated v1.35; KubeProxyIPVS gate added v1.37; off by default from v1.40 (planned) |
IPVS was added because iptables "struggles to scale to tens of thousands of services because it is designed purely for firewalling purposes and is based on in-kernel rule chains" (KEP-265); it is being removed because SIG Network "lacks maintainers who are familiar with the ipvs backend code" and IPVS still needs iptables for the non-load-balancing parts anyway. |
| (post predates nftables) | The nftables backend is GA and is the recommended mode on modern kernels — but not the default at v1.37. |
Alpha v1.29, beta v1.31, GA/stable v1.33 (KEP-3866) | iptables has two unfixable costs: the kernel API cannot make incremental changes (adding one rule means lock, download the whole ruleset, re-upload it), and rule count is proportional to Service count so every packet traverses all of them. nftables gives a single atomic transaction plus verdict maps/sets, turning per-Service linear matching into a hash lookup. |
| — | KEP-5343 "Make nftables the default" landed its alpha in v1.37: kube-proxy now warns when the mode was defaulted rather than explicitly set, and kubeadm now writes mode: iptables explicitly. Beta targeted 1.39, default flip at 1.40. |
v1.37 (the warnings) | Two blockers had to clear first — nftables needs kernel ≥5.13 (5.10/5.15 leave LTS in Dec 2026), and localhost NodePorts were unsupported. Rather than silently switch the service data path mid-upgrade, kube-proxy warns for several releases so nobody's cluster changes backend by surprise. |
| — | nftables mode gained an opt-in userspace TCP proxy for localhost NodePort Services, and now talks netlink directly instead of shelling out to the nft binary (NFTablesNetlink, beta/on by default). |
v1.37 (KEP-6032; netlink PR #137536) | The nftables kernel API cannot DNAT traffic originating on 127.0.0.1, so localhost NodePorts — which iptables mode supported — needed a deliberate userspace relay; and parsing nft CLI output was the remaining per-sync bottleneck. |
Ingress lives in extensions/v1beta1. |
extensions/v1beta1 Ingress is gone, and so is networking.k8s.io/v1beta1. Only networking.k8s.io/v1 is served. |
Moved to networking.k8s.io/v1beta1 in v1.14; GA networking.k8s.io/v1 in v1.19; both beta versions removed in v1.22 |
extensions was the API group of last resort with no owning SIG and no version lifecycle; splitting Ingress/IngressClass/NetworkPolicy into networking.k8s.io gave SIG Network a group it owned and could version independently. |
Backends are serviceName + servicePort. |
backend.service.name plus backend.service.port.number (numeric) or backend.service.port.name (string); backends may also be resource references. |
v1.19 (networking.k8s.io/v1) |
A bare servicePort was type-ambiguous (string-or-int), and hard-coding "service" into the field name blocked non-Service backends such as object storage. |
The top-level catch-all is spec.backend. |
spec.defaultBackend. |
v1.19 | A rename to say what it actually is: the fallback used when no rule matches. |
path is a POSIX regular expression. |
pathType is required in v1 and must be one of Exact, Prefix, ImplementationSpecific; path need not be a valid regex. |
pathType added v1.18, required with no default in v1.19 |
The spec claimed POSIX regex but nginx used PCRE, Envoy ECMAScript, skipper re2, and most cloud load balancers no regex at all — so the field that existed for portability was the least portable thing in the API. Prefix defines element-wise path matching that every implementation can honour identically. |
Pick your controller with the kubernetes.io/ingress.class annotation. |
spec.ingressClassName field plus a cluster-scoped IngressClass resource (with spec.parameters for controller config and a default-class marker). |
Both added v1.18; GA v1.19 | Annotations are unvalidated free text and invisible to RBAC; making the class a field plus a resource lets the API server reject bad values and lets cluster operators — not app owners — define which classes exist at all. |
| Ingress supports exact hostnames. | Single-label wildcard hosts (*.foo.com) are formally part of the spec. |
v1.19 | Wildcard hosts were already widely implemented but unspecified, so behaviour varied per controller; GA froze one interpretation. |
| Ingress is where HTTP routing is going. | Ingress is frozen, not deprecated. kubernetes.io: "The Ingress API is generally available… The Kubernetes project has no plans to remove Ingress from Kubernetes… The Ingress API is no longer being developed, and will have no further changes or updates made to it." Gateway API is the recommended direction. | Statement current at the pin | Everything expressive had to be smuggled through vendor-specific annotations, so the portable API stopped being portable in practice; rather than keep bolting fields onto a design that could not express header matching or traffic weighting, SIG Network froze it and started a new API. |
| (post predates Gateway API) | Gateway API is the successor: "the next generation of Kubernetes Ingress, Load Balancing, and Service Mesh APIs… designed to be generic, expressive, and role-oriented", split across GatewayClass/Gateway/Route for three personas (infrastructure provider, cluster operator, application developer). | Gateway, GatewayClass and HTTPRoute reached v1 GA in gateway-api v1.0.0 (2023-10-31) — a gateway-api version, not a Kubernetes version |
The four stated design goals: role-oriented (shared infrastructure usable by teams that do not coordinate, under operator-set constraints), portable, expressive (header matching and traffic weighting, "only possible in Ingress through custom annotations"), and extensible (typed policy attachment instead of annotations). |
| — | GRPCRoute is GA (v1, Standard channel) and service mesh / GAMMA support is GA. |
gateway-api v1.1.0 (2024-05-09) | One API for north-south and east-west traffic: in the mesh case Routes attach directly to a Service and Gateway/GatewayClass drop out entirely, because a mesh has no distinct ingress point to model. |
| — | BackendTLSPolicy → Standard (gateway-api v1.4.0, 2025-10-06); TLSRoute and ListenerSet → Standard as v1 (v1.5.0, 2026-02-27); TCPRoute and UDPRoute → GA v1 (v1.6.0, 2026-06-29). Still Experimental at the pin: XBackend, SessionPersistence. Latest stable release at the pin: gateway-api v1.6.1 (2026-07-16). |
per-resource, gateway-api v1.1.0 → v1.6.x | Gateway API ships two channels — Standard is the stable/GA install, Experimental is the proving ground — and features graduate individually via GEPs rather than by a whole-API version bump. That is precisely why the bundle version and the per-resource maturity are different numbers. |
| Every Service has one Endpoints object listing all its endpoints. | EndpointSlice is the real API; kube-proxy no longer reads Endpoints for anything. v1.Endpoints is officially deprecated — still served, will never be removed. |
EndpointSlice alpha v1.16, beta v1.17, GA v1.21; kube-proxy consumes EndpointSlices by default on Linux in v1.19 (gates GA v1.22); Endpoints deprecated v1.33 (KEP-4974) | KEP-752: "Whenever a single pod in a service is added/updated/deleted, the whole Endpoints object… is re-computed, written to storage (etcd) and sent to all watchers." The object is also bounded by etcd's 1.5 MB limit, so a large Service simply stopped propagating endpoint updates at all, and a rolling update cost work proportional to endpoint count times one watch per node. Slices bound the object size and make each update incremental. |
| — | New Service features exist only on EndpointSlice: dual-stack, topology hints, and more than 1000 endpoints. Conformance now formally requires proxies and DNS implementations to read EndpointSlices. | conformance promotions v1.34; the DNS test and the service/proxy subresource moved off Endpoints in v1.37 |
Endpoints cannot be extended — no per-endpoint hints, no zone, one address family — so every feature invented after 2019 landed on the new API only; the old one survives as a frozen mirror maintained by a controller. |
ClusterIP, NodePort, LoadBalancer and headless (clusterIP: None) Services. |
All four still exist and behave the same way. ExternalName was added later. |
All four present at v1.0.0 (2015, verified in pkg/api/types.go); ExternalName added v1.4 |
The Service type ladder is one of the very few 2015 designs that survived untouched — all the churn was in the implementation beneath it. |
spec.topologyKeys steers traffic to same-zone endpoints. |
Removed. The field is gone from the API. | Removed v1.22 | Node-label-based topology could not express "spill over when the local zone has no capacity" and forced kube-proxy to reason about node labels itself; it was replaced by controller-computed per-endpoint hints plus internalTrafficPolicy. |
| — | Topology-aware routing via EndpointSlice hints, now GA. The service.kubernetes.io/topology-aware-hints annotation was renamed to service.kubernetes.io/topology-mode, and that annotation is now deprecated too — the field to use is spec.trafficDistribution: PreferClose. |
Hints alpha v1.21, beta v1.23, GA v1.33; annotation renamed v1.27; topology-mode deprecated v1.33; trafficDistribution alpha v1.30, beta v1.31, GA v1.33 |
The EndpointSlice controller allocates endpoints to zones in proportion to each zone's CPU capacity, so kube-proxy only has to filter on a hint it was handed rather than compute topology; the knob then moved from annotation to a real spec field so it could be validated and versioned. |
| Services are IPv4. | Dual-stack IPv4/IPv6 is GA: spec.ipFamilies[], spec.ipFamilyPolicy, spec.clusterIPs[]. |
First alpha v1.16; redesigned (breaking) alpha v1.20; beta v1.21; GA v1.23 (IPv6DualStack gate removed) |
The 1.16 design modelled dual-stack as a cluster-wide toggle and was scrapped; the 1.20 rewrite made address family a per-Service property so single-stack and dual-stack Services can coexist in one cluster. |
| — | spec.ports[].appProtocol carries the L7 protocol hint. |
Alpha v1.18, beta v1.19, GA v1.20 | protocol only says TCP/UDP/SCTP; load balancers and meshes needed to know a port was HTTP or gRPC without every vendor inventing another annotation for it. |
Only external traffic policy exists, set via the service.beta.kubernetes.io/external-traffic annotation. |
spec.externalTrafficPolicy is a field (Local/Cluster, renamed from OnlyLocal/Global), plus spec.internalTrafficPolicy for in-cluster traffic. spec.externalIPs is now deprecated. |
Annotation → field v1.7; internalTrafficPolicy beta v1.22, GA v1.26; externalIPs deprecated v1.36 (KEP-5707) |
Traffic policy became load-bearing enough to deserve schema validation; splitting internal from external let one Service preserve client IP in one direction without changing the other. |
| — | Terminating endpoints are used as a fallback rather than blackholed. | Proxy Terminating Endpoints alpha v1.22, beta v1.26, GA v1.28 | With externalTrafficPolicy: Local, a node whose only endpoints were terminating used to drop the traffic outright; serving terminating-but-still-ready endpoints removes that connection black hole during rollouts. |
Cluster DNS is kube-dns (kubedns + dnsmasq + sidecar containers). |
CoreDNS is the cluster DNS. kube-dns is not even an option in kubeadm. | CoreDNS alpha v1.9, beta v1.10, GA v1.11; kubeadm default v1.11; kube-up default v1.13 (KEP-566 stable v1.12); kube-dns rejected by kubeadm as of v1.21 | KEP-427: CoreDNS "has fewer moving parts than kube-dns… making it a single executable and single process", is "written in Go, making it memory-safe (kube-dns includes dnsmasq which is not)", is plugin-extensible, and shows "greater QPS, lower latency, and lower memory consumption". |
Pod /etc/resolv.conf gets search <ns>.svc.cluster.local svc.cluster.local cluster.local plus options ndots:5. |
Unchanged. Same search list, same ndots:5, same four-lookups-for-one-name amplification. |
no change | ndots:5 exists so that svc, svc.ns and svc.ns.svc all resolve; the cost is that any name with fewer than five dots walks the whole search list first — a tradeoff nobody has been willing to break compatibility to fix. |
| — | You can now shape it: dnsPolicy + dnsConfig are GA; the search-path limits rose from 6 → 32 entries and 256 → 2048 chars; and searches: ["."] is now a legal value that suppresses cluster-domain expansion entirely. |
KEP-504 GA v1.14; ExpandedDNSConfig alpha v1.22, beta v1.26, stable v1.28; relaxed search validation alpha v1.32, beta v1.33, stable v1.34 (KEP-4427) |
The 6/256 limits came from "obsolete criteria" that modern glibc/musl/Go resolvers no longer have; the single-dot search entry was legalised specifically so users could "avoid unnecessary DNS lookup calls to internal Kubernetes domain names". |
| — | NodeLocal DNSCache runs a per-node CoreDNS cache reachable at a link-local IP. | Alpha v1.13, beta v1.15, GA v1.18 (KEP-1024) | Every pod DNS query otherwise crossed the node boundary to a central Service through kube-proxy's conntrack; a node-local cache removes both the network hop and the UDP conntrack pressure that produced the classic 5-second DNS timeouts. |
| Every pod gets its own IP, flat network, no NAT pod-to-pod. | Still exactly true — the one part of the 2015 model that never changed. But now "a CNI plugin is required to implement the Kubernetes network model". | Model unchanged; the CNI requirement was formalised | The flat/no-NAT model was the durable contract; only the mechanism for satisfying it was replaced. |
The kubelet sets up pod networking itself — --configure-cbr0, the docker bridge, "classic" mode. |
Removed. The kubelet does no bridge setup at all. | --configure-cbr0 and "classic" networking mode removed in v1.5 |
Baking one bridge topology into the kubelet meant every new network model needed a kubelet patch; pushing it behind a plugin interface made the network pluggable without touching the node agent. |
| — | CNI (and kubenet) first appeared as kubelet network plugins in v1.2 (pkg/kubelet/network/cni, .../kubenet), later moved under dockershim; at the pin CNI is outside kubelet's scope entirely. |
CNI + kubenet added v1.2; kubelet CNI management removed v1.24 | kubernetes.io: "Prior to Kubernetes 1.24, the CNI plugins could also be managed by the kubelet using the cni-bin-dir and network-plugin command-line parameters. These command-line parameters were removed in Kubernetes 1.24, with management of the CNI no longer in scope for kubelet" — the CRI runtime owns the pod sandbox, so it owns the sandbox's network. |
| — | dockershim is removed, and kubenet went with it — kubenet lived at pkg/kubelet/dockershim/network/kubenet, which is present at v1.23.0 and 404s at v1.37. Kubelet flags --network-plugin, --cni-conf-dir, --cni-bin-dir, --cni-cache-dir, --network-plugin-mtu are gone. |
Deprecated v1.20, both removed v1.24 (kubenet removal inferred from the source tree — see the caveat below) | kubenet was only ever reachable through dockershim's network plugin manager, so deleting the in-tree Docker CRI shim removed the last in-tree network plugin as a side effect; a third-party CNI plugin is now mandatory. |
| (post predates NetworkPolicy) | NetworkPolicy is GA in networking.k8s.io/v1; the extensions/v1beta1 version is gone, and the net.beta.kubernetes.io/network-policy namespace-isolation annotation was removed. |
GA networking.k8s.io/v1 in v1.7; extensions/v1beta1 removed v1.16 |
v1beta1 opted a whole namespace into isolation via an annotation; v1 made isolation a per-pod property derived from whether any policy's podSelector matches the pod, so policies compose additively instead of fighting a namespace-wide switch. |
[UNVERIFIED] — kubenet's removal release. No CHANGELOG entry announces kubenet's removal by name; a grep of every CHANGELOG from 1.20 to 1.37 for kubenet returns zero hits, and PR/commit search found nothing either. The v1.24 figure is instead established structurally: pkg/kubelet/dockershim/network/kubenet exists at tag v1.23.0 and the parent directory 404s at v1.24.0. The conclusion is sound but is inferred from the source tree, not stated in a release note.
Caveat on the nftables default-flip schedule. KEP-5343's kep.yaml carries status: provisional while listing alpha: "v1.37". The v1.37 warning behaviour is independently confirmed in CHANGELOG-1.37 (PRs #139957, #139777), so the alpha did ship — but the 1.39-beta / 1.40-default schedule is the KEP's stated plan, not a landed fact, and a provisional KEP can still move. Same caution for the "ipvs off by default in v1.40" line, which comes from the current docs' forward-looking statement.
Caveat on the Gateway API rationale. No GEP number is cited for the role-oriented design rationale: that text lives in the project's site docs (site/content/en/docs/introduction/_index.md) rather than in a numbered GEP.
Traps — networking¶
--proxy-mode=userspaceis not a mode you can fall back to. Readers treat it as the safe, boring option; it has been out of the tree since v1.26. Relatedly, the userspace proxy used to open a real listening socket per Service on the node, so any 2015 troubleshooting advice of the form "runnetstaton the node and look for kube-proxy's port for your Service" finds nothing on any supported cluster — iptables/ipvs/nftables modes do DNAT with no listener at all. The one exception is new and narrow: v1.37's opt-in userspace TCP proxy for localhost NodePorts in nftables mode.- "iptables is the default" is correct today and wrong from 1.40, and the warning is easy to miss. v1.37 kube-proxy logs a warning when the mode was defaulted rather than set. Any cluster that has never written an explicit
mode:is scheduled to silently change its entire service data path on a future upgrade. The actionable advice is to write the mode down now, whichever one you want. spec.topologyKeysis silently ignored, not rejected. It was removed from the schema in v1.22 rather than made invalid, so a manifest carrying it applies cleanly and routes traffic with no topology awareness whatsoever, with no warning. The same shape of trap applies toservice.kubernetes.io/topology-aware-hints: renamed in v1.27 (both forms accepted, and they must agree), and its successorservice.kubernetes.io/topology-modewas itself deprecated in v1.33 and will never go GA.kubernetes.io/ingress.classis an annotation, so it cannot fail validation. Anetworking.k8s.io/v1Ingress carrying the old annotation and noingressClassNameis accepted unconditionally. Whether it works depends entirely on whether that particular controller still honours the legacy annotation — several do, several don't, and none of them tell the API server. The symptom is "my Ingress exists and never gets an address".pathTypereverses the direction of the default. Inv1beta1the field was absent orImplementationSpecific; inv1it is required with no default. So a converted manifest either fails to apply (loud, fine) or — if a conversion tool fills it in — getsPrefixsemantics where regex-ish behaviour was assumed. AsPrefix,/foo/barmatches/foo/bar/bazbut not/foo/barbaz; a regex interpretation may have matched both. Routing changes quietly.kubectl get endpointsstill works, and that is the problem. The Endpoints controller still runs at v1.37, so the command returns plausible output forever — but kube-proxy has not read Endpoints in years, Gateway API conformance forbids implementations from using them, and dual-stack, topology hints and Services above ~1000 endpoints are simply not represented there. Debugging a routing problem throughkubectl get endpointsis debugging a mirror, not the data path.kubenetis gone but still reads like an option. Old posts (and some docs pages) mention it without stating the removal, which is only visible as a side effect of dockershim's deletion in v1.24. There is no "use kubenet if you don't need a real CNI plugin" fallback, and--network-plugin=cniis not a deprecated kubelet flag — it is an unknown flag, so the kubelet refuses to start.- Gateway API version numbers are not Kubernetes version numbers, and per-resource maturity is a third axis. At v1.6.1, HTTPRoute/GRPCRoute/TLSRoute/TCPRoute/UDPRoute are all
v1Standard — but installing the Experimental channel gets you resources (XBackend, SessionPersistence) whose names and API group will change on graduation; the v1.3.0 notes state that migrating to Standard "will require recreating these resources with the standard channel names and API Group (both lacking thexprefix)". Anx-prefixed resource in production is a scheduled rewrite. - "Ingress is deprecated" is wrong in a way that matters. Ingress is frozen: full GA stability guarantees, no removal planned, and Gateway API's own FAQ says "There are no plans to deprecate this API." Do not schedule an Ingress migration against an imaginary removal deadline — but do migrate if you need any feature invented after 2020, because none will ever land in Ingress.
ndots:5did not change, so every stale DNS-tuning post is still live advice — including the bad parts. What did change is the tooling:dnsConfig.searchesaccepts 32 entries / 2048 chars since v1.28, and since v1.34searches: ["."]suppresses cluster-domain expansion entirely. A post that says "you can't fix search-path amplification, just use FQDNs everywhere" is describing a constraint that no longer holds.
Storage¶
Removals in this section were verified against the source tree at release tags/branches as well as the CHANGELOGs, because at least one CHANGELOG entry is wrong (see the azureFile row and the Traps).
| What the post says | What is true at the pin | Changed in | The decision behind it |
|---|---|---|---|
Volume plugins live in the Kubernetes tree; to support new storage you add code to kubernetes/kubernetes. |
CSI is the only supported extension path; the in-tree csi plugin is a thin adapter that speaks gRPC to an out-of-tree driver over a Unix socket. |
CSI alpha v1.9, beta v1.10, GA v1.13 (CSI spec v1.0) | Per the CSI design proposal, in-tree plugins were "linked, compiled, built, and shipped with the core kubernetes binaries", so "bugs in volume plugins can crash critical Kubernetes components" and plugins "get full privileges of kubernetes components (kubelet and kube-controller-manager)". An RPC boundary contains both the blast radius and the privilege. |
| Vendors ship storage support by getting a PR into the next Kubernetes release. | Vendors ship a container image on their own cadence; Kubernetes versions only the CSI spec. | v1.9 → v1.13 | "Volume plugin development is tightly coupled and dependent on Kubernetes releases" and "Kubernetes developers/community are responsible for testing and maintaining all volume plugins, instead of just testing and maintaining a stable plugin API." |
| Cloud credentials and vendor SDKs live inside kube-controller-manager, which provisions disks itself. | In-tree cloud providers are gone; provisioning is done by external CSI controllers holding their own credentials. | AWS/EBS removed v1.27, OpenStack/Cinder v1.26, Azure v1.30, vSphere v1.30, last GCP v1.31; k8s.io/legacy-cloud-providers deleted from staging v1.31 |
Vendor code linked into core binaries meant plugin developers were "forced to make plugin source code available", every cluster shipped every vendor's dependency tree, and that code ran with control-plane credentials; extraction shrank the binaries and the security surface at once. |
| FlexVolume is the way to write a driver without touching core. | flexVolume is still compiled in and functional at v1.37 (pkg/volume/flexvolume, with the kubelet's dynamic plugin prober), but deprecated. No removal release has been announced. |
Introduced v1.2 (absent at v1.1.1, present at v1.2.0); deprecated v1.23 |
Flex "requires access to the root filesystem of node and master machines in order to deploy the third party driver files" and inherits the host's mount/filesystem tool dependencies; a containerised CSI driver needs neither. |
awsElasticBlockStore: in a Pod or PV just works. |
The field is still accepted by the API, but the in-tree plugin is gone; it works only if ebs.csi.aws.com is installed, via the CSI translation shim. |
Deprecated v1.17, migration beta v1.17, on by default v1.23, GA v1.25, plugin removed v1.27 (PR #115838) | csi-translation-lib rewrites the legacy PV spec into a CSI spec at bind/mount time, because Kubernetes' API deprecation policy forbids removing the GA field itself — so existing PV objects have to keep working. |
gcePersistentDisk: just works. |
Same: field accepted, redirected to pd.csi.storage.gke.io. Docs: the driver "was deprecated in the Kubernetes v1.17 release and then removed entirely in the v1.28 release". |
Deprecated v1.17, beta v1.17, on by default v1.23, GA v1.25, plugin removed v1.28 (PR #117055) | Same translation-shim mechanism; the in-tree code path was deleted once migration was GA and the gate locked. |
azureDisk: just works. |
Redirected to disk.csi.azure.com; in-tree code deleted. |
Migration alpha v1.15, beta v1.19, GA v1.24, gate locked v1.25, plugin removed v1.27 (PR #116301) | Same. |
azureFile: just works. |
Redirected to file.csi.azure.com; in-tree code deleted. |
Migration alpha v1.15, beta v1.21, GA v1.26, plugin removed v1.30 (PR #122576). CHANGELOG-1.28 claims removal in v1.28 (PR #118236) — that commit was reverted, and pkg/volume/azure_file is demonstrably still present on release-1.28 and release-1.29. |
The revert happened because the removal outran the migration's readiness; the lesson for this table is that the source tree, not the CHANGELOG, is the authority on removals. |
cinder: for OpenStack. |
Removed; use cinder.csi.openstack.org. |
Migration beta v1.18, GA v1.24, removed v1.26 together with the whole in-tree OpenStack provider (PR #67782) | Cinder provisioning was entangled with the in-tree OpenStack cloud provider, so the two could only leave together. |
vsphereVolume: / VMDK paths. |
Redirected to csi.vsphere.vmware.com; in-tree code deleted. |
Alpha v1.18, beta v1.19, GA v1.26 — and then briefly un-locked again in v1.26 (PR #116396) pending vSphere CSI GA for Windows/XFS/raw block; plugin removed v1.30 | The unlock is the mechanism worth teaching: SIG Storage deliberately let users fall back to the in-tree plugin for a release because the CSI driver lacked feature parity, rather than stranding those workloads. |
rbd: / cephfs: for Ceph. |
Completely non-functional. Both plugins were removed and their CSI-migration support was removed, so there is no translation shim — old PVs cannot be rescued in place. | RBD: migration alpha v1.23, beta v1.27, deprecated v1.28, removed v1.31. CephFS: migration KEP withdrawn (alpha v1.26 only), deprecated v1.28, removed v1.31 | Migration was abandoned rather than finished, so unlike the cloud drivers there is no spec-rewriting path: per the v1.31 notes, "a re-deployment of your application is required to use the new driver". |
glusterfs: volumes and the Gluster dynamic provisioner. |
Removed entirely; "volumes must be migrated to an alternate storage solution before upgrading to v1.26". | Deprecated v1.25, removed v1.26 (PR #112015) | No CSI-migration shim was ever built, so this is a hard break by design rather than by neglect. |
scaleIO:, flocker:, quobyte:, storageos:. |
All removed, no shim. | scaleIO v1.22; flocker, quobyte, storageos v1.25 | Untested, unmaintained plugins with no in-tree CSI counterpart — deleted rather than migrated. |
portworxVolume: is a normal in-tree volume type. |
In-tree plugin removed; operations are redirected to pxd.portworx.com, which must be installed before the upgrade. |
Alpha v1.23, beta v1.25 (off by default), GA v1.33, in-tree plugin removed v1.36 (PR #135322; gates CSIMigrationPortworx and InTreePluginPortworxUnregister deleted) |
The last in-tree vendor plugin to complete migration. The translation shim (csi-translation-lib/plugins/portworx.go) stays behind so existing PVs keep binding. |
| Any of the ~25 in-tree volume types is fair game. | Only these remain in pkg/volume at v1.37: configmap secret downwardapi projected emptydir hostpath local nfs iscsi fc image csi flexvolume git_repo — and the last two are deprecated/disabled. |
Cumulative, v1.22 → v1.36 | This is the end state of the CSI migration programme: the only plugins that stayed are the ones with no external storage system behind them, or that are the extension mechanism. |
--feature-gates=CSIMigration... is how you opt in. |
No CSIMigration* or InTreePlugin*Unregister gate exists at v1.37. |
CSIMigration locked GA v1.25, gate removed v1.27; per-driver gates removed v1.27 → v1.36; the six InTreePlugin*Unregister gates deleted in v1.31 |
Once a migration is GA and the in-tree code is deleted there is nothing left to toggle, so the gates were garbage-collected. |
| PersistentVolume + PersistentVolumeClaim are the storage abstraction. | Unchanged — still core/v1 PersistentVolume / PersistentVolumeClaim. |
Both present at tag v1.0.0 (pkg/api/v1/types.go) |
The stable part of the story: the object model survived intact while every implementation beneath it was replaced. That is why a 2015 PVC example is still pedagogically valid. |
| An admin pre-creates PVs by hand; a PVC binds to whatever matches. | Dynamic provisioning is the norm: a PVC with no storageClassName gets the default class and a freshly provisioned volume. |
Dynamic provisioning alpha v1.2 (volume.alpha.kubernetes.io/storage-class); StorageClass beta v1.4 (storage.k8s.io/v1beta1 + volume.beta.kubernetes.io/storage-class); GA v1.6 (storage.k8s.io/v1 + spec.storageClassName) |
"Kubernetes previously required all volumes to be manually provisioned by a cluster administrator before use"; a class object lets the provisioner be named and parameterised per-claim instead of per-cluster, which is what removed the human from the critical path. |
Use volume.alpha.kubernetes.io/storage-class to trigger provisioning. |
Silently ignored. CHANGELOG-1.6: "Alpha version of dynamic volume provisioning is removed in this release. Annotation volume.alpha.kubernetes.io/storage-class does not have any special meaning." |
v1.6 (PR #40000) | The alpha annotation was replaced by a real API object; stripping its meaning rather than rejecting it kept old manifests admissible — which is exactly what makes it a trap. |
Use volume.beta.kubernetes.io/storage-class. |
Still honoured at v1.37 — GetPersistentVolumeClaimClass falls back to it when spec.storageClassName is unset — but the API emits deprecated since v1.8; use "storageClassName" attribute instead. |
Field added v1.6; annotation deprecated v1.8; still read at v1.37 | Kept for compatibility, with spec.storageClassName winning when both are present — so a stale annotation is invisible rather than fatal. |
| There is no such thing as a default StorageClass. | storageclass.kubernetes.io/is-default-class: "true" plus the DefaultStorageClass admission plugin stamp the class onto class-less PVCs. If several classes are marked default, the newest wins. |
DefaultStorageClass plugin + storageclass.beta.kubernetes.io/is-default-class in v1.4; GA annotation name from v1.6 (both constants are still read at v1.37); multiple-defaults tie-break by newest in v1.26 |
Resolving the class at admission time means the PVC object records the class it actually got, so later changes to which class is default cannot silently re-target claims that already exist. |
| A PVC created before any StorageClass existed stays class-less forever. | Retroactive default StorageClass assignment back-fills the default class onto still-unbound class-less PVCs. | Alpha v1.25, beta v1.26, GA v1.28, gate removed v1.29 | Before this, a PVC created while no default existed was stuck Pending permanently even after an admin added one, because nothing re-evaluated it; the PV controller now does. |
| A PVC is provisioned immediately on creation. | With volumeBindingMode: WaitForFirstConsumer the PVC stays Pending until a Pod that consumes it is scheduled — the scheduler picks the node first, and the volume is provisioned to match. |
volumeBindingMode/WaitForFirstConsumer in storage/v1 at release-1.9 (alpha, VolumeScheduling gate), beta v1.10, GA v1.13, gate deleted v1.16 |
The volume-topology design doc states it directly: "The process of binding and provisioning volumes are done before a pod is scheduled. Therefore, it cannot take into account any of the pod's other scheduling constraints. This makes it possible for the PV controller to bind a PVC to a PV or provision a PV with constraints that can make a pod unschedulable" — a zonal disk provisioned into a zone with no remaining CPU strands the pod forever, so the binding decision had to move to where the node is chosen. |
| The scheduler knows nothing about how much space a driver has left. | Storage capacity tracking (CSIStorageCapacity) lets the scheduler avoid nodes whose driver cannot satisfy the claim. |
Alpha v1.19, beta v1.21, GA v1.24; storage.k8s.io/v1beta1 CSIStorageCapacity removed v1.27 |
It is the necessary complement to WaitForFirstConsumer: once the scheduler commits to a node before provisioning, it needs capacity data or it will confidently commit to a node where provisioning then fails. |
| Snapshots aren't a thing. | VolumeSnapshot / VolumeSnapshotContent / VolumeSnapshotClass in snapshot.storage.k8s.io/**v1** — shipped as CRDs plus an out-of-tree controller (kubernetes-csi/external-snapshotter, latest v8.6.0, 2026-05-28). v1beta1 is still in the CRD but served: false. |
Alpha v1.12 (with a breaking second alpha in v1.13), beta v1.17, GA v1.20; VolumeSnapshotDataSource GA v1.20 |
Snapshots ship as CRDs + sidecar rather than built-in types precisely so the snapshot controller follows the CSI out-of-tree lifecycle instead of the Kubernetes release train — the same reasoning that moved the drivers out. |
| — (no 2015 equivalent) | Volume group snapshots: groupsnapshot.storage.k8s.io, with v1 served. |
Alpha v1.27, beta v1.32, stable v1.36 (KEP-3476) | A crash-consistent snapshot across several volumes needs one CSI call spanning the set, which a per-PVC snapshot API cannot express at all. |
| A PV's size is fixed once created. | allowVolumeExpansion: true on the StorageClass plus editing spec.resources.requests.storage on the PVC grows the volume, online, with no Pod restart. |
Field present in storage/v1 at release-1.9; alpha v1.8, beta v1.11, online expansion beta v1.15, GA v1.24; gates ExpandPersistentVolumes/ExpandInUsePersistentVolumes/ExpandCSIVolumes deleted v1.27 |
Expansion is a two-phase controller-then-kubelet operation (ControllerExpandVolume then NodeExpandVolume), which is why growing a volume that is in use needed its own separate graduation from growing an unattached one. |
| — | Recover from volume expansion failure: a failed expansion request can now be shrunk back down; the PVC carries status.allocatedResourceStatus. |
Alpha v1.23, beta v1.32, GA v1.34 (PR #132662). Note v1.31 required manually clearing allocatedResourceStatus on ControllerResizeFailed/NodeResizeFailed PVCs. |
Previously an over-large expansion request was terminal, because PVC size requests were monotonic and could not be lowered; recording allocated resources in a separate status field breaks the one-way door. |
emptyDir is your scratch volume; medium: Memory gives you a tmpfs. |
Still true, but sizeLimit now actually bounds the tmpfs. Without the feature, "the size of this memory backed volume is defaulted to 50% of the memory on a Linux host". |
SizeMemoryBackedVolumes alpha v1.20, beta v1.22, GA v1.32, gate removed v1.35. v1.37 adds alpha InPlacePodVerticalScalingMemoryBackedVolumes (resize the tmpfs live) and an emptyDir mode field. |
KEP-1967: "The coupling of default memory backed volume size with the host that runs the pod makes pod definitions less portable across node instance types and providers" — sizing to pod-allocatable memory instead of host memory makes the manifest self-describing. |
| — | ReadWriteOncePod access mode: exactly one Pod (not one node) may use the PVC. |
Alpha v1.22, beta v1.27, GA v1.29, gate removed v1.31 | ReadWriteOnce only ever meant one node, so two Pods co-scheduled on the same node could both mount a volume whose data format requires a single writer — a silent corruption path with no way to express the constraint. |
| — | Generic ephemeral volumes: an inline ephemeral.volumeClaimTemplate in the Pod spec gives a full PVC lifecycle scoped to the Pod. |
Alpha v1.19, beta v1.21, GA v1.23, gate removed v1.25 | Gives emptyDir ergonomics with real StorageClass features (snapshot restore, expansion, topology) by driving the ordinary PVC provisioning path instead of a parallel one. |
| — | CSI inline ephemeral volumes: csi: directly in the Pod spec, for driver-managed ephemeral data such as secrets stores. |
Alpha v1.15, beta v1.16, GA v1.25, gate removed v1.27 | Distinct from generic ephemeral: no PVC and no provisioning round-trip, because the driver handles the whole lifecycle inside NodePublishVolume. |
gitRepo: clones a repo into a volume. |
Dead. The field is still in core/v1 and the plugin is still registered, but SetUp returns git-repo volume plugin has been disabled; if necessary, it may be re-enabled by enabling the feature-gate GitRepoVolumeDriver — and since v1.36 that gate is LockToDefault: true, Default: false, so the error's own advice is impossible to follow. Docs: "The last version that provided a way to use this driver was Kubernetes v1.35." |
Deprecated v1.11; disabled by default with an opt-in gate v1.33; locked off v1.36; removal targeted v1.39 (KEP-5040 stable: "1.39" — a plan, not a shipped fact) |
KEP-5040: "gitRepo volume types can be exploited to gain remote code execution as root on the nodes as shown in CVE-2024-10220" — the kubelet ran git as root against attacker-controlled repository contents, and git hooks execute. |
| — | VolumeAttributesClass + ModifyVolume: change a volume's QoS/IOPS class after provisioning. |
Alpha v1.29, storage.k8s.io/v1beta1 v1.31, GA storage.k8s.io/v1 in v1.34 (PR #131549) |
StorageClass parameters are immutable after provisioning, so retuning IOPS previously meant recreating the volume and copying the data. |
A PVC dataSource can only be a snapshot or a PVC clone. |
AnyVolumeDataSource lets dataSource/dataSourceRef point at any CRD, handled by a volume populator controller. |
Alpha v1.18, beta v1.24, GA v1.33, gate removed v1.37 | Populators run out-of-tree as controllers watching PVCs, which keeps "restore from X" logic out of the core PV controller — the same out-of-tree principle as CSI itself. |
Reclaim policy Delete deletes the backing volume. |
It does — but before v1.33, deleting the PV before the PVC could leak the backing volume. HonorPVReclaimPolicy now finalizes correctly. |
Alpha v1.23, beta v1.31, GA v1.33 (KEP-2644) | A finalizer on the PV makes reclaim ordering-independent, instead of the outcome depending on which of the two objects a user happened to delete first. |
| Volumes are filesystems. | Raw block volumes (volumeMode: Block, volumeDevices) are GA. |
Alpha v1.9, beta v1.13, GA v1.18 | Databases that want the raw device should not pay for a filesystem layer they immediately bypass. |
[UNVERIFIED] — FlexVolume's removal release: there isn't one. There is no FlexVolume-removal KEP under keps/sig-storage, no removal entry in any CHANGELOG through 1.37, and pkg/volume/flexvolume is still registered by the v1.37 kubelet. The docs still say only "FlexVolume is deprecated" (feature-state for_k8s_version="v1.23" state="deprecated"). This row asserts the absence of a removal, not a release number — do not write "removed in 1.x" for FlexVolume.
Caveat on the v1.36 rows. The v1.36 "Removals and Major Changes" and sneak-peek blog posts both fetched truncated before their storage sections. Every v1.36 claim above (Portworx in-tree removal, the gitRepo lock-off) is instead sourced from CHANGELOG-1.36.md plus a release-1.35 vs release-1.36 source-tree diff and kube_features.go — equally primary, and more precise than the blog prose would have been.
Traps — storage¶
- The API still accepts every dead volume type. At
release-1.37,core/v1VolumeSourceandPersistentVolumeSourcestill declareawsElasticBlockStore,gcePersistentDisk,azureDisk,azureFile,cinder,vsphereVolume,rbd,cephfs,glusterfs,portworxVolume,quobyte,storageos,scaleIO,flockerandgitRepo. Deprecation policy forbids removing GA API fields, so a 2015 manifest is validated and accepted:kubectl applysucceeds, the PV is created, the Pod is admitted and scheduled — and then the kubelet has no plugin for it. The failure is a mount-time event on a node minutes later, not an admission error in CI. - There are two very different kinds of "removed", and readers will conflate them. For
awsElasticBlockStore,gcePersistentDisk,azureDisk,azureFile,cinder,vsphereVolumeandportworxVolume,csi-translation-lib/plugins/still ships a translator at v1.37, so the old spec is rewritten into a CSI spec at bind/mount time — those manifests work if and only if the matching CSI driver is installed. Same YAML, same cluster version, works or doesn't depending on a Deployment that appears nowhere in the manifest. Forrbd,cephfs,glusterfs,quobyte,storageos,scaleIOandflockerthere is no translator: unconditionally dead, and per the v1.31 notes "a re-deployment of your application is required". Anyone who learns the slogan "in-tree plugins were migrated to CSI" will assume the first case covers Ceph. It does not. - The
azureFileCHANGELOG is wrong. CHANGELOG-1.28 says "Removed the deprecatedazureFilein-tree storage plugin" (PR #118236); that commit was reverted on 2023-06-02, andpkg/volume/azure_fileis present on bothrelease-1.28andrelease-1.29. The real removal was PR #122576 in v1.30. Citing CHANGELOGs without checking the tree publishes a wrong release number here. - Annotations that are ignored rather than rejected.
volume.alpha.kubernetes.io/storage-classhas had no meaning since v1.6, so a 2015 manifest relying on it produces a PVC with no class, which then either binds to nothing or silently picks up the cluster default — neither is an error. Meanwhilevolume.beta.kubernetes.io/storage-classdoes still work at v1.37, but only as a fallback: addspec.storageClassNamelater and the annotation is silently outranked, provisioning from a different class. And bothis-default-classannotation names (storageclass.andstorageclass.beta.) are still read, so a cluster can have a default set through a path nobody thinks to check. - A PVC that stays
Pendingforever, and is working correctly. WithvolumeBindingMode: WaitForFirstConsumer— the default in most modern StorageClasses, including cloud CSI and in-treelocal—kubectl get pvcshowsPendinganddescribesayswaiting for first consumer to be created before binding. A reader whose 2015 mental model is "create the PVC, watch it Bind, then create the Pod" concludes storage is broken and starts debugging the driver. Nothing is broken. The inverse is worse: a StorageClass copied from an old post withoutvolumeBindingModegetsImmediate, silently reintroducing exactly the zonal-stranding bug the field was created to fix — invisible until a Pod becomes unschedulable in a zone that has the disk but no CPU. gitRepofails at mount time, in a security-relevant way, and its error message lies to you. The field validates, the Pod is admitted, the plugin is still listed incmd/kubelet/app/plugins.go— andSetUptells you to enableGitRepoVolumeDriver, a gate that has beenLockToDefault: truesince v1.36 and therefore cannot be enabled. The reason it is locked shut is CVE-2024-10220, root RCE on the node via git hooks — name that in any exercise, or someone "fixes" it with a fork or an initContainer that shells out togitas root.emptyDir: {medium: Memory}with asizeLimitused to be a lie. On any cluster before v1.32 the tmpfs was sized at 50% of host memory regardless ofsizeLimit. A post from that era that pairsmedium: Memorywith a smallsizeLimitand calls it a memory cap is describing a bound that was not enforced; at the pin it is. Note also that tmpfs bytes count against the writing container's memory limit, so a pod that "worked" under the old unenforced sizing can now hit a different wall.
Cluster lifecycle¶
| What the post says | What is true at the pin | Changed in | The decision behind it |
|---|---|---|---|
"Bring up a cluster with cluster/kube-up.sh; set KUBERNETES_PROVIDER=gce\|aws\|vagrant\|azure\|libvirt-coreos\|photon-controller\|openstack-heat." |
cluster/kube-up.sh still exists in kubernetes/kubernetes but only GCE plus the skeleton/pre-existing stubs remain; cluster/README.md declares the whole directory in "maintenance mode … not accepting new providers". Every other provider directory is gone. |
Azure kube-up removed v1.6 (commit 4708714dd, 2017-02-17); AWS, Vagrant, photon-controller, openstack-heat, libvirt-coreos all removed v1.10 (commits 7a75a7b43, 8d44e0b38, 553a3f049, 29fa89d9d, e77373199, Jan 2018); centos/local removed v1.15 (8762bc39a, 2019-04-17); juju removed v1.19 (b08cd0e2c, 2020-04-23) |
Each provider in cluster/ was a shell-and-Salt fork of the same bootstrap logic that only its original author tested, so the release team was gating Kubernetes releases on provider scripts nobody could reproduce; the project chose to keep exactly one (GCE, because CI used it) and push everything else to out-of-tree installers with their own owners and their own CI. |
"Node configuration is driven by Salt — see cluster/saltbase/salt/kube-apiserver/… and the pillar files." |
There is no Salt anywhere in kubernetes/kubernetes. cluster/saltbase returns 404. |
v1.10 (commit 5f9735de5 "cluster: delete saltbase", 2018-01-13; confirmed as an ancestor of v1.10.0 and not of v1.9.0) |
Salt was a second, parallel configuration language that had to be kept in sync by hand with every flag change in the Go binaries, and it only ran on the distros the cluster/ scripts supported; moving node config into the kubelet's own versioned KubeletConfiguration file and the control plane into static-pod manifests removed the need for an external config-management system inside the Kubernetes repo. |
"The install docs are in the Kubernetes repo at docs/getting-started-guides/…, and the user guide at docs/user-guide/…." |
docs/ in kubernetes/kubernetes contains only .gitignore and OWNERS. All prose lives in kubernetes/website and is published at kubernetes.io. |
v1.7 (commit ac86803a7 "Remove docs/ other than auto generated", 2017-06-02, plus 5da316d44 "Remove stubs from docs/", 2017-06-25; both ancestors of v1.7.0) |
Docs pinned to a source tag could not be corrected without cutting a Kubernetes patch release, and every doc PR had to pass the code repo's CI; splitting the website out let docs be versioned per-release and fixed continuously. |
"Generate the CA and server certificates with cluster/…/make-ca-cert.sh and copy them onto each node." |
Certificates are generated by kubeadm init phase certs into /etc/kubernetes/pki, and node certificates are issued by the cluster itself via the certificates.k8s.io CSR API (kubelet TLS bootstrapping) rather than pre-placed. |
Kubelet TLS bootstrapping beta v1.7; kubelet client-certificate rotation beta v1.8, RotateKubeletClientCertificate GA v1.19 |
Hand-copying a long-lived key onto every node meant one leaked key compromised the cluster forever and there was no rotation story; the CSR API lets a node prove identity once with a short-lived bootstrap token, receive a signed cert, and then renew it itself before expiry, so no long-lived node key ever leaves the node. |
| "Nodes join because their kubelet was pre-configured with a kubeconfig you copied there." | Nodes join with kubeadm join --token <id>.<secret> --discovery-token-ca-cert-hash sha256:<hash>. A bootstrap token is a Secret of type bootstrap.kubernetes.io/token in kube-system, is time-limited, and authenticates into the system:bootstrappers:kubeadm:default-node-token group, which is RBAC-bound to auto-approve node client CSRs. |
Bootstrap Tokens alpha v1.6, beta v1.8; default token TTL changed from infinite to 24h in v1.8. Bootstrap-token GA release: [UNVERIFIED] — I grepped CHANGELOG-1.6 → 1.34 for "bootstrap token" co-occurring with GA/stable/graduated and found no promotion note; treat "beta since 1.8" as the last confirmed status word. | The join step needs mutual trust with a single short string: the token authenticates the node to the API server, and the --discovery-token-ca-cert-hash pins the cluster CA so a node cannot be tricked into joining an impostor API server — the copied-kubeconfig approach gave neither expiry nor CA pinning. |
"Authorization is AlwaysAllow (or ABAC with a policy file you hand-write)." |
RBAC is the norm; kubeadm turns on RBAC plus the Node authorizer and NodeRestriction admission plugin by default. |
RBAC beta (rbac.authorization.k8s.io/v1beta1, with default control-plane/node/controller roles) v1.6; RBAC v1 GA v1.8; Node authorizer + NodeRestriction beta and enabled by kubeadm v1.7 |
ABAC policy was a file on the apiserver's disk, so changing a permission meant editing a file and restarting the apiserver — not an API object, therefore not manageable by the cluster itself; RBAC made permissions first-class API objects, and the Node authorizer added the specific restriction ABAC could not express: a kubelet may only read the Secrets and ConfigMaps of pods actually bound to its own node. |
"Start the apiserver with --insecure-port=8080 and talk to it over plain HTTP on localhost." |
The insecure port is gone. --address/--insecure-bind-address have no effect and --port/--insecure-port may only be 0. |
Deprecated v1.10; serving on the insecure port removed v1.20; the flags themselves removed in v1.24 | An unauthenticated, unauthorized HTTP endpoint with full cluster privileges was a permanent privilege-escalation primitive for anything that could reach the host's loopback (including a hostNetwork pod), and it existed only so bootstrap scripts could skip TLS; once TLS bootstrapping and kubeconfig generation were automated the shortcut had no remaining justification. |
"Point the apiserver at etcd with --storage-backend=etcd2 / run etcd 2.x." |
etcd v3 only. kubeadm v1.37 deploys etcd 3.7.0; the minimum supported external etcd is 3.5.24. etcd itself removed the --enable-v2 and --experimental-enable-v2v3 flags in etcd v3.6.0 (2025-05-15). |
etcd2 storage backend deprecated v1.9/v1.10; removed from kube-apiserver in v1.13 ("kube-apiserver has removed support for the etcd2 storage backend … existing clusters must migrate etcd v2 data to etcd v3 storage before upgrading to v1.13"); v2 API/store serving flags removed in etcd v3.6.0 (component version, not a k8s release) |
etcd v2 kept the whole keyspace in memory as a tree with no transactions, no leases usable for watch bookmarks, and no efficient range reads, which capped cluster size; etcd v3's flat MVCC keyspace with gRPC watch and multi-key transactions is what made resourceVersion-based watch, the aggregated API server, and 5000-node clusters viable. The migration is one-way — that is why the removal came with a hard "back up first" note. |
| "The control plane runs from init scripts / a Salt-managed systemd unit on the master." | The control plane runs as static pods: kubeadm writes kube-apiserver, kube-controller-manager, kube-scheduler and (local) etcd manifests into /etc/kubernetes/manifests, which the kubelet watches and reconciles. |
Established as the kubeadm default from v1.4 onward and documented as the etcd + control-plane phases of kubeadm init |
Static pods let the kubelet be the only process-supervisor on the box, so upgrading the control plane becomes "write a new manifest with a new image tag" rather than "reconfigure the host's init system" — which is exactly what makes kubeadm upgrade possible. |
| "kubeadm? Never heard of it." | kubeadm is the reference bootstrapper: kubeadm init runs the phases preflight → certs → kubeconfig → etcd → control-plane → kubelet-start → wait-control-plane → upload-config → upload-certs → mark-control-plane → bootstrap-token → kubelet-finalize → addon/coredns → addon/kube-proxy → show-join-command. |
Introduced alpha in v1.4 ("[alpha] The kubeadm tool makes it much easier to bootstrap Kubernetes"); beta in v1.6; GA in v1.13 ("SIG Cluster Lifecycle is pleased to announce the long awaited promotion of kubeadm to stable GA") |
The cluster/ scripts configured Kubernetes "from the outside, by poking all the bits into place"; SIG Cluster Lifecycle deliberately chose instead to improve the core so bootstrap became a supported first-class flow — and to draw a hard line at machine provisioning, networking and non-critical addons so kubeadm stayed a composable building block rather than a second installer. |
| "Upgrading means re-running the install scripts and hoping." | kubeadm upgrade plan / kubeadm upgrade apply / kubeadm upgrade node upgrade the control plane (including etcd) and are covered by the GA deprecation policy. |
kubeadm upgrade plan first appeared v1.8; --etcd-upgrade added v1.9; external-etcd setups supported v1.10; kubeadm upgrade GA v1.13 |
Upgrade is only tractable if the installer knows the exact desired state of the previous version; kubeadm stores its own ClusterConfiguration in the kube-system/kubeadm-config ConfigMap during upload-config, so upgrade can diff intent-vs-actual instead of re-deriving it from shell variables that were never recorded. |
"kubeadm has an experimental --feature-gates=SelfHosting=true that pivots the control plane into DaemonSets." |
Gone. There is no self-hosted control-plane mode in kubeadm. | Deprecated v1.12; moved out of kubeadm init under kubeadm alpha v1.13; kubeadm alpha selfhosting pivot removed v1.21 |
Self-hosting created a bootstrap circularity — the apiserver that schedules the apiserver — so any control-plane outage became unrecoverable without an out-of-band static-pod fallback; the static-pod design keeps the kubelet as an unconditional escape hatch. |
| "Configure the kubelet with a couple of dozen command-line flags." | The kubelet reads a versioned KubeletConfiguration file via --config; flags that have a config-file equivalent are deprecated. kubeadm writes /var/lib/kubelet/config.yaml. The API is still kubelet.config.k8s.io/v1beta1 at the pin — k8s.io/kubelet/config/v1 exists but holds CredentialProviderConfig, not KubeletConfiguration. |
kubeletconfig API graduated alpha→beta and was renamed to kubelet.config.k8s.io/v1beta1 in v1.10; the KubeletConfigFile feature gate was removed as redundant in v1.10; config-file-equivalent flags deprecated v1.10; --config is loaded with strict deserialization (unknown/duplicate fields are a fatal error) since v1.17 |
Flags are unversioned, undiscoverable and unstructured, so a node's effective configuration could not be read back, diffed, or validated; a versioned API object can be defaulted, converted between versions, and shipped in a ConfigMap — which is the prerequisite for any installer being able to upgrade node config rather than regenerate a command line. |
| "componentconfig is a vague plan on a design doc." | ComponentConfig has landed unevenly and the maturity level differs per component at the pin: scheduler kubescheduler.config.k8s.io/v1 (GA), kubelet kubelet.config.k8s.io/v1beta1 (still beta), kube-proxy kubeproxy.config.k8s.io/v1alpha1 (still alpha), kubeadm kubeadm.k8s.io/v1beta4 with a v1 still an explicitly-marked placeholder. |
Scheduler ComponentConfig GA in v1.25 ("Kube-Scheduler ComponentConfig is graduated to GA, kubescheduler.config.k8s.io/v1 is available now"); the legacy scheduler --policy-config-file/--policy-configmap path was removed in v1.23 |
Each SIG had to design its own config schema and carry its own conversion/round-trip tests, so promotion happened per component rather than as one flag day; the practical consequence is that a config snippet copied from a blog can be for an API version that is alpha, beta or GA depending on which binary it configures. |
"kubeadm config uses MasterConfiguration / kubeadm.k8s.io/v1alpha1." |
The current kubeadm config API is kubeadm.k8s.io/v1beta4, the default since v1.31; v1beta3 no longer exists. A kubeadm.k8s.io/v1 package exists in tree but its own doc.go calls it a placeholder with an open tracking issue (kubernetes/kubeadm#3160) and a TODO for the migration release, so v1beta4 is the version to write at the pin. |
MasterConfiguration split into InitConfiguration/ClusterConfiguration/JoinConfiguration/KubeletConfiguration/KubeProxyConfiguration in v1alpha3 (v1.12); v1beta1 in v1.13; v1alpha2 removed v1.13; v1beta1-and-older unsupported from v1.22; v1beta2-and-older unsupported from v1.27; v1beta4 enabled and made the stored format in v1.31; v1beta3 removed in v1.37 (commit 852f1e23e "kubeadm: remove the v1beta3 API and PublicKeysECDSA feature gate", 2026-01-04; ancestor of v1.37.0, not of v1.36.0) |
kubeadm's config is a file on an operator's disk, so it needs the same deprecation discipline as a served API but without an apiserver to do conversion; kubeadm therefore ships kubeadm config migrate and drops old versions on a fixed cadence, which is why a config file from a 2018 post is not merely deprecated but unparseable. |
| "Dynamic Kubelet Configuration lets you push a ConfigMap to change node config at runtime." | Gone entirely. | DynamicKubeletConfig deprecated v1.22 (gate off by default); removed from the kubelet in v1.24; removed from the API server in v1.26 |
The feature let the API server change a node's config, which inverted the trust boundary (a compromised control plane could reconfigure kubelets) and produced nodes whose running config matched no file on disk, making rollback and skew-debugging impossible. |
"--cloud-provider=aws\|gce\|azure\|vsphere links the cloud integration into the kubelet and controller-manager." |
In-tree cloud provider code is gone; integrations run as an out-of-tree cloud-controller-manager. |
KEP-2395 Removing In-Tree Cloud Provider Code: alpha v1.21, beta v1.29, stable/complete v1.31. In-tree Azure and vSphere providers removed in v1.30; the remaining "legacy cloud provider integration code" removed in v1.31. | Vendor SDKs compiled into kube-controller-manager meant a cloud API change forced a Kubernetes release, cloud credentials had to be readable by a core binary, and every cluster shipped every vendor's code; the CCM interface let each provider ship on its own cadence with its own credentials and its own blast radius. |
| "Kubernetes supports the last N releases — probably whatever you're running." | The project maintains release branches for the most recent three minor releases (at the pin: 1.37, 1.36, 1.35), with ~14 months of support per minor (12 months standard + 2 months maintenance mode). | Support window extended to ~1 year for v1.19 and newer (was ~9 months for v1.18 and older), per the yearly-support KEP (sig-release KEP-1498) | Three releases a year against a nine-month window meant many operators were forced to upgrade more than once a year just to stay supported; the yearly window was chosen so an organisation can upgrade once per year and never fall out of support. |
| "Skew doesn't matter much; keep everything on the same version." | Explicit skew policy at the pin, with apiserver 1.37: HA kube-apiserver instances within 1 minor of each other; kubelet and kube-proxy may be up to 3 minor versions older (1.37/1.36/1.35/1.34) and never newer; kube-controller-manager/kube-scheduler/cloud-controller-manager at most 1 minor older (1.37, 1.36); kubectl ±1 (1.38, 1.37, 1.36). |
The kubelet/kube-proxy allowance widened from 2 to 3 minor versions for components ≥ v1.25 ("kubelet < 1.25 may only be up to two minor versions older") |
The three-release support window and the ~14-month patch life only pay off if a fleet can upgrade the control plane and drain nodes on different schedules; widening the kubelet allowance to three minors means one control-plane upgrade per year does not force a same-quarter reimage of every node. |
"hack/local-up-cluster.sh starts a dev cluster." |
Still there, still the in-repo dev loop — and since v1.8 it enables RBAC and the Node authorizer by default, so the ENABLE_RBAC variable a 2015/2016 post sets is no longer read (use AUTHORIZATION_MODE). |
v1.8 | Dev clusters that ran with AlwaysAllow hid every RBAC bug until it reached a real cluster; making the dev loop match the production default moved authorization failures to the earliest possible point. |
Traps — cluster lifecycle¶
- The
KUBERNETES_PROVIDERvariable is silently inert for anything but GCE.cluster/kube-up.shstill exists, so a 2015 post's command still runs; it just cannot find the provider directory. Readers conclude their environment is broken rather than that the instruction is a decade stale. - A
kubeadm.k8s.io/v1alpha1/v1alpha3/v1beta1/v1beta2/v1beta3config file is a hard parse failure, not a deprecation warning. kubeadm drops old config API versions outright (v1beta1-and-older gone in v1.22, v1beta2-and-older in v1.27, v1beta3 in v1.37) andkubeadm config migratecannot bridge more than one hop — a v1beta1 file needs kubeadm 1.15–1.21 to get to v1beta2 first. Any tutorial withMasterConfigurationin it is unusable as written. kubelet --configuses strict deserialization since v1.17. A single misspelled or since-renamed field in a copiedKubeletConfigurationmakes the kubelet refuse to start, with an error about the file rather than about the field's meaning. This is the opposite failure mode from flags, which silently ignored typos in some tooling.kubelet.config.k8s.io/v1beta1looks like it must be old. It is current. A reader who "modernises" the apiVersion tov1produces a file the kubelet rejects, becausek8s.io/kubelet/config/v1is the credential-provider API, notKubeletConfiguration. The same trap applies in reverse to kube-proxy, whose config API is genuinely stillv1alpha1at the pin.--insecure-port=8080no longer opens a port, and no longer errors either (the flag is accepted only as0, and was removed in v1.24). Every 2015 example that doescurl http://localhost:8080/api/v1/podson a control-plane node just hangs or gets connection-refused; the reader has no signal that the concept was removed.- etcd2→etcd3 is one-way. A post that says "back up etcd and upgrade" is understating:
etcdctl migrate/etcdutl migrateproduces a v3 datadir that cannot be read by the older server, so the only rollback is the pre-migration snapshot. And at the pin the v2 store is not even present in etcd (3.6.0 removed--enable-v2), so there is no "just re-enable v2" fallback. - A post that says "Salt is in
cluster/saltbase, edit the pillar to change a kubelet flag" sends the reader to a directory that 404s. The equivalent operation today is editing thekubelet-configConfigMap inkube-system(or/var/lib/kubelet/config.yaml) and restarting the kubelet — a completely different mechanism, not a renamed file. - Self-hosting sounds like the modern design and is the abandoned one. "Run the control plane as DaemonSets/Deployments so Kubernetes manages itself" reads as more advanced than static pods, so readers pursue it; it was removed in v1.21 precisely because it is a bootstrap cycle.
- Version skew asymmetry bites in one direction only.
kubectlmay be one minor newer than the apiserver, butkubeletmay never be newer. A post that says "just use the latest of everything" is fine for kubectl and a policy violation for nodes.
Monitoring and observability¶
| What the post says | What is true at the pin | Changed in | The decision behind it |
|---|---|---|---|
"Heapster collects cluster metrics; deploy it from cluster/addons/cluster-monitoring (Heapster + InfluxDB + Grafana) or set ENABLE_CLUSTER_MONITORING." |
Heapster does not exist. kubernetes-retired/heapster is archived (last push 2018-11-30) and its README says "RETIRED: Heapster is now retired". The cluster/addons/cluster-monitoring directory is gone; cluster/addons/metrics-server is what remains. |
Heapster's own deprecation timeline: initial deprecation Kubernetes 1.11 (no new features or sinks), setup-script removal Kubernetes 1.12, retirement / move to kubernetes-retired Kubernetes 1.13 ("Heapster remains at v1.6.0-beta, but is now retired in Kubernetes 1.13"). The cluster/addons/cluster-monitoring directory was deleted in v1.18 (commit 31fb04fa9, 2019-11-21). |
Heapster was one binary trying to be two products: a metrics API for the autoscaler and a fan-out shipper into a dozen third-party "sinks". The sinks were unmaintainable (each needed a vendor-specific client in-tree) and the short-term in-memory metrics store did not scale, so the project split the job: a minimal, scalable component for the Metrics API, and Prometheus-format scraping for everything else. |
"Heapster is where kubectl top and the autoscaler get their numbers." |
kubectl top and the HPA read the Metrics API (metrics.k8s.io), served by metrics-server through the aggregation layer. metrics-server is an optional addon — a cluster without it answers kubectl top with "Metrics API not available". Latest metrics-server at the pin: v0.9.0 (2026-07-13, component version). |
HPA could opt into the resource metrics API with --horizontal-pod-autoscaler-use-rest-clients=true in v1.8; it became the default in v1.9; the Heapster path and the flag were removed in v1.22. kubectl top's Heapster-related flags were deprecated in v1.19. |
Reading metrics through an aggregated API rather than a hard-coded client means the provider is swappable: the HPA no longer needs to know whether the numbers come from metrics-server, a vendor adapter, or a Prometheus adapter — it only needs the metrics.k8s.io contract. |
| "The Metrics API is a beta API you shouldn't rely on." | metrics.k8s.io reached GA in v1.37 — KEP-5207 metrics.k8s.io API definition, status: implemented, stage: stable, latest-milestone: v1.37, with milestone: {alpha: v1.6, beta: v1.8, stable: v1.37}. The v1 package landed in tree as commit 0ad054896 "KEP-5207: Promote metrics.k8s.io to GA" (2026-05-01), an ancestor of v1.37.0 and not of v1.36.0. v1beta1 and v1alpha1 are still present in tree. |
v1.37 (alpha v1.6, beta v1.8) | The API had been de facto stable and load-bearing (kubectl top, HPA, VPA, every cloud autoscaler) for eight years while formally beta, which meant no compatibility guarantee covered the one metrics surface everything depended on; GA'ing the existing shape — rather than redesigning it — locked in what the ecosystem had already built against. |
| "Custom metrics come from Heapster's model API." | Custom and external metrics come from the aggregated custom.metrics.k8s.io and external.metrics.k8s.io APIs, implemented by adapters (e.g. prometheus-adapter). These are still beta at the pin: custom.metrics.k8s.io has v1beta1 + v1beta2, external.metrics.k8s.io has v1beta1 only. |
Consumed via autoscaling/v2beta1 from v1.6 onwards; not GA at the pin |
Custom metrics have no single canonical shape — the API is a query interface over arbitrary user metrics — so it stayed beta while the resource-metrics API (a fixed CPU/memory shape) could be frozen and promoted. |
"The HPA uses autoscaling/v1 with a target CPU percentage." |
autoscaling/v2 is the recommended API for all use cases: multiple metrics, target.type: Utilization\|Value\|AverageValue, resource/pods/object/external metric sources, and spec.behavior scale-up/scale-down stabilisation windows and policies. autoscaling/v1 is still served for the simple CPU case. |
autoscaling/v2beta1 v1.6; v2beta2 v1.12; spec.behavior added in v2beta2 in v1.18; autoscaling/v2 GA in v1.23; v2beta1 removed v1.25; v2beta2 removed v1.26 |
autoscaling/v1's targetCPUUtilizationPercentage was a single scalar, so it could not express "scale on queue depth and CPU" or "scale up fast but down slowly"; v2 generalised the target into a typed metric source so the controller's decision logic became data-driven rather than one hard-coded ratio. |
"cAdvisor runs on each node and you can browse its UI at http://node:4194/." |
Nothing listens on 4194. cAdvisor is a library compiled into the kubelet; there is no cAdvisor web server. | --cadvisor-port deprecated v1.10; default changed to 0 (server off) in v1.11 ("The formerly publicly-available cAdvisor web UI that the kubelet ran on port 4194 by default is now turned off by default"); the UI and the --cadvisor-port flag were entirely removed in v1.12. Recommended replacement at the time: run cAdvisor yourself as a DaemonSet. |
An unauthenticated HTTP server exposing every container's resource usage and filesystem layout on a fixed port of every node is an information-disclosure surface that no cluster operator asked for; the same data was already available, authenticated, through the kubelet's own API — so the second server was pure attack surface. |
"Scrape cAdvisor's JSON API on the kubelet at /stats/container." |
Removed. Container metrics come from the kubelet's Prometheus endpoints or the Summary API (/stats/summary). |
--enable-cadvisor-json-endpoints deprecated v1.15, disabled by default in v1.18, and the cAdvisor JSON endpoints (/stats/container, /stats/<podname>/<containername>, /stats/<namespace>/<podname>/<poduid>/<containername>) were removed from the kubelet in v1.21. |
The JSON endpoints were cAdvisor's own schema leaking through the kubelet — an unversioned surface Kubernetes had never designed and could not evolve; consolidating on Prometheus text format plus the versioned Summary API gave one schema Kubernetes actually owns. |
| "Monitoring means running Heapster and pointing it at a sink." | Two separate pipelines, per the official split: a resource metrics pipeline (kubelet → metrics-server → metrics.k8s.io → kubectl top/HPA) and a full metrics pipeline (Prometheus-format scraping of /metrics, /metrics/cadvisor, /metrics/resource, /metrics/probes on each kubelet, plus kube-state-metrics for object state). Latest kube-state-metrics at the pin: v2.20.0 (2026-08-18, component version). |
The two-pipeline model dates from the Heapster deprecation in v1.11; the retirement README names it explicitly ("For basic CPU/memory HPA metrics: metrics-server … For general monitoring: a third-party monitoring pipeline that can gather Prometheus-formatted metrics") | The autoscaler needs a small number of current values with low latency and hard availability requirements; a dashboard needs a large number of historical values and can tolerate gaps. One component optimised for both is optimised for neither — which is why metrics-server deliberately dropped Heapster's historical store and its sinks. |
"Node/container metrics live on the kubelet's /metrics." |
The kubelet splits its Prometheus surface across several endpoints: /metrics (kubelet's own), /metrics/cadvisor (container metrics, added in v1.8, commit f5f989382 "Restore cAdvisor prometheus metrics to the main port"), /metrics/probes (liveness/readiness probe results, added in v1.11), and /metrics/resource (the resource-metrics endpoint metrics-server scrapes). |
/metrics/cadvisor v1.8; /metrics/probes v1.11; /metrics/resource/v1alpha1 deprecated in v1.18 in favour of /metrics/resource, and metrics/resource/v1alpha1 removed in v1.20. /metrics and /metrics/probes were labelled ALPHA stability in v1.16. |
Container metrics are enormous compared to the kubelet's own, so serving them from one path forced every scraper to pay for both; splitting the paths lets a Prometheus scrape config take cadvisor at a slow interval and probe results at a fast one. /metrics/resource exists separately because it is a minimal, stable CPU/memory surface designed for metrics-server rather than a general-purpose dump. |
"Give the metrics collector RBAC on nodes/stats." |
metrics-server needs nodes/metrics, not nodes/stats, because it scrapes /metrics/resource rather than /stats/summary. |
metrics-server v0.6.0 (2022-01-20, component version) — "Metrics Server now requires access to nodes/metrics RBAC resource instead of nodes/stats" + "Migrate to Kubelet resource metrics endpoint" |
Reading /stats/summary meant metrics-server parsed the whole Summary API — filesystem stats, network stats, per-volume stats — to extract two numbers; scraping the purpose-built /metrics/resource cut both the payload and the kubelet's CPU cost of computing stats it did not need. |
| "InfluxDB + Grafana is the standard cluster monitoring stack." | Neither ships with Kubernetes. InfluxDB cluster monitoring was deprecated as part of the Heapster deprecation; Grafana is a fine dashboard but is deployed by the user, typically against Prometheus. | InfluxDB cluster monitoring deprecated v1.11; the addon directory removed v1.18 | The addon locked every cluster into one time-series database chosen in 2014, shipped and CVE-patched by the Kubernetes release team; making the storage layer the user's choice removed a whole class of "Kubernetes ships a database" maintenance from the release process. |
| "The Kubernetes Dashboard shows graphs (it gets them from Heapster)." | The graphs that depended on Heapster's short-term in-memory store were explicitly named as functionality metrics-server does not replace ("it doesn't include some features of Heapster, such as short term metrics for graphs in kube-dashboard and dedicated push sinks, which proved hard to maintain and scale"). Dashboards at the pin come from Prometheus/Grafana. | v1.11 | Serving graph history required Heapster to be a time-series database inside the cluster's critical path; the project judged that job belonged to an actual TSDB, accepting a visible feature regression in the dashboard as the cost. |
Traps — monitoring and observability¶
- Port 4194 fails closed, silently. A 2015 post's
curl http://<node>:4194/api/v1.3/subcontainersgets connection-refused, which reads as a firewall problem. There is nothing to unblock: the server was deleted in v1.12. The reader will spend the debugging time on iptables, not on the docs. kubectl topfailing is not a cluster fault. On a cluster with no metrics-server,kubectl top nodeserrors out. A post from 2015 assumes the monitoring addon was installed by the bootstrap script; at the pin nothing installs it for you unless your distro does. The fix is "install metrics-server", not "fix the API server".nodes/statsRBAC still exists as a subresource, so a copied ClusterRole grants a permission that is no longer the one needed. The apply succeeds; metrics-server then fails to scrape with a 403 onnodes/metrics. Nothing in the RBAC output hints at the mismatch.metrics.k8s.io/v1beta1manifests and clients still work at the pin, so nothing forces you to notice v1 exists. Conversely, a reader who sees "GA in v1.37" and writesmetrics.k8s.io/v1against a 1.36 cluster gets a not-found on the API group — the GA is one release old at the pin, so version-check before recommending it.autoscaling/v2beta1andv2beta2manifests are rejected, not warned about (removed in v1.25 and v1.26 respectively). Every HPA example between 2017 and 2021 uses one of them. The error is a plain "no matches for kind" and does not say "useautoscaling/v2".- The
--horizontal-pod-autoscaler-use-rest-clientsflag is gone from kube-controller-manager (v1.22), so a post that tells you to set it tofalseto "keep using Heapster" makes the controller-manager refuse to start. This is one of the rare loud failures in this table — worth using as an exercise precisely because it does fail loudly. - "Heapster is deprecated" understates it by five years. Some 2018-era posts say "deprecated, still works". At the pin there is no image being rebuilt, no CVE patching, and the HPA cannot consume it at all. Treat any Heapster instruction as delete-and-replace, never as migrate-later.
/metrics/resource/v1alpha1is a path, not an API version you can negotiate. A scrape config from 2019 pointing at it returns 404 since v1.20, and Prometheus reports the target as down with an HTTP error — easy to misread as a kubelet auth problem.
Sources¶
Grouped by part; each part's claims were verified against its own list.
Part 1 — API surface, workload objects, kubectl¶
Every URL below was actually opened (via gh api, WebFetch, or the GitHub web path) in the course of this research. Grouped by kind of source, in the order of authority the ticket specified.
Official kubernetes.io reference docs¶
- https://kubernetes.io/docs/reference/using-api/deprecation-guide/ — authoritative removals; read in full as
kubernetes/websitecontent/en/docs/reference/using-api/deprecation-guide.md. Newest section is v1.32; per-item anchors used: #v1-16, #v1-22, #v1-25, #deployment-v116, #networkpolicy-v116, #psp-v116, #psp-v125, #ingress-v122, #customresourcedefinition-v122, #webhook-resources-v122, #cronjob-v125, #poddisruptionbudget-v125, #horizontalpodautoscaler-v125, #horizontalpodautoscaler-v126, #flowcontrol-resources-v126, #csistoragecapacity-v127, #flowcontrol-resources-v129, #flowcontrol-resources-v132 - https://kubernetes.io/docs/reference/using-api/deprecation-policy/ — the numbered rules (Rule #1, #3, #4a, #4b) and the beta support window
- https://kubernetes.io/docs/reference/using-api/server-side-apply/
- https://kubernetes.io/docs/concepts/workloads/controllers/deployment/
- https://kubernetes.io/docs/concepts/workloads/controllers/replicationcontroller/ — the "Legacy API… Superseded by the Deployment and ReplicaSet APIs" wording, and the absence of the word "deprecated"
- https://kubernetes.io/docs/concepts/security/pod-security-admission/
- https://kubernetes.io/docs/reference/kubectl/conventions/ — checked for the (now absent) generators section
- https://kubernetes.io/docs/reference/kubectl/generated/kubectl_run/ — the pin-era synopsis and full flag list
- https://kubernetes.io/docs/reference/kubectl/generated/kubectl_apply/ —
--field-managerdefaultkubectl-client-side-apply - https://kubernetes.io/docs/reference/kubectl/generated/kubectl_scale/ —
--current-replicasstill documented - https://kubernetes.io/docs/reference/kubectl/generated/kubectl_rollout/
KEPs (kubernetes/enhancements)¶
- KEP-555 Apply (server-side apply) and its
kep.yaml - KEP-753 Sidecar Containers and its
kep.yaml - KEP-3136 Beta APIs Off by Default
- KEP-1635 Prevent PermaBeta
- KEP-859 kubectl command headers
- KEP-3659 kubectl apply --prune redesign
- Also enumerated (to rule items in or out):
keps/sig-api-machinery/2155-clientgo-apply,keps/sig-api-machinery/4222-cbor-serializer,keps/sig-api-machinery/5958-client-opt-out-managedfields,keps/sig-node/4438-sidecar-restart-termination
CHANGELOG files in kubernetes/kubernetes¶
Read in full or grepped: 1.2, 1.5, 1.6, 1.8, 1.9, 1.10, 1.11, 1.12, 1.13, 1.14, 1.16, 1.17, 1.18, 1.19, 1.20, 1.21, 1.22, 1.24, 1.25, 1.26, 1.27, 1.31, 1.33, 1.34, 1.35, 1.36, 1.37 — all at https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-<x.y>.md. There is no CHANGELOG-1.0.md or CHANGELOG-1.1.md; pre-1.2 history came from release bodies instead.
GitHub release bodies (the only release notes for pre-1.2)¶
- https://github.com/kubernetes/kubernetes/releases/tag/v0.15.0
- https://github.com/kubernetes/kubernetes/releases/tag/v0.18.0
- https://github.com/kubernetes/kubernetes/releases/tag/v0.19.0
- https://github.com/kubernetes/kubernetes/releases/tag/v1.0.0
- https://github.com/kubernetes/kubernetes/releases/tag/v1.37.0 — establishes the pin: published
2026-08-26T16:29:12Z
In-tree design docs and proposals, read at a release tag¶
docs/api.md@ v1.0.0 — the v1beta1/2/3 deletion dates and both sets of conversion tipsdocs/proposals/api-group.md@ v1.1.1docs/design/versioning.md@ v1.1.1docs/proposals/deployment.md@ v1.2.0docs/design/simple-rolling-update.md@ v1.2.0, and its archived copycli/simple-rolling-update.mdcontributors/devel/sig-architecture/api_changes.md— alpha/beta/stable conventions, multiple API versions, the star-topology conversion walkthrough
Source-tree reads used as proof of behaviour at a given release¶
- v1.37.0:
pkg/registry/core,pkg/registry/apps/rest/storage_apps.go,pkg/registry/{autoscaling,batch}/rest/storage_*.go,pkg/registrylisting (noextensions),staging/src/k8s.io/kubectl/pkg/cmd/cmd.go,.../cmd/alpha.go,.../cmd/apply/apply.go,.../cmd/run/run.go,.../cmd/scale/scale.go,staging/src/k8s.io/cli-runtime/pkg/genericclioptions/record_flags.go,staging/src/k8s.io/api/{apps,extensions,resource} pkg/apis/apps/v1alpha1/register.go@ v1.3.0pkg/apis/extensions/v1beta1/defaults.go@ v1.15.0 vspkg/apis/apps/v1/defaults.gostaging/src/k8s.io/kubectl/pkg/polymorphichelpers/rollback.gopkg/kubectl/cmd/apply.go@ v1.1.1 vspkg/kubectl/cmd@ v1.0.0pkg/kubectl/cmd/stop.go@ v1.7.0 (absent at v1.8.0)- Earlier tags read the same way for flag/alias presence:
get.goandget_flags.goat v1.13.0/v1.17.0/v1.18.0/v1.19.0,cmd.goat v1.6.0–v1.20.0,run.goat v1.11.0/v1.17.0/v1.18.0,expose.goat v1.19.0–v1.23.0,generate/versioned/generator.goat v1.17.0,pkg/features/kube_features.goat v1.14.0–v1.32.0,apimachinery/.../meta/v1/types.goat v1.13.0/v1.14.0,pkg/kubectl/kubectl.goat v0.18.0/v1.0.0/v1.1.0
Commits¶
a3f4e6e— "Rename --experimental- flags to -- for server-side apply"1c3233a1— the "Remove v1beta3" commit, dated after v1.0.0; attributed to the v1.1 cycle viacompare/1c3233a1...v1.1.1→aheadkubernetes/websitePR #1856 "Add deprecation policy" (merged 2017-01-03) — first appearance of the numbered-rule deprecation policy document
Issues¶
Pull requests in kubernetes/kubernetes¶
All at https://github.com/kubernetes/kubernetes/pull/<n>. Where a release attribution rests on a PR, the milestone field was read via gh api repos/kubernetes/kubernetes/pulls/<n> rather than inferred from the merge date.
#6098 #6118 #7454 #8596 #8634 #8917 #9027 #9269 #10525 #13958 #14621 #14626 #19042 #19840 #19882 #25816 #28844 #35663 #35731 #36021 #36355 #38614 #39683 #46151 #46927 #48746 #49340 #49930 #49935 #50998 #51465 #53679 #53920 #54445 #55148 #56206 #60210 #60793 #61285 #68132 #69167 #69255 #70728 #70820 #71006 #72249 #72947 #73787 #74057 #78098 #81956 #87077 #87498 #88004 #88057 #88649 #88655 #91946 #96138 #96190 #96878 #98111 #99155 #99423 #99557 #99732 #99900 #99915 #102873 #106824 #110564 #111319 #113819 #114228 #114584 #116429 #116510 #117160 #120246 #121579 #124598 #125437 #125842 #127058 #129373 #129731 #129970 #131463 #133841 #134625 #135003 #135391 #136016 #136643 #137145 #137755 #139726 #140294 #140296
Part 2 — Security, authorization, policy, and the container runtime¶
kubernetes/enhancements (kep.yaml on master, via gh api repos/kubernetes/enhancements/contents/keps/...)
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-auth/2579-psp-replacement — PSA: alpha v1.22, beta v1.23, stable v1.25
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-auth/1205-bound-service-account-tokens — alpha v1.13, beta v1.21, stable v1.22
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-auth/2799-reduction-of-secret-based-service-account-token — beta v1.24, stable v1.30
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-auth/4193-bound-service-account-token-improvements — alpha v1.29, beta v1.30, stable v1.32
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-auth/3299-kms-v2-improvements — alpha v1.25, beta v1.27, stable v1.29
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-auth/3221-structured-authorization-configuration — alpha v1.29, beta v1.30, stable v1.32
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-auth/3331-structured-authentication-configuration — alpha v1.29, beta v1.30, stable v1.34
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-auth/4601-authorize-with-selectors — alpha v1.31, beta v1.32, stable v1.34
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-auth/4633-anonymous-auth-configurable-endpoints — alpha v1.31, beta v1.32, stable v1.34
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-auth/4940-psa-block-host-field-in-probes — stable v1.34
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-auth/5681-conditional-authorization — alpha v1.37
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-api-machinery/3488-cel-admission-control — VAP: alpha v1.26, beta v1.28, stable v1.30
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-api-machinery/3962-mutating-admission-policies — MAP: alpha v1.32, beta v1.34, stable v1.36
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/2221-remove-dockershim — stage stable, latest-milestone v1.24
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/2040-kubelet-cri — alpha v1.5, beta v1.23
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/2254-cgroup-v2 — alpha v1.18, beta v1.22, stable v1.25
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/4569-cgroup-v1-maintenance-mode — alpha/beta/stable v1.31
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/5573-remove-cgroup-v1 — implementable, beta v1.35
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/127-user-namespaces — alpha v1.25, beta v1.35, stable v1.36
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/2033-kubelet-in-userns-aka-rootless — alpha v1.22, beta v1.37
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/2413-seccomp-by-default — alpha v1.22, beta v1.25, stable v1.27
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/24-apparmor — alpha v1.4, beta v1.4, stable v1.31
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/135-seccomp — seccomp to GA
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/277-ephemeral-containers — alpha v1.16, beta v1.23, stable v1.25
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/4210-max-image-gc-age — alpha v1.29, beta v1.30, stable v1.35
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/4033-group-driver-detection-over-cri — alpha v1.28, beta v1.31, stable v1.34
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-release/3031-signing-release-artifacts — alpha v1.24, beta v1.26
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-release/3027-slsa-compliance — provisional, latest-milestone v1.23
kubernetes/kubernetes CHANGELOGs
- https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.3.md — alpha RBAC API group; beta PodSecurityPolicy; NetworkPolicy v1beta1 (#25638); PSP admission controller (#24600); "Switch to ABAC authorization from AllowAll" (#24210)
- https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.4.md — beta AppArmor; container image policies
- https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.5.md — CRI v1alpha1 alpha (#54 feature); rkt 1.21.0 in CI matrix
- https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.6.md — RBAC beta / v1beta1; kubeadm creates RBAC-enabled clusters; ENABLE_LEGACY_ABAC (#43544); --authorization-rbac-super-user deprecated (#38121)
- https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.7.md — NetworkPolicy GA networking.k8s.io/v1 (#39164); encryption-at-rest alpha
- https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.8.md — RBAC v1 GA (#49642); rbac/v1alpha1 disabled by default; advanced auditing beta
- https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.10.md — PSP moved to policy/v1beta1 (#54933); rktnetes deprecated (#58418); KMS gRPC envelope service (#55684)
- https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.11.md — rktnetes code removed (#61432)
- https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.12.md — audit.k8s.io/v1 default change announced for 1.13
- https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.13.md — EncryptionConfiguration / apiserver.config.k8s.io/v1 stable (#67383); --encryption-provider-config rename (#71206); audit v1 default (#70476)
- https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.19.md — seccomp GA; annotations deprecated, removal in v1.22
- https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.20.md — dockershim deprecation (#94624)
- https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.21.md — PSP deprecated (#97171)
- https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.24.md — dockershim removed (#97252); dockershim flags removed (#106907); kubeadm containerd default + crictl (#107317); LegacyServiceAccountTokenNoAutoGeneration beta on by default (#108309)
- https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.25.md — PSP removed (#109798); PodSecurity GA + pod-security.admission.config.k8s.io/v1 (#110459); cgroups v2 stable
- https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.26.md — CRI v1alpha2 support dropped (#110618)
- https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.27.md — kubelet --container-runtime removed (#114017); endpoint flags migrated to config (#112136)
- https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.30.md — ValidatingAdmissionPolicy GA, enabled by default (#123405)
- https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.31.md — AppArmor GA (#125257); AppArmor hard reject (#125776); cgroup v1 warning/event/metric (#125328)
- https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.35.md — failCgroupV1 true by default, cgroup v1 deprecated (#134298); kubeadm opt-out (#134744); UserNamespacesHostNetworkSupport gate (#134893)
- https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.36.md — MutatingAdmissionPolicy GA, enabled by default (#136039); UserNamespacesSupport GA (#136792)
- https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.37.md — Conditional Authorization machinery alpha (#137513, #137204); Pod Certificates GA (#139579); NodeRestriction PodCertificateRequest check (#140006); CRI CheckpointPod/RestorePod (#140366)
kubernetes/kubernetes source at historical tags
- cmd/kubelet/app/options/options.go @ v1.2.0 — --container-runtime "Possible values: 'docker', 'rkt'. Default: 'docker'.", --rkt-path, --rkt-stage1-image
- pkg/apis/extensions/types.go and plugin/pkg/admission/ @ v1.1.1, v1.2.0, v1.3.0 — PSP types absent at v1.1.1, present at v1.2.0; enforcing admission plugin directory first appears at v1.3.0
kubernetes.io (docs at v1.37 unless noted)
- https://kubernetes.io/docs/reference/access-authn-authz/authorization/
- https://kubernetes.io/docs/reference/command-line-tools-reference/kube-apiserver/ — --authorization-mode "Defaults to AlwaysAllow if --authorization-config is not used"
- https://kubernetes.io/docs/reference/access-authn-authz/abac/
- https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/ — v1.37 default-enabled plugin list
- https://kubernetes.io/docs/concepts/security/pod-security-admission/
- https://kubernetes.io/docs/tasks/configure-pod-container/migrate-from-psp/
- https://kubernetes.io/docs/concepts/security/service-accounts/
- https://kubernetes.io/docs/concepts/architecture/cgroups/ — cgroup v2 stable v1.25; cgroup v1 "Kubernetes v1.35 [deprecated]"; failCgroupV1
- https://kubernetes.io/docs/setup/production-environment/container-runtimes/ — containerd / CRI-O / Docker Engine via cri-dockerd / Mirantis; cgroup driver; KubeletCgroupDriverFromCRI at v1.37
- https://kubernetes.io/docs/tasks/debug/debug-cluster/crictl/
- https://kubernetes.io/docs/tasks/administer-cluster/verify-signed-artifacts/ — "Kubernetes v1.26 [beta]", cosign verify examples against v1.37.0
- https://kubernetes.io/dockershim/ — dockershim removal FAQ; v1.24, May 2022; OCI image compatibility
- https://kubernetes.io/blog/2020/12/02/dont-panic-kubernetes-and-docker/ — deprecation announcement, original v1.22 removal target
- https://kubernetes.io/blog/2022/12/12/kubernetes-release-artifact-signing/ — image signing alpha in v1.24/v1.25, beta + binaries/SBOM/provenance in v1.26
- https://kubernetes.io/blog/2021/04/06/podsecuritypolicy-deprecation-past-present-and-future/ (referenced from CHANGELOG-1.21 and CHANGELOG-1.25)
Other
- https://github.com/rkt/rkt — repository archived, description "[Project ended]", last release v1.30.0 (2018-04-16), last push 2020-02-24 (via gh api repos/rkt/rkt)
- https://github.com/kubernetes/community/blob/master/contributors/devel/sig-node/container-runtime-interface.md — CRI design doc referenced by CHANGELOG-1.5
Unverified items, restated¶
- ImagePolicyWebhook's current stability level. Confirmed present and documented at v1.37 via the admission-controllers page; the fetched excerpt truncated before the feature-state banner, and no KEP exists for it under
keps/sig-authorkeps/sig-node. - Kubernetes' achieved SLSA build level at the pin. KEP-3027 is
status: provisionaland enumerates Levels 1–4 as goals without recording an achieved level. Provenance files are produced and cosign-signed (verified). A web search returned "SLSA Level 1 from v1.23" but only as summary prose, not a quotable primary statement. - Whether the
MutatingAdmissionPolicyadmission plugin is in the default-enabled set at v1.37. GA in v1.36 with "enabled by default" per CHANGELOG-1.36 (#136039), but the v1.37 default-plugin list quoted from the admission-controllers page namesValidatingAdmissionPolicyand both webhook plugins without namingMutatingAdmissionPolicy. Do not assert its presence in that list. - Runtime on-disk paths (
/var/lib/containerd,/var/lib/containers). Not asserted in kubernetes.io primary docs; these belong to the runtime projects, outside this scope's allowed sources. DynamicAuditing/AuditSinkend state. Confirmed added as alpha in v1.13 (#67257, #67547); I did not verify its removal release and so have omitted it from the table rather than guess.- User namespaces beta history.
kep.yamlrecords beta at v1.35 and stable at v1.36 (the latter corroborated by CHANGELOG-1.36 #136792). I searched CHANGELOG-1.30 forUserNamespacesand found no beta-graduation entry, so the single beta milestone inkep.yamlis what is cited; if an exercise needs the gate's intermediate default-off beta iterations, re-verify against the feature-gates reference.
Part 3 — Networking, storage, cluster lifecycle, observability¶
Pin, releases and support policy¶
- https://kubernetes.io/releases/version-skew-policy/
- https://kubernetes.io/releases/patch-releases/
- https://git.k8s.io/enhancements/keps/sig-release/1498-kubernetes-yearly-support-period/README.md
Kubernetes CHANGELOGs (raw, kubernetes/kubernetes master)¶
- https://raw.githubusercontent.com/kubernetes/kubernetes/master/CHANGELOG/CHANGELOG-1.4.md
- https://raw.githubusercontent.com/kubernetes/kubernetes/master/CHANGELOG/CHANGELOG-1.5.md
- https://raw.githubusercontent.com/kubernetes/kubernetes/master/CHANGELOG/CHANGELOG-1.6.md
- https://raw.githubusercontent.com/kubernetes/kubernetes/master/CHANGELOG/CHANGELOG-1.7.md
- https://raw.githubusercontent.com/kubernetes/kubernetes/master/CHANGELOG/CHANGELOG-1.8.md
- https://raw.githubusercontent.com/kubernetes/kubernetes/master/CHANGELOG/CHANGELOG-1.9.md
- https://raw.githubusercontent.com/kubernetes/kubernetes/master/CHANGELOG/CHANGELOG-1.10.md
- https://raw.githubusercontent.com/kubernetes/kubernetes/master/CHANGELOG/CHANGELOG-1.11.md
- https://raw.githubusercontent.com/kubernetes/kubernetes/master/CHANGELOG/CHANGELOG-1.12.md
- https://raw.githubusercontent.com/kubernetes/kubernetes/master/CHANGELOG/CHANGELOG-1.13.md
- https://raw.githubusercontent.com/kubernetes/kubernetes/master/CHANGELOG/CHANGELOG-1.15.md
- https://raw.githubusercontent.com/kubernetes/kubernetes/master/CHANGELOG/CHANGELOG-1.16.md
- https://raw.githubusercontent.com/kubernetes/kubernetes/master/CHANGELOG/CHANGELOG-1.17.md
- https://raw.githubusercontent.com/kubernetes/kubernetes/master/CHANGELOG/CHANGELOG-1.18.md
- https://raw.githubusercontent.com/kubernetes/kubernetes/master/CHANGELOG/CHANGELOG-1.19.md
- https://raw.githubusercontent.com/kubernetes/kubernetes/master/CHANGELOG/CHANGELOG-1.20.md
- https://raw.githubusercontent.com/kubernetes/kubernetes/master/CHANGELOG/CHANGELOG-1.21.md
- https://raw.githubusercontent.com/kubernetes/kubernetes/master/CHANGELOG/CHANGELOG-1.22.md
- https://raw.githubusercontent.com/kubernetes/kubernetes/master/CHANGELOG/CHANGELOG-1.23.md
- https://raw.githubusercontent.com/kubernetes/kubernetes/master/CHANGELOG/CHANGELOG-1.24.md
- https://raw.githubusercontent.com/kubernetes/kubernetes/master/CHANGELOG/CHANGELOG-1.25.md
- https://raw.githubusercontent.com/kubernetes/kubernetes/master/CHANGELOG/CHANGELOG-1.26.md
- https://raw.githubusercontent.com/kubernetes/kubernetes/master/CHANGELOG/CHANGELOG-1.30.md
- https://raw.githubusercontent.com/kubernetes/kubernetes/master/CHANGELOG/CHANGELOG-1.31.md
Git history / tree state in kubernetes/kubernetes (via gh api)¶
- https://github.com/kubernetes/kubernetes/commit/5f9735de5 —
cluster: delete saltbase(v1.10) - https://github.com/kubernetes/kubernetes/commit/7a75a7b43 — remove AWS from
cluster/(v1.10) - https://github.com/kubernetes/kubernetes/commit/8d44e0b38 — remove Vagrant kube-up (v1.10)
- https://github.com/kubernetes/kubernetes/commit/553a3f049 — remove photon-controller (v1.10)
- https://github.com/kubernetes/kubernetes/commit/29fa89d9d — remove openstack-heat (v1.10)
- https://github.com/kubernetes/kubernetes/commit/e77373199 — remove libvirt-coreos (v1.10)
- https://github.com/kubernetes/kubernetes/commit/4708714dd — remove Azure kube-up (v1.6)
- https://github.com/kubernetes/kubernetes/commit/8762bc39a — remove centos/local (v1.15)
- https://github.com/kubernetes/kubernetes/commit/b08cd0e2c — remove
cluster/juju(v1.19) - https://github.com/kubernetes/kubernetes/commit/ac86803a7 —
Remove docs/ other than auto generated(v1.7) - https://github.com/kubernetes/kubernetes/commit/5da316d44 —
Remove stubs from docs/(v1.7) - https://github.com/kubernetes/kubernetes/commit/852f1e23e —
kubeadm: remove the v1beta3 API and PublicKeysECDSA feature gate(v1.37) - https://github.com/kubernetes/kubernetes/commit/cecb92bbd —
kubeadm: introduce placeholder v1 API(v1.37) - https://github.com/kubernetes/kubernetes/commit/57d563403 — remove
kubeadm alpha selfhosting pivot(v1.21) - https://github.com/kubernetes/kubernetes/commit/31fb04fa9 —
Remove cluster-monitoringaddon (v1.18) - https://github.com/kubernetes/kubernetes/commit/f5f989382 —
Restore cAdvisor prometheus metrics to the main port→/metrics/cadvisor(v1.8) - https://github.com/kubernetes/kubernetes/commit/0ad054896 —
KEP-5207: Promote metrics.k8s.io to GA(v1.37) - https://github.com/kubernetes/kubernetes/blob/master/cluster/README.md —
cluster/deprecation notice - https://github.com/kubernetes/kubernetes/tree/master/cmd/kubeadm/app/apis/kubeadm — API versions present at the pin (
v1,v1beta4) - https://github.com/kubernetes/kubernetes/blob/master/cmd/kubeadm/app/apis/kubeadm/v1/doc.go — v1 placeholder note + kubeadm config migration ladder
- https://github.com/kubernetes/kubernetes/blob/v1.37.0/cmd/kubeadm/app/constants/constants.go —
DefaultEtcdVersion = "3.7.0-0",MinExternalEtcdVersion = "3.5.24-0",SupportedEtcdVersion - https://github.com/kubernetes/kubernetes/tree/master/staging/src/k8s.io/kubelet/config —
v1alpha1/v1beta1/v1(v1 = CredentialProviderConfig) - https://github.com/kubernetes/kubernetes/tree/master/staging/src/k8s.io/kube-scheduler/config —
v1only - https://github.com/kubernetes/kubernetes/tree/master/staging/src/k8s.io/kube-proxy/config —
v1alpha1only - https://github.com/kubernetes/kubernetes/tree/master/staging/src/k8s.io/metrics/pkg/apis/metrics —
v1alpha1/v1beta1/v1 - https://github.com/kubernetes/kubernetes/tree/master/staging/src/k8s.io/metrics/pkg/apis/custom_metrics —
v1beta1/v1beta2 - https://github.com/kubernetes/kubernetes/tree/master/staging/src/k8s.io/metrics/pkg/apis/external_metrics —
v1beta1 - https://github.com/kubernetes/kubernetes/tree/master/staging/src/k8s.io/api/autoscaling —
v1/v2only - https://github.com/kubernetes/kubernetes/tree/master/cluster/addons — addon set at the pin
KEPs¶
- https://github.com/kubernetes/enhancements/blob/master/keps/sig-instrumentation/5207-metrics-k8s-io-api-definition/kep.yaml
- https://github.com/kubernetes/enhancements/blob/master/keps/sig-cloud-provider/2395-removing-in-tree-cloud-providers/kep.yaml
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-cluster-lifecycle/kubeadm
kubernetes.io docs and blog¶
- https://kubernetes.io/docs/reference/using-api/deprecation-guide/
- https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-init/
- https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/
- https://kubernetes.io/blog/2016/09/how-we-made-kubernetes-easy-to-install/
- https://kubernetes.io/blog/2018/12/03/kubernetes-1-13-release-announcement/
- https://kubernetes.io/blog/2018/12/04/production-ready-kubernetes-cluster-creation-with-kubeadm/
- https://kubernetes.io/blog/2019/06/24/automated-high-availability-in-kubeadm-v1.15-batteries-included-but-swappable/
Heapster / metrics-server / kube-state-metrics / etcd (independently versioned)¶
- https://github.com/kubernetes-retired/heapster — archived repo metadata
- https://raw.githubusercontent.com/kubernetes-retired/heapster/master/README.md — "RETIRED"
- https://raw.githubusercontent.com/kubernetes-retired/heapster/master/docs/deprecation.md — 1.11/1.12/1.13 timeline
- https://github.com/kubernetes-sigs/metrics-server/releases — release list and dates (v0.9.0 latest at the pin)
- https://github.com/kubernetes-sigs/metrics-server/releases/tag/v0.6.0 —
nodes/metricsRBAC change, migration to the kubelet resource metrics endpoint - https://github.com/kubernetes/kube-state-metrics/releases — v2.20.0 at the pin
- https://github.com/etcd-io/etcd/releases — v3.6.0 (2025-05-15), v3.7.0 (2026-07-08)
- https://raw.githubusercontent.com/etcd-io/etcd/main/CHANGELOG/CHANGELOG-3.6.md —
--enable-v2/--experimental-enable-v2v3removed
Networking — KEPs (kubernetes/enhancements)¶
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/3866-nftables-proxy — alpha v1.29, beta v1.31, stable v1.33
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/5343-nftables-to-default — alpha v1.37, beta 1.39, GA 1.40 (
status: provisional) - https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/5495-deprecate-ipvs-mode-in-kube-proxy
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/6032-nftables-localhost-nodeport-userspace-proxy
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/265-ipvs-based-load-balancing
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/0752-endpointslices
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/4974-deprecate-endpoints
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/2433-topology-aware-hints
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/536-topology-aware-routing
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/4444-service-traffic-distribution
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/563-dual-stack
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/1669-proxy-terminating-endpoints
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/5707-deprecate-service-externalips
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/427-coredns
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/566-coredns-default
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/1024-nodelocal-cache-dns
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/504-configurable-pod-dns
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/2595-expanded-dns-config
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/4427-relaxed-dns-search-validation
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/1453-ingress-api
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/758-ingress-api-group
Networking — CHANGELOGs¶
All at https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-<v>.md, for <v> in:
1.2 (iptables default), 1.5 (--configure-cbr0 removed), 1.7 (NetworkPolicy GA, externalTrafficPolicy field), 1.11 (CoreDNS GA + kubeadm default, IPVS GA), 1.13 (CoreDNS kube-up default), 1.14 (Ingress → networking.k8s.io/v1beta1), 1.16 (dual-stack alpha), 1.18 (IngressClass, pathType, appProtocol alpha), 1.19 (Ingress v1 GA + field renames, EndpointSlice default for kube-proxy, appProtocol beta), 1.20 (appProtocol GA, dual-stack redesign alpha), 1.21 (kube-dns removed from kubeadm, dual-stack beta), 1.22 (Ingress beta removal, topologyKeys removed, EndpointSliceProxying GA, internalTrafficPolicy beta), 1.23 (dual-stack GA), 1.24 (dockershim + kubelet network flags removed), 1.25 (userspace deprecated), 1.26 (userspace removed, internalTrafficPolicy GA), 1.27 (topology-mode annotation), 1.29 (nftables alpha), 1.30 (trafficDistribution alpha), 1.31 (nftables beta, trafficDistribution beta), 1.33 (nftables GA, Endpoints deprecated, hints GA), 1.35 (ipvs deprecated), 1.37 (proxy-mode warning, KubeProxyIPVS gate, NFTablesNetlink, localhost NodePort userspace proxy)
Networking — kubernetes.io docs¶
- https://kubernetes.io/docs/reference/networking/virtual-ips/ — "In Kubernetes 1.37, this is
iptables, but a future version of Kubernetes will change the default tonftables" - https://kubernetes.io/docs/concepts/services-networking/ingress/ — "The Ingress API has been frozen"
- https://github.com/kubernetes/website/blob/main/content/en/docs/reference/using-api/deprecation-guide.md — v1.22 Ingress/IngressClass removal, v1.16 NetworkPolicy removal
- https://kubernetes.io/docs/concepts/cluster-administration/network-plugins/ — CNI required; kubelet CNI params removed in 1.24
- https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/ — search list and
ndots:5
Networking — kubernetes-sigs/gateway-api (independently versioned)¶
gh api repos/kubernetes-sigs/gateway-api/releases --paginate— full tag/date list; latest stable at the pin = v1.6.1 (2026-07-16)- https://github.com/kubernetes-sigs/gateway-api/releases/tag/v1.0.0 (and v1.1.0 / v1.2.0 / v1.3.0 / v1.4.0 / v1.5.0 / v1.6.0)
- https://github.com/kubernetes-sigs/gateway-api/blob/main/site/content/en/docs/introduction/_index.md — role-oriented / portable / expressive / extensible design goals, personas
- https://github.com/kubernetes-sigs/gateway-api/blob/main/site/content/en/docs/faq/_index.md — "Will Gateway API replace the Ingress API? No."
Networking — source-tree verification (used where CHANGELOGs were silent)¶
pkg/api/types.go@v1.0.0— ClusterIP/NodePort/LoadBalancer andclusterIP: Noneall present in 2015;ServiceTypeExternalNameabsent atv1.2.0/v1.3.0, present atv1.4.0pkg/kubelet/network/@v1.1.0(nocni, nokubenet) vsv1.2.0(both present) — CNI arrived in the kubelet in v1.2pkg/kubelet/dockershim/network/present @v1.23.0, 404 @v1.24.0— kubenet removed with dockershim
Storage — KEPs (kubernetes/enhancements)¶
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/625-csi-migration
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/1487-csi-migration-aws
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/1488-csi-migration-gce-pd
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/1489-csi-migration-cinder
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/1490-csi-migration-azuredisk
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/1491-csi-migration-vsphere
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/1885-csi-migration-azurefile
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/2589-csi-migration-portworx
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/2923-csi-migration-ceph-rbd
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/2924-csi-migration-cephfs — withdrawn
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/177-volume-snapshot
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/284-enable-volume-expansion
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/1790-recover-resize-failure
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/1698-generic-ephemeral-volumes
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/596-csi-inline-volumes
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/2485-read-write-once-pod-pv-access-mode
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/3333-retroactive-default-storage-class
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/3751-volume-attributes-class
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/5040-remove-gitrepo-driver — CVE-2024-10220
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/3476-volume-group-snapshot
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/1472-storage-capacity-tracking
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/2644-honor-pv-reclaim-policy
- https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/1967-size-memory-backed-volumes
Storage — CHANGELOGs¶
https://raw.githubusercontent.com/kubernetes/kubernetes/master/CHANGELOG/CHANGELOG-1.X.md for X in 2, 4, 6, 9, 10, 12, 13, 15, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37
Storage — source tree (definitive for removals), via gh api repos/kubernetes/kubernetes/contents/...?ref=<branch|tag>¶
pkg/volume/listings @ tagsv1.1.1,v1.2.0,v1.6.0and branchesrelease-1.20…release-1.37pkg/volume/git_repo/git_repo.go@release-1.37— the "plugin has been disabled" error textpkg/features/kube_features.go@release-1.37—GitRepoVolumeDriverLockToDefault: true, Default: falsecmd/kubelet/app/plugins.go@release-1.37— plugins still registered (incl.flexvolume,git_repo)staging/src/k8s.io/csi-translation-lib/plugins/@release-1.37— which legacy specs still get translatedstaging/src/k8s.io/api/core/v1/types.go@release-1.37— dead volume fields still declaredstaging/src/k8s.io/api/storage/v1/types.go@release-1.9/1.10/1.11—volumeBindingMode,allowVolumeExpansionpkg/api/v1/types.go@v1.0.0,v1.1.1— PV/PVC present in 2015pkg/apis/storage/v1beta1/types.go@release-1.4;pkg/apis/storage/util/helpers.go@release-1.6/1.8/1.10plugin/pkg/admission/storageclass/default/@release-1.4—DefaultStorageClassadmission plugin- commit history for
pkg/volume/{awsebs,gcepd,azure_file,vsphere_volume}— incl. the reverted #118236 - https://github.com/kubernetes/kubernetes/releases/tag/v1.37.0 — published 2026-08-26
Storage — design proposals and docs¶
- https://github.com/kubernetes/design-proposals-archive/blob/main/storage/container-storage-interface.md — the "linked, compiled, built, and shipped with the core kubernetes binaries" rationale
- https://github.com/kubernetes/design-proposals-archive/blob/main/storage/volume-topology-scheduling.md — the delayed-binding rationale
- https://kubernetes.io/blog/2019/01/15/container-storage-interface-ga/
- https://github.com/kubernetes/website/blob/main/content/en/docs/concepts/storage/volumes.md
- https://github.com/kubernetes-csi/external-snapshotter — releases (v8.6.0 at the pin) and
client/config/crd/*.yamlserved versions