Every plan this post makes for apiextensions.k8s.io/v1 was carried out, and the pin now states its three rules as four with one banned keyword traded for another, names the condition reporting violations once in the tree, and misnames the extension that replaces the opt-out¶
Post — Future of CRDs: Structural Schemas,
20 June 2019, by Stefan Schimanski (Red Hat). 245 lines, 10,704 bytes. Frontmatter carries
slug: crd-structural-schema, which is why the URL says nothing about the future of anything; 23 of
this year's 52 posts set a slug that way. No k8s version, but the post dates itself repeatedly:
Kubernetes 1.15 at :39, :222 and :237, and 1.16 at :43.
This is the post that decides what a CustomResourceDefinition is allowed to be, and it is the only one in the year whose worked example is an attack. It also has the year's worst code fences: eight YAML blocks, of which four cannot be parsed at all and two more parse into values that are not the values shown. Both facts matter, because a reader who tries to follow this post by pasting from it gets errors before they get to the argument, and the argument is sound.
As written — an attack, a gap, a definition, an algorithm, three extensions and a plan.
:10 sets up the history: CRDs arrived "roughly two years ago" storing arbitrary JSON, with only
kind, apiVersion and metadata required to follow the API conventions, and Kubernetes 1.8 gave
them "an optional OpenAPI v3 based validation schema". :12 names the gap that leaves. Because an
OpenAPI specification describes "only what must be there, not what shouldn't", and may itself be
incomplete, "the Kubernetes API server never knew the complete structure of CustomResource
instances", so kube-apiserver "stores all JSON data received in an API request (if it validates
against the OpenAPI spec). This especially includes anything that is not specified in the OpenAPI
schema."
:14-33, under The story of malicious, unspecified data, turns that into the exercise's subject.
A CRD for nightly maintenance jobs run as a service user; a submitted object at :18-27 whose
spec.shell appends a line to /etc/passwd and which also carries privileged: true. :29: the
field "is not specified by the operations team. Their controller does not know it, and their
validating admission webhook does not know about it either. Nevertheless, kube-apiserver persists
this suspicious, but unknown field without ever validating it." :31 says the job is harmless for
now, because the service user cannot write /etc/passwd. :33 is the payload of the whole post:
the team later adds privileged support, carefully authorised so that only a few people may create
privileged jobs — "That malicious job though has long been persisted to etcd. The next night
arrives and the malicious job is executed." The vulnerability is a field that was stored before it
meant anything and became meaningful afterwards.
:35-45, under Towards complete knowledge of the data structure, is the fix and the plan. :37:
"we cannot trust CustomResource data in etcd. Without having complete knowledge about the JSON
structure, the kube-apsierver cannot do anything to prevent persistence of unknown data" — with
kube-apsierver misspelled. :39 announces that "Kubernetes 1.15 introduces the concept of a
(complete) structural OpenAPI schema". :41 says non-structural schemas are reported "in a
NonStructural condition in the CRD". :43 is the plan, and it is exact: "A structural schema for
CRDs in apiextensions.k8s.io/v1beta1 will not be required. But we plan to require structural
schemas for every CRD created in apiextensions.k8s.io/v1, targeted for 1.16."
:47-120, under Structural Schema, is the definition. :49-57 gives the core as seven constructs
— properties, items, additionalProperties, type, nullable, title, descriptions — the
last written in the plural here and in the singular at :116, where the same list is restated.
:59 adds that "all types must be non-empty, and in each sub-schema only one of properties,
additionalProperties or items may be used." :63-77 is a structural example; :79 says why;
:81 notes that "we leave out apiVersion, kind and metadata. These are implicitly defined for
each object."
:83 then allows the core to be "enhanced for value validation purposes with nearly all other
OpenAPI constructs", shows one at :85-106, and constrains it at :110-111: "the last 5 of the
core constructs are not allowed: additionalProperties, type, nullable, title,
description", and "every properties field mentioned, must also show up in the core (without the
blue value validations)" — a reference to a colour that nothing in the rendered post applies. :113
allows oneOf, allOf, anyOf and not. :115-120 restates the whole definition as three
numbered rules, the third carrying two sub-clauses, and :122-159 makes the same example
non-structural on purpose and keys its three violations to those rules: "rule 2", "rule 3-i", "rule
3-ii".
:161 is the turn: having shown that a structural schema cannot forbid a field, "the good news is
that we don't have to explicitly attempt to forbid unwanted fields in advance." :163-196, under
Pruning, says how. :165 promises pruning by default in apiextensions.k8s.io/v1 "with ways to
opt-out of it", and shows the beta opt-in at :167-173. :175 requires a structural schema for
pruning to be enabled at all. :177-183 gives the algorithm's three assumptions and its three
trigger points: API request data, after conversion and admission, and when reading from etcd.
:185-196 re-shows the malicious object with privileged pruned.
:198-218, under Extensions, adds three vendor extensions for the shapes a structural schema
cannot otherwise express — intstr.IntOrString, runtime.RawExtension and pure JSON (:200).
x-kubernetes-embedded-resource: true at :204, x-kubernetes-int-or-string: true at :206-214
with a permitted oneOf pattern, and x-kubernetes-preserve-unknown-fields: true at :216-218,
which "can be combined with x-kubernetes-embedded-resource" and which, at the root, restores "the
traditional CRD behaviour that nothing is pruned".
:220-244 is the conclusion. Three bullets at :224-226: structural schemas optional in v1beta1,
pruning requires a structural schema, violations signalled by the NonStructural condition. :228
and :235 note that a two-line schema of type: object plus
x-kubernetes-preserve-unknown-fields: true is itself structural and "will lead to the old
schema-less behaviour". :237-242 lists four features that from 1.15 onward will require a
structural schema, each with its state: OpenAPI publishing and kubectl explain (beta), CRD
conversion (beta), CRD defaulting (alpha), and server-side apply (alpha, "CRD support pending").
:244 points at the 1.15 documentation.
As it runs now — the plan was carried out in full. Every promise at :43, :165 and :224-226
is now the documented behaviour of apiextensions.k8s.io/v1, and the post's own subject was never
gated: zero of the pin's 488 feature-gate files mention structural schemas or pruning. What did
not survive is the naming. Three of the identifiers this post uses are wrong, and the page that
replaced it has an invented one of its own.
The post's YAML mostly does not parse. There are eight fenced blocks: :18-27, :63-77,
:85-106, :124-153, :167-173, :187-196, :208-212 and :230-233. Fed to a YAML parser, four
of them fail. :63-77, :85-106 and :124-153 all fail the same way, on properties written
without its colon at :68, :90 and :128 — the parser reports while scanning a simple key.
:167-173 fails on the bare ellipsis at :171 with mapping values are not allowed here. The four
that do parse include both versions of the malicious object, :18-27 and :187-196 — but twelve
lines across the post carry typographic quotes instead of ASCII ones (:24, :25, :101,
:103-105, :139, :144, :148, :152, :193, :194), and inside a plain YAML scalar a curly
quote is an ordinary character. Those blocks parse into values that are not the values the post is
showing. The first Do step is to watch this happen rather than take my word for it.
Three rules became four, and one banned keyword was traded for another. The post's definition at
:115-120 is three rules. The pin states it at
tasks/extend-kubernetes/custom-resources/custom-resource-definitions.md:209-220 as four. Rule 1 is
the post's non-empty-type requirement, now with two written exceptions — nodes carrying
x-kubernetes-int-or-string: true or x-kubernetes-preserve-unknown-fields: true. Rule 2 is the
post's 3-ii. Rule 3 is the post's 3-i, and its list has changed: the pin forbids description,
type, default, additionalProperties and nullable inside the junctors, where the post forbids
additionalProperties, type, nullable, title and description. title came out and
default went in — which follows, since default was not yet a thing a CRD schema could contain
when the post was written. Rule 4 has no counterpart at all: "if metadata is specified, then only
restrictions on metadata.name and metadata.generateName are allowed". The post's advice at :81
was to leave metadata out; the pin now says what happens if you do not.
spec.preserveUnknownProperties never existed. The post names the pruning opt-out three times.
At :172, inside YAML, it is preserveUnknownFields: false, which is right. At :218 and :225,
in prose, it is spec.preserveUnknownProperties: false, which is not a field the API has ever had.
Searched across the whole pinned checkout, preserveUnknownProperties occurs on exactly those two
lines of this post and nowhere else — not in the API reference, not in the CRD task page, not in any
other blog post in eleven years of archive.
And apiextensions/v1beta1 is not an API group. The opt-in YAML at :168 reads apiVersion:
apiextensions/v1beta1, missing the .k8s.io. It is one of the four blocks that does not parse
anyway, so the error a reader hits is the ellipsis on the next line, not this.
The int-or-string pattern the post permits is not one the pin accepts. :206-214 shows
x-kubernetes-int-or-string: true beside a oneOf of type: integer and type: string, and calls
the oneOf "permitted, though optional". The pin gives two patterns for that extension, at
custom-resource-definitions.md:483-487 and :493-499, and neither is the post's: one is a bare
anyOf of type: integer and type: string, the other an allOf wrapping that same anyOf. The
page says at :479-480 that a schema must use "exactly those, without variations in order to
additional fields" — a sentence with its own missing word. The post's oneOf is a variation.
Whether the API server actually refuses it is Do step 9.
The condition that reports violations is named once in the entire documentation tree.
NonStructural appears at custom-resource-definitions.md:325 and on no other line of
content/en/docs. It is not in the API reference's own list:
reference/kubernetes-api/apiextensions/custom-resource-definition-v1.md:246 says "Types include
Established, NamesAccepted and Terminating". There is a reason to suspect that is not an oversight.
If a structural schema is mandatory at v1, a non-structural v1 CRD should be rejected at admission
rather than accepted and flagged, which would make the condition unreachable for anything created at
v1. Do step 5 tests that, and the answer decides whether :41 and :226 are still true or have
quietly become v1beta1-only.
The reference names a replacement extension that does not exist either.
custom-resource-definition-v1.md:84-85 still carries preserveUnknownFields as a v1 field and
describes it as "deprecated in favor of setting x-preserve-unknown-fields to true in
spec.versions[*].schema.openAPIV3Schema". The extension is x-kubernetes-preserve-unknown-fields.
x-preserve-unknown-fields occurs exactly once in the pinned tree — on that line. So the post gets
the opt-out's name wrong in prose, and the page that outlived it gets the opt-out's replacement
wrong in prose, in the same field's documentation.
One of the four dependent features is still documented behind a gate that has been removed.
:237-242 lists OpenAPI publishing, conversion, defaulting and server-side apply as the features
that will require a structural schema, with defaulting alpha in 1.15.
custom-resource-definition-v1.md:472-474 still says "Defaulting is a beta feature under the
CustomResourceDefaulting feature gate. Defaulting requires spec.preserveUnknownFields to be false."
CustomResourceDefaulting went beta in v1.16, stable in v1.17, and its gate file is marked removed.
Half that sentence is nine releases stale; the other half is the post's argument, still correct.
What is still exactly right is the part that reads like a story. :29-33 — a field nobody
specified, persisted unvalidated, harmless until the day the team implements it — is the reason the
whole mechanism exists, and the pin makes the same case in custom-resource-definitions.md:334-348:
CRDs converted up from v1beta1 may lack a structural schema and carry spec.preserveUnknownFields
true (:334-335), and for those legacy objects :341-342 states two consequences as bullets:
"Pruning is not enabled." and "You can store arbitrary data." The algorithm's three trigger points
at :177-183 are still the three; the pin states the effect at :396 as "By default, all
unspecified fields for a custom resource, across all versions, are pruned", and the sub-tree opt-out
at :397-398 is the post's x-kubernetes-preserve-unknown-fields: true.
The page's own pasted output is older than the post. The worked pruning example at
custom-resource-definitions.md:350-392 shows a created object whose creationTimestamp at :375
is 2017-05-31T12:56:35Z and whose uid at :380 is from the same era — output pasted at least two
years before this post was published and carried unchanged to a 2026 pin. The command above it, at
:366, is kubectl create --validate=false -f my-crontab.yaml -o yaml;
reference/kubectl/generated/kubectl_create/_index.md:156-159 gives --validate a default of
strict since well after that output was captured, so a reader who drops the flag gets a
client-side rejection instead of a pruned object. :509 has a doubled slash in a link path. The
prose on that page has been maintained; the terminal blocks in it have not.
What this exercise does not cover, and where it lives. OpenAPI publishing, kubectl explain
against a custom resource, and the CustomResourcePublishOpenAPI ladder belong to the OpenAPI
exercise, which also creates a CRD at
apiextensions.k8s.io/v1 and points forward to this page's structural-schema section. Server-side
apply, ServerSideFieldValidation, --dry-run and what --validate does on the client are the
dry-run and diff exercise; this file uses
--validate only where the post's own YAML is the thing being validated. The pin's advice not to
use admission webhooks to validate CustomResource specs — the advice this post's attack story is the
argument for — is the extensible admission exercise.
Conversion webhooks are read here only as a ladder row; nothing in this exercise runs one.
The diff, and why — four of the seven cases, and the dominant one is the rare one. What happened
to this post is that it was retired by being agreed with. :43 plans mandatory structural
schemas for apiextensions.k8s.io/v1 "targeted for 1.16"; v1 arrived in 1.16 and
custom-resource-definitions.md:205-207 now states the requirement in the past tense. :165 plans
pruning by default at v1 with an opt-out; :396 states it as the default and :397-398 gives the
opt-out. :224's "not required in v1beta1" was made moot from a different direction:
reference/using-api/deprecation-guide.md:189 records that apiextensions.k8s.io/v1beta1 has not
been served since v1.22, and :203 adds that spec.preserveUnknownFields: true is disallowed when
creating a v1 object at all. There is nothing left of the escape route the post was careful to
leave open, because the version it led out of was withdrawn.
Still right is the argument. The attack at :29-33 is not a 2019 problem that got fixed; it is
the reason the fix is shaped the way it is, and the pin makes the identical case for legacy objects
at :334-348. Everything in Pruning that is not a version number still describes what the API
server does.
The post broke in its code. Four of eight YAML blocks will not parse, two more parse into
mangled values, and the int-or-string pattern it says is permitted is a variation on the two the pin
says are the only ones allowed. None of that was true when it was written — the missing colons and
curly quotes are damage from the rendering pipeline the post passed through, and the oneOf was
presumably right in 1.15. The reader still hits all of it.
Wrong when it was published is the naming. spec.preserveUnknownProperties at :218 and :225
was never a field; the correct preserveUnknownFields appears once, at :172, inside a block that
does not parse, so the only place the post gets the name right is the one place a reader cannot run.
apiVersion: apiextensions/v1beta1 at :168 was never an API group. Both are prose slips rather
than design errors, which is what makes them worth an exercise: the mechanism this post describes is
intact seven years on, and the identifiers it hands you are not.
The ladder — the post's own subject has no gate. Searched across all 488 feature-gate files at
the pin, none mentions structural schemas or pruning: the requirement arrived attached to an API
version, not to a switch, which is why there is no --feature-gates line anywhere in this exercise
and why the only way to observe the change is to try both API versions. Two of the four dependent
features at :237-242 do have ladders that nothing else in this archive transcribes.
CustomResourceWebhookConversion
| stage | default | locked | releases |
|---|---|---|---|
| alpha | false |
— | v1.13 – v1.14 |
| beta | true |
— | v1.15 – v1.15 |
| stable | true |
— | v1.16 – v1.18 |
Body: "Enable webhook-based conversion on resources created from CustomResourceDefinition." Beta in
v1.15 is exactly what :239 claims for it. Note the length of that beta: one release. It went
stable in v1.16, the same release that made structural schemas mandatory, which is the post's
sentence about conversion requiring a structural schema arriving as a single change rather than two.
CustomResourceDefaulting
| stage | default | locked | releases |
|---|---|---|---|
| alpha | false |
— | v1.15 – v1.15 |
| beta | true |
— | v1.16 – v1.16 |
| stable | true |
— | v1.17 – v1.18 |
Body: "Enable CRD support for default values in OpenAPI v3 validation schemas." Alpha in v1.15 is
what :241 claims. One release at alpha, one at beta, stable by v1.17 — and this is the gate that
custom-resource-definition-v1.md:473-474 still describes as governing "a beta feature", nine
releases after it stopped doing so and several after it was deleted.
Both files are marked removed: true and carry _build: {list: never, render: false}, so neither
renders on the website; both were read from the pinned source. Neither sets lockToDefault, and
neither repeats a stage, so both climb once and stop — and in both cases stopping meant the gate
file was retained as a tombstone rather than the feature regressing. What the tables cannot show is
that removal of the gate did not remove the requirement: unlike the pid limits in the PID-limiting
exercise, where the gates went and the settings stayed off, here the gate went
and the behaviour became unconditional.
Topology — solo: one node at 10.10.10.180, 4096MB, 4
CPUs, 25G. Everything this exercise touches is decided inside kube-apiserver before anything is
persisted, so a second node would observe nothing a first does not. No workload runs: the custom
resources created here have no controller, which is the point — the post's malicious object is
dangerous precisely because it sits in etcd waiting for one. What the topology cannot show is the
half of the story that needs time: a field persisted under one schema and executed months later
under another. The nearest this can get is Do step 3 and step 4, which persist an object and then
read it back, and the gap between them is a shell prompt rather than a night.
Do
- Start by finding out that you cannot follow this post by pasting from it.
:63-77is the post's own structural example; it is reproduced here character for character.:85-106and:124-153fail the same way, on the same missing colon, and:167-173fails differently:
cat > /tmp/post-63.yaml <<'EOF'
type: object
properties:
spec:
type: object
properties
command:
type: string
shell:
type: string
machines:
type: array
items:
type: string
EOF
kubectl create --dry-run=client -f /tmp/post-63.yaml 2>&1 | head -4
cat > /tmp/post-167.yaml <<'EOF'
apiVersion: apiextensions/v1beta1
kind: CustomResourceDefinition
spec:
…
preserveUnknownFields: false
EOF
kubectl create --dry-run=client -f /tmp/post-167.yaml 2>&1 | head -4
- Now build the CRD the post is describing, correctly. This is
:63-77with the colon restored and nothing else changed, wrapped in theapiextensions.k8s.io/v1envelope the post was planning for. Note what is absent from the schema:privileged, exactly as in the post's story — the operations team has not implemented it yet:
# /tmp/mnj-crd.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: maintenancenightlyjobs.blogwalk.example
spec:
group: blogwalk.example
scope: Namespaced
names:
plural: maintenancenightlyjobs
singular: maintenancenightlyjob
kind: MaintenanceNightlyJob
shortNames: ["mnj"]
versions:
- name: v1
served: true
storage: true
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
properties:
command:
type: string
shell:
type: string
machines:
type: array
items:
type: string
kubectl create -f /tmp/mnj-crd.yaml
kubectl get crd maintenancenightlyjobs.blogwalk.example \
-o jsonpath='{.spec.preserveUnknownFields}{"\n"}'
kubectl get crd maintenancenightlyjobs.blogwalk.example \
-o json | python3 -c 'import sys,json; print([c["type"] for c in json.load(sys.stdin)["status"]["conditions"]])'
- Submit the post's malicious object, copied from
:18-27with theapiVersionchanged to name the group you just created and nothing else touched. Use the pin's own--validate=falsefromcustom-resource-definitions.md:366so that the client gets out of the way and the API server's behaviour is what you see:
cat > /tmp/mnj-object.yaml <<'EOF'
apiVersion: blogwalk.example/v1
kind: MaintenanceNightlyJob
metadata:
name: nightly
spec:
shell: >
grep backdoor /etc/passwd ||
echo “backdoor:76asdfh76:/bin/bash” >> /etc/passwd || true
machines: [“az1-master1”,”az1-master2”,”az2-master3”]
privileged: true
EOF
kubectl create --validate=false -f /tmp/mnj-object.yaml
kubectl get mnj nightly -o json \
| python3 -c 'import sys,json; print(json.dumps(json.load(sys.stdin)["spec"], indent=2, ensure_ascii=False))'
- Read the same object again through the two things the post's story turns on: whether the field survived, and what the post's own typography did to the fields that did. The payload is a shell command; look at the characters in it:
kubectl get mnj nightly -o json \
| python3 -c 'import sys,json; s=json.load(sys.stdin)["spec"]; print("privileged present:", "privileged" in s); print("machines:", s.get("machines")); print("curly quotes in shell:", sum(s["shell"].count(c) for c in "\u201c\u201d"))'
- Delete that object and submit it again without the flag, so the client behaves the way a reader in
2026 will find it behaving by default.
kubectl's own default for--validateisstrict(reference/kubectl/generated/kubectl_create/_index.md:156-159), which is a different mechanism from pruning and produces a different result:
kubectl delete mnj nightly
kubectl create -f /tmp/mnj-object.yaml 2>&1 | head -4
kubectl create --validate=warn -f /tmp/mnj-object.yaml 2>&1 | head -4
kubectl get mnj nightly -o json \
| python3 -c 'import sys,json; print("privileged present:", "privileged" in json.load(sys.stdin)["spec"])'
- Now the question the post's
:41and:226leave open. Submit the post's deliberately non-structural schema from:124-153, keeping one instance of each of the three violations it names at:157-159and nothing else wrong with it, as a v1 CRD — and find out whether v1 reports it or refuses it:
cat > /tmp/mnj-nonstructural.yaml <<'EOF'
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: badjobs.blogwalk.example
spec:
group: blogwalk.example
scope: Namespaced
names: {plural: badjobs, singular: badjob, kind: BadJob}
versions:
- name: v1
served: true
storage: true
schema:
openAPIV3Schema:
properties:
spec:
type: object
properties:
command:
type: string
minLength: 1
oneOf:
- properties:
command:
type: string
required: ["command"]
not:
properties:
privileged: {}
required: ["spec"]
EOF
kubectl create -f /tmp/mnj-nonstructural.yaml 2>&1 | head -20
kubectl get crd badjobs.blogwalk.example -o json 2>/dev/null \
| python3 -c 'import sys,json; print([(c["type"], c["status"]) for c in json.load(sys.stdin)["status"]["conditions"]])' \
|| echo "not created"
- Try the API version the post is writing about.
:43and:224are careful to say that structural schemas are not required inapiextensions.k8s.io/v1beta1, which was true and is now unreachable (reference/using-api/deprecation-guide.md:189):
sed 's|apiextensions.k8s.io/v1$|apiextensions.k8s.io/v1beta1|' /tmp/mnj-nonstructural.yaml \
| kubectl create -f - 2>&1 | head -4
kubectl api-versions | grep apiextensions
kubectl api-resources --api-group=apiextensions.k8s.io
- Test both names the post gives the opt-out.
spec.preserveUnknownFields: trueis the field that exists and thatdeprecation-guide.md:203says is disallowed at v1;spec.preserveUnknownProperties: falseis the name at:218and:225, which exists nowhere. Send each and compare the two failures:
for FIELD in 'preserveUnknownFields: true' 'preserveUnknownProperties: false'; do
echo "== $FIELD"
printf 'apiVersion: apiextensions.k8s.io/v1\nkind: CustomResourceDefinition\nmetadata:\n name: probes.blogwalk.example\nspec:\n group: blogwalk.example\n scope: Namespaced\n names: {plural: probes, singular: probe, kind: Probe}\n %s\n versions:\n - name: v1\n served: true\n storage: true\n schema:\n openAPIV3Schema:\n type: object\n' "$FIELD" \
| kubectl create --dry-run=server -f - 2>&1 | head -4
done
- Test both names for the pruning escape hatch: the extension the post names correctly at
:216-218, and the onecustom-resource-definition-v1.md:85says has replaced it. Only one of the two occurs anywhere in the pin outside that line:
for EXT in x-kubernetes-preserve-unknown-fields x-preserve-unknown-fields; do
echo "== $EXT"
printf 'apiVersion: apiextensions.k8s.io/v1\nkind: CustomResourceDefinition\nmetadata:\n name: exts.blogwalk.example\nspec:\n group: blogwalk.example\n scope: Namespaced\n names: {plural: exts, singular: ext, kind: Ext}\n versions:\n - name: v1\n served: true\n storage: true\n schema:\n openAPIV3Schema:\n type: object\n %s: true\n' "$EXT" \
| kubectl create --dry-run=server -f - 2>&1 | head -4
done
-
Last, the two patterns.
:206-214says aoneOfoftype: integerandtype: stringbesidex-kubernetes-int-or-string: trueis "permitted, though optional";custom-resource-definitions.md:479-480says only the two forms at:483-487and:493-499are allowed, "exactly those, without variations". Send both. Then add adefault:, which needed a gate when the post was written, and ask the API server whether it still knows that gate's name:for FORM in oneOf anyOf; do echo "== $FORM" printf 'apiVersion: apiextensions.k8s.io/v1\nkind: CustomResourceDefinition\nmetadata:\n name: intstrs.blogwalk.example\nspec:\n group: blogwalk.example\n scope: Namespaced\n names: {plural: intstrs, singular: intstr, kind: IntStr}\n versions:\n - name: v1\n served: true\n storage: true\n schema:\n openAPIV3Schema:\n type: object\n properties:\n port:\n x-kubernetes-int-or-string: true\n %s:\n - type: integer\n - type: string\n' "$FORM" \ | kubectl create --dry-run=server -f - 2>&1 | head -4 done kubectl patch crd maintenancenightlyjobs.blogwalk.example --type=json \ -p '[{"op":"add","path":"/spec/versions/0/schema/openAPIV3Schema/properties/spec/properties/command/default","value":"/bin/true"}]' printf 'apiVersion: blogwalk.example/v1\nkind: MaintenanceNightlyJob\nmetadata:\n name: defaulted\nspec:\n shell: /bin/date\n' \ | kubectl create -f - kubectl get mnj defaulted -o jsonpath='{.spec.command}{"\n"}' kubectl get --raw /metrics | grep kubernetes_feature_enabled | grep -i customresource
Expect
-
Both files fail before
kubectllooks atkindat all: the error is YAML syntax, not a schema complaint./tmp/post-63.yamlfails on the line afterproperties, because without its colonpropertiesand the key below it read as one scalar;/tmp/post-167.yamlfails on the ellipsis. Record the exact wording of each — the two errors are different, and neither points at the line a reader would suspect. -
The CRD is created.
spec.preserveUnknownFieldsprints empty orfalse: at v1 there is nothing to set, because:165's forecast is now the default. The condition list should beNamesAcceptedandEstablished, which is whatcustom-resource-definition-v1.md:246names.NonStructuralis not there, and this schema is structural, so that proves nothing yet — step 6 is the test. -
The object is created, and the
specyou read back hasshellandmachinesand noprivileged. You did not ask for that. This is:187-196, and the post's whole argument, working by default seven years later on a schema that never mentioned the field. -
privileged present: False.machinescomes back as three strings, each still carrying a typographic quote inside the value, andcurly quotes in shell:reports 2. Worth sitting with: the post's payload, exactly as printed, would append a line containing curly quotes to/etc/passwd, which is not a line that does what the post says it does. Pruning removes unknown fields; it has no opinion about bad values in known ones. -
Without the flag,
kubectlrefuses the file client-side and namesprivilegedas unknown; the object is not created. With--validate=warnyou get the same information as a warning and the object is created anyway, pruned. So a reader in 2026 meets two mechanisms where the post describes one, and the difference matters: strict validation tells you the field was dropped, pruning does not. Record both messages. -
Genuinely open, and the most interesting step here. Three outcomes are possible. Rejected at admission with the violations named: then a non-structural schema cannot exist at v1,
NonStructuralis unreachable for anything created at v1, and the fact that the pin names it once in the whole tree is explained. Created with aNonStructuralcondition: then:41and:226are still literally true and the condition is simply undocumented. Created silently: then the requirement atcustom-resource-definitions.md:205-207is not enforced where the page says it is. Record which, and if the error names rule numbers, record whether they are the post's three or the pin's four. -
no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1", andapiextensions.k8s.io/v1as the only line fromapi-versions. The version this post spends four paragraphs being careful about cannot be reached from this cluster. -
Two different failures, from two different places.
preserveUnknownFields: trueis a real field carrying a valuedeprecation-guide.md:203says is disallowed at v1, so expect the rejection to come from the server and to say so.preserveUnknownProperties: falseis not a field at all, so expect strict client-side validation to stop it before the server is involved. The post gives the second name twice, in prose, in its conclusion. -
x-kubernetes-preserve-unknown-fields: truebesidetype: objectshould be accepted: that is exactly the two-line schema:230-233calls valid and structural.x-preserve-unknown-fields: trueshould be rejected as unknown. If it is instead accepted, record that, because it would mean the namecustom-resource-definition-v1.md:85tells you to use is accepted and ignored, which is worse than being rejected. -
anyOfshould be accepted. WhetheroneOfis rejected is open: the pin says only those two forms are allowed and the post saysoneOfis permitted, and one of them is describing something the implementation does not do. Record which. Thedefaultshould come back as/bin/trueon an object that never setcommand, with no feature gate involved anywhere — which is the answer to:241. The metric grep may print nothing; a removed gate has no reason to appear, and finding akubernetes_feature_enabledseries forCustomResourceDefaultingwould be the surprise, not the absence.
Read on — four of these the pin will answer, and one it will not.
-
Put post
:115-120besidecustom-resource-definitions.md:209-220and account for every difference: the two exceptions added to rule 1,titleleaving rule 3's list,defaultjoining it, and rule 4 appearing from nowhere. Then read post:81again and decide whether it is advice or a warning. -
custom-resource-definitions.md:394-465works pruning through four examples. Find the one where pruning restarts inside a preserved sub-tree for each property that is specified, and work out what it means for a schema that specifiesspecbut says nothing about what is inside it. -
custom-resource-definitions.md:507-536onRawExtension.:533-534says pairingx-kubernetes-embedded-resourcewith the preserve extension is optional;:536saysapiVersion,kindandmetadataare implicitly specified and validated. Post:204says those three "are not pruned and are automatically validated". Are those the same claim? -
Search
reference/kubernetes-api/apiextensions/custom-resource-definition-v1.mdforNonStructuraland note what you find. Then read:246and decide whether step 6's outcome explains the absence or contradicts it. -
Unanswerable from the pin: why
preserveUnknownFieldsremains a field on the v1 type atcustom-resource-definition-v1.md:84-85at all, whendeprecation-guide.md:203disallows the only value that ever made it do anything. The reference says the field is deprecated and names its replacement wrongly; it does not say why the field is still there.
Teardown — two CRDs at most were created; the --dry-run=server probes in steps 8, 9 and 10
persisted nothing, so nothing needs undoing for those. Deleting a CRD deletes its objects with it:
kubectl delete crd maintenancenightlyjobs.blogwalk.example --ignore-not-found
kubectl delete crd badjobs.blogwalk.example --ignore-not-found
kubectl get crd | grep blogwalk.example || echo "no blogwalk.example CRDs remain"
rm -f /tmp/post-63.yaml /tmp/post-167.yaml /tmp/mnj-crd.yaml /tmp/mnj-object.yaml /tmp/mnj-nonstructural.yaml
Nothing was changed on the node, no kubelet was restarted and no feature gate was set, which is the shape of an exercise about a requirement that arrived attached to an API version. The one thing worth keeping is your record of step 6.