Research B — The mechanics of schema evolution in serialization formats

Q1 — Tagged vs untagged unions

05Answer: no untagged unions. Protobuf's only union is oneof, and its variant is identified by the field number on the wire, never by which fields are present.

06Protobuf has no construct where the reader infers the variant from the shape of the payload. A oneof is encoded as ordinary fields with ordinary tags; the runtime records which tag arrived last. The evolution rules confirm the union is tag-identified, because moving a pre-existing field into a oneof is unsafe (its tag is already in flight and old readers do not know it participates in a discriminated group):

07

"Moving fields into an existing oneof is not safe." [WF] — https://protobuf.dev/programming-guides/proto3/

08

"Changing a single explicit presence field or extension into a member of a new oneof is safe." [WF] — https://protobuf.dev/programming-guides/proto3/

09

"Changing a oneof which contains only one field to an explicit presence field is safe." [WF] — https://protobuf.dev/programming-guides/proto3/

10The known evolution hazard of oneof is that its "unset" state is indistinguishable from "set to a variant I do not know", because the unknown variant's tag lands in the unknown-field set rather than in the oneof:

11

"If checking the value of a oneof returns None/NOT_SET, it could mean that the oneof has not been set or it has been set to a field in a different version of the oneof." [WF] — https://yokota.blog/2021/08/26/understanding-protobuf-compatibility/

12

"removing a field from a oneof is considered a backward incompatible change. Likewise, adding a field to a oneof is considered a forward incompatible change" [WF] — https://yokota.blog/2021/08/26/understanding-protobuf-compatibility/

13Stated rationale for not having untagged unions: NOT FOUND. Searched protobuf.dev proto3/proto2/editions guides, dos-donts, and the editions design docs. Protobuf never had untagged unions to reject, so no rejection rationale exists. The nearest thing to a rationale is Cap'n Proto's and JTD's, in §1.4 and §1.5.

Q2 — Absent vs empty vs null for collections

14Answer: NO. proto3 cannot distinguish an empty repeated field from an absent one, and this was never fixed — the 3.15 presence restoration covers singular scalars only, not repeated or map.

15

"When serializing, fields with implicit presence are not serialized if they contain their default value." [WF] — https://protobuf.dev/programming-guides/field_presence/

16The JSON mapping makes the collapse explicit and names the empty-list case directly:

17

"If the field doesn't support field presence and has the default value (for example any empty repeated field) serializers should omit it from the output." [WF] — https://protobuf.dev/programming-guides/json/

18The presence document is candid that the collapsed state is genuinely three-ways ambiguous under implicit presence:

19

"The default value may mean: the field was explicitly set to its default value, which is valid in the application-specific domain of values; the field was notionally 'cleared' by setting its default; or the field was never set." [WF] — https://protobuf.dev/programming-guides/field_presence/

20There is a JSON-only escape hatch: JSON has null, which the wire format does not, and the presence doc flags the mismatch:

21

"JSON may include fields that are 'not present,' unlike the implicit presence discipline for other formats: JSON defines a null value, which may be used to represent a defined but not-present field." [WF] — https://protobuf.dev/programming-guides/field_presence/

22But ProtoJSON deliberately throws that information away on parse — null is normalised to "unset":

23

"Parsers accept null as a legal value for any field" where "The field should remain unset, as though it was not present in the input at all." [WF] — https://protobuf.dev/programming-guides/json/

24Was it ever changed? Yes for singular scalars (see §3, Reversal R2). No for repeated/map — those remain no-presence in all of proto2, proto3, and editions. The optional label cannot be applied to a repeated field.

Q3 — Closed vocabularies (enums)

25Answer: proto3 and editions use OPEN enums; proto2 used CLOSED. The switch was made specifically because closed enums misbehave. This is the second-clearest reversal in the whole corpus (see §3, R4).

26

"Open enums will parse the value 2 and store it directly in the field. Accessor will report the field as being set and will return something that represents 2." [WF] — https://protobuf.dev/programming-guides/enum/

27

"Closed enums will parse the value 2 and store it in the message's unknown field set. Accessors will report the field as being unset and will return the enum's default value." [WF] — https://protobuf.dev/programming-guides/enum/

28

"Prior to the introduction of syntax = \"proto3\" all enums were closed." [WF] — https://protobuf.dev/programming-guides/enum/

29The rationale sentence — this is the money quote:

30

"Proto3 and editions use open enums specifically because of the unexpected behavior that closed enums cause." [WF] — https://protobuf.dev/programming-guides/enum/

31The documented incident class (closed enums silently reorder repeated fields):

32

"When a repeated Enum field is parsed, all unknown values will be placed in the unknown field set. When it is serialized those unknown values will be written again, but not in their original place in the list." [WF] — https://protobuf.dev/programming-guides/enum/

33

"A wire format containing the values [0, 2, 1, 2] for field 1 will parse so that the repeated field contains [0, 1] and the value [2, 2] will end up stored as an unknown field. After reserializing the message, the wire format will correspond to [0, 1, 2, 2]." [WF] — https://protobuf.dev/programming-guides/enum/

34

"Maps with closed enums for their value will place entire entries (key and value) in the unknown fields whenever the value is unknown." [WF] — https://protobuf.dev/programming-guides/enum/

35Second documented incident — closed enums interact catastrophically with required:

36

"A second issue with required fields appears when someone adds a value to an enum. In this case, the unrecognized enum value is treated as if it were missing, which also causes the required value check to fail." [WF] — https://protobuf.dev/programming-guides/proto2/

37The UNSPECIFIED/UNKNOWN sentinel convention:

38

"In proto3, the first value defined in an enum definition must have the value zero and should have the name ENUM_TYPE_NAME_UNSPECIFIED or ENUM_TYPE_NAME_UNKNOWN." [WF] — https://protobuf.dev/programming-guides/proto3/

39

"Enums should include a default FOO_UNSPECIFIED value as the first value in the declaration." [WF] — https://protobuf.dev/best-practices/dos-donts/

40

"the first declared enum value should be a default FOO_UNSPECIFIED value and should use tag 0." [WF] — https://protobuf.dev/best-practices/dos-donts/

41Editions makes the open/closed choice a per-file/per-type feature rather than a syntax-wide law:

42

"In edition 2023, the first value defined in an enum definition must have the value zero and should have the name ENUM_TYPE_NAME_UNSPECIFIED or ENUM_TYPE_NAME_UNKNOWN." [WF] — https://protobuf.dev/programming-guides/editions/

43

"If an enum type has been migrated from proto2 using option features.enum_type = CLOSED; there is no restriction on the first value in the enum." [WF] — https://protobuf.dev/programming-guides/editions/

44And adding enum values is a wire-safe change:

45

"Adding additional values to an enum is safe." [WF] — https://protobuf.dev/programming-guides/proto3/

Q4 — Strictness (reject vs ignore unknown fields)

46Answer: protobuf is asymmetric BY ENCODING, not by role — the binary codec is tolerant-and-preserving, the JSON codec is strict-and-lossy. This is the single most important finding for a JSON-in-a-repo format.

47Binary:

48

"Proto3 messages preserve unknown fields and include them during parsing and in the serialized output, which matches proto2 behavior." [WF] — https://protobuf.dev/programming-guides/proto3/

49JSON — the opposite default:

50

"The protobuf JSON parser should reject unknown fields by default but may provide an option to ignore unknown fields in parsing." [WF] — https://protobuf.dev/programming-guides/json/

51So the same schema, encoded in the two blessed encodings, gives you opposite unknown-field policies. Protobuf's own advice is to prefer binary precisely to keep the tolerant behaviour:

52

"Use binary; avoid using text formats for data exchange." [WF] — https://protobuf.dev/programming-guides/proto3/

53"Be liberal in what you accept" and its modern critique. Protobuf does not cite Postel. The authoritative modern critique is IETF RFC 9413 (Maintaining Robust Protocols):

54

"Be strict when sending and tolerant when receiving. Implementations must follow specifications precisely when sending to the network, and tolerate faulty input from the network." [WF, §2] — https://www.rfc-editor.org/rfc/rfc9413.html

55

"An implementation that reacts to variations in the manner recommended in the robustness principle enters a pathological feedback cycle. Over time: Implementations progressively add logic to constrain how data is transmitted or to permit variations in what is received." [WF, §4.1] — https://www.rfc-editor.org/rfc/rfc9413.html

56

"A flaw can become entrenched as a de facto standard. Any implementation of the protocol is required to replicate the aberrant behavior, or it is not interoperable." [WF, §4.1] — https://www.rfc-editor.org/rfc/rfc9413.html

57

"If non-compliance is tolerated by existing implementations, non-compliant implementations can be deployed successfully. Newer implementations then have a strong incentive to tolerate any existing non-compliance in order to be successfully deployed." [WF, §4.2] — https://www.rfc-editor.org/rfc/rfc9413.html

58

"Choosing to generate fatal errors for unspecified conditions instead of attempting error recovery can ensure that faults receive attention." [WF, §5.1] — https://www.rfc-editor.org/rfc/rfc9413.html

59Crucial distinction the RFC forces you to make (and which the naive reading of "tolerant reader" blurs): tolerating unknown extension points that the spec declared extensible is not the same as tolerating malformed or non-conformant data. Protobuf's unknown-field preservation is the former. RFC 9413 attacks the latter.

Q5 — Version semantics

60Answer: protobuf has NO schema version in the data. Evolution is entirely field-level. Editions version the LANGUAGE, not the message.

61There is no version number anywhere on the wire, no schema id, no $schema. Compatibility is a property of each field's tag/type/label history, which is why the entire evolution surface is a list of per-field rules ("Updating A Message Type"). The edition = "2024" marker is a compiler directive about which language defaults apply to a .proto file; it never appears in a serialized message.

62

"The last radical change to Protobuf (syntax = \"proto3\";) split the ecosystem." [WF] — https://raw.githubusercontent.com/protocolbuffers/protobuf/main/docs/design/editions/what-are-protobuf-editions.md

63

"Protobuf Editions replace the proto2 and proto3 designations" [WF] — https://protobuf.dev/news/2023-06-29/

64

"Instead of the hardcoded behaviors in older versions, editions will represent a collection of 'features'" [WF] — https://protobuf.dev/news/2023-06-29/

65Why field-level rather than versioned: NOT FOUND as an explicit design statement. Inferable from the Cap'n Proto FAQ's message-bus story (§1.6 / §3 R1): a version-gated format requires every intermediary to know the version, which is exactly the coupling protobuf's design avoids. But I found no protobuf document that says this in so many words.

Q6 — Field identity

66Answer: numeric tags are the identity. Names are decoration in binary and identity in JSON — a split that matters enormously for a JSON-at-rest format.

67

"Never re-use a tag number. It messes up deserialization. Even if you think no one is using the field, don't re-use a tag number." [WF] — https://protobuf.dev/best-practices/dos-donts/

68

"When you delete a field that's no longer used, reserve its tag number so that no one accidentally re-uses it in the future." [WF] — https://protobuf.dev/best-practices/dos-donts/

69

"Changing field numbers for any existing field is not safe." [WF] — https://protobuf.dev/programming-guides/proto3/

70What breaks when the policy is violated — this list is the best "consequences" enumeration in the whole corpus, because it names data-corruption and PII leakage, not just parse errors:

71

"Reusing a field number makes decoding wire-format messages ambiguous." [WF] — https://protobuf.dev/programming-guides/proto3/

72

"Encoding a field using one definition and then decoding that same field with a different definition can lead to: Developer time lost to debugging; A parse/merge error (best case scenario); Leaked PII/SPII; Data corruption" [WF] — https://protobuf.dev/programming-guides/proto3/

73

"If you [update] a message type by entirely deleting a field, or commenting it out, future developers can reuse the field number when making their own updates to the type. This can cause severe issues" [WF] — https://protobuf.dev/programming-guides/proto2/

74reserved has two halves with different force:

75

"If you [update] a message type by entirely deleting a field...you must [reserve the deleted field number]." [WF] — https://protobuf.dev/programming-guides/proto3/

76

"Reserved field names affect only the protoc compiler behavior and not runtime behavior." [WF] — https://protobuf.dev/programming-guides/proto3/

77That last sentence is the key asymmetry: reserving a number protects the data; reserving a name protects only the build. In a JSON format where names are the identity, only the weaker half exists on the wire — reserving a name is your only tool and it has no runtime force at all unless you build one.

78The name-fragility of ProtoJSON is stated directly in the ecosystem literature:

79

"ProtoJSON format does not support unknown fields, and it puts field and enum value names into encoded messages which makes it much harder to change those names later." [WF] — via https://protobuf.dev/programming-guides/json/ discussion, surfaced in search; see also §6 re-fetch note

80Other identity-adjacent don'ts:

81

"Almost never change the type of a field; it'll mess up deserialization, same as re-using a tag number." [WF] — https://protobuf.dev/best-practices/dos-donts/

82

"Almost never change the default value of a proto field. This causes version skew between clients and servers." [WF] — https://protobuf.dev/best-practices/dos-donts/

83

"Although it won't cause crashes, you'll lose data." (on repeated→scalar) [WF] — https://protobuf.dev/best-practices/dos-donts/

Q7 — Round-trip preservation of unknown fields

84Answer: binary preserves (since 3.5, after a removal-and-restoration — see §3 R1); JSON does not; and several innocent-looking operations silently destroy preservation even in binary.

85

"Proto3 messages preserve unknown fields and include them during parsing and in the serialized output, which matches proto2 behavior." [WF] — https://protobuf.dev/programming-guides/proto3/

86

"Some actions can cause unknown fields to be lost. For example, if you do one of the following, unknown fields are lost: Serialize a proto to JSON. Iterate over all of the fields in a message to populate a new message." [WF] — https://protobuf.dev/programming-guides/proto3/

87

"Use message-oriented APIs, such as CopyFrom() and MergeFrom(), to copy data rather than copying field-by-field" [WF] — https://protobuf.dev/programming-guides/proto3/

88What concretely goes wrong when a reader drops what it did not understand. The protobuf issue #272 thread (2015-04-07 → closed 2017-12-11, 69 comments) is the best corpus of real failure modes anywhere in this research. All of the following are [RAW], extracted from the GitHub REST API:

89The proxy / intermediary case, stated with a diagram by stevvooe, 2016-11-18:

90

"Producer and Consumer could be updated with new fields, while intermediate can remain on the same version. If intermediate is a proxy of sorts, then this is important." [RAW] — https://github.com/protocolbuffers/protobuf/issues/272

91The silent-data-loss-is-indistinguishable-from-default case, matthewrj, 2016-08-31 — this is the sharpest statement of the harm in the entire thread:

92

"We have the same use case where A sends data to B which reads some fields and forwards the message to C. We don't want to have to constantly update B when the schema changes even though it doesn't read any of the new fields. The current behaviour is quite dangerous since C can't tell if one of the new fields was set to the default value or if B is just out of date and lost data." [RAW] — https://github.com/protocolbuffers/protobuf/issues/272

93The cryptographic-signature case, chmod007, 2016-10-05:

94

"Include a signature in the same protobuf as the payload to be signed. To verify the signature, I deserialize, extract and remove the signature, reserialize and verify the signature. This breaks if the signed message contains any new fields unknown to the process verifying the signature." [RAW] — https://github.com/protocolbuffers/protobuf/issues/272

95The stream-processing case, Kaiserchen, 2016-08-24:

96

"This allows the stream processor to continue working even when upstream schema changes happen, we do not need to redeploy our stream processing application, and the new fields end up in the output for free." [RAW] — https://github.com/protocolbuffers/protobuf/issues/272

97

"To add some drama: I think loosing the unknown fields will force us to move to avro" [RAW] — https://github.com/protocolbuffers/protobuf/issues/272

98The deploy-ordering / topological-sort case, Xorlev, 2016-11-18:

99

"Depending on any cycles in data flows, there may be no topological order that produces valid schema updates without doing a 2-step deploy: 1) upgrade proto schema, redeploy all the (many) things that might rely on it 2) update producer to fill in field, deploy producer. Pray all the systems were updated." [RAW] — https://github.com/protocolbuffers/protobuf/issues/272

100The org-coupling case, fducat, 2016-12-12:

101

"The interest of using unknown fields is simply development efficiency by removing team dependencies. Usually one or two BE in the row are interested in the change. Forcing all 12 to update the version in coordination is what we cannot afford." [RAW] — https://github.com/protocolbuffers/protobuf/issues/272

102The silent-deletion objection, jeremyong, 2016-03-14:

103

"I have a lot of concerns about silently deleting data upon deserialization, to the point that even though we have internally been using proto3 for several months, I am considering changing things back to proto2." [RAW] — https://github.com/protocolbuffers/protobuf/issues/272

Q8 — Compatibility taxonomy

104Answer: protobuf does NOT use backward/forward vocabulary. It uses a three-tier wire-safety taxonomy instead. This is a genuine and under-appreciated finding: the backward/forward vocabulary people attribute to protobuf actually comes from Avro-and-Kafka-land.

105Protobuf's own tiers, from "Updating A Message Type" [all WF, https://protobuf.dev/programming-guides/proto3/]:

• 106Binary wire-unsafe: "Changing field numbers for any existing field is not safe." / "Moving fields into an existing oneof is not safe."
• Binary wire-safe: "Adding new fields is safe." / "Removing fields is safe." / "Adding additional values to an enum is safe."
• Binary wire-compatible (conditionally safe): "int32, uint32, int64, uint64, and bool are all compatible." / "For string, bytes, and message fields, singular is compatible with repeated."

107Note that "safe" here is symmetric — protobuf's model is that the same change is fine in both directions, because unknown fields are preserved and missing fields default. The backward/forward split only becomes necessary once you have a format (like Avro or like a strict JSON reader) where the two directions genuinely differ. Protobuf's taxonomy is bidirectional-by-construction; Avro's is not. See §4.

108The proto2 rule list adds many more conditional compatibilities [WF, https://protobuf.dev/programming-guides/proto2/]: "Integer type conversions (int32↔int64, etc.)", "sint32/sint64 compatibility with each other only", "string/bytes compatibility with valid UTF-8", "Embedded messages compatible with bytes", "fixed32↔sfixed32, fixed64↔sfixed64", "Singular↔repeated for string/bytes/message fields", "enum↔int32/uint32/int64/uint64", "map↔repeated message field conversions".

Q1 — Tagged vs untagged unions

109Answer: no untagged unions. Avro unions are tagged by branch INDEX in binary and by TYPE NAME in JSON. The JSON encoding is the interesting one for our purposes, because Avro faced exactly the "JSON union" problem and chose explicit tagging.

110Binary:

111

"A union is encoded by first writing an int value indicating the zero-based position within the union of the schema of its value. The value is then encoded per the indicated schema within the union." [WF] — https://avro.apache.org/docs/1.11.1/specification/

112JSON — the explicit tagging decision:

113

"if its type is null, then it is encoded as a JSON null; otherwise it is encoded as a JSON object with one name/value pair whose name is the type's name and whose value is the recursively encoded value." [WF] — https://avro.apache.org/docs/1.11.1/specification/

114So {"int": 5} rather than 5. Avro's own reason: the JSON encoding still cannot be read without the schema, and the wrapper is what keeps it unambiguous:

115

"The original schema is still required to correctly process JSON-encoded data" because the encoding cannot distinguish between semantically similar types like int versus long. [WF] — https://avro.apache.org/docs/1.12.0/specification/

116Note the branch-index dependency: because binary unions are indexed positionally, reordering union branches is a data-corrupting change, exactly as reordering FlatBuffers union variants is. JSON's name-tagging removes that hazard — a rare case where the JSON encoding is more evolvable than the binary one.

Q2 — Absent vs empty vs null for collections

117Answer: YES, cleanly, and this is Avro's structural advantage over protobuf. [] is an empty array; absence is expressed by ["null", {"type":"array",...}] and encoded as a distinct union branch. Avro has a real null type, so empty/null/absent are three different encodable states.

118The mechanism that makes "absent" meaningful across versions is that defaults live in the schema, not in the code:

119

"A default value for this field, only used when reading instances that lack the field for schema evolution purposes. The presence of a default value does not make the field optional at encoding time. Avro encodes a field even if its value is equal to its default." [WF] — https://avro.apache.org/docs/1.11.1/specification/

120That last sentence is important and frequently misread: Avro always writes the field. There is no "omit if default" optimisation, so there is no ambiguity of the protobuf implicit-presence kind. The default is purely a reader-side mechanism for fields the writer's schema never had.

Q3 — Closed vocabularies (enums)

121Answer: Avro enums were CLOSED and hard-fail, which was a five-year known-bad; a reader-side default was added in 1.9.0 to soften it — and it then didn't work for years. This is a reversal-shaped story (§3, R5).

122The base rule:

123

"if the writer's symbol is not present in the reader's enum and the reader has a default value, then that value is used, otherwise an error is signalled." [WF] — https://avro.apache.org/docs/1.11.1/specification/

124

"A default value for this enumeration, used during resolution when the reader encounters a symbol from the writer that isn't defined in the reader's schema (optional). The value provided here must be a JSON string that's a member of the symbols array." [WF] — https://avro.apache.org/docs/1.11.1/specification/

125The actual discussion (AVRO-1340, reported Jim Donofrio 25/May/13, resolved 02/May/18, fix version 1.9.0):

126

"if the writer's symbol is not present in the reader's enum, then an error is signalled" [WF, quoting the pre-fix spec] — https://issues.apache.org/jira/browse/AVRO-1340

127

"makes it difficult to use enum's because you can never add a enum value and keep old reader's compatible" [WF] — https://issues.apache.org/jira/browse/AVRO-1340

128Five years from report to fix. And then AVRO-3313 (affects 1.9.0, 1.9.1, 1.9.2, 1.10.0, 1.10.1, 1.10.2, 1.11.0) reported the fix did not actually work:

129

Writer schema (v2) enum ["A","B","C"] default "A"; reader schema (v1) enum ["A","B"] default "A"; expected the reader to deserialize unknown "C" as "A"; actual: org.apache.avro.AvroTypeException: No match for C [WF, paraphrase of the reproduction] — https://issues.apache.org/jira/browse/AVRO-3313

130

Resolved: Not A Bug (27/Sep/23) [WF] — https://issues.apache.org/jira/browse/AVRO-3313

131The "Not A Bug" resolution is itself a finding: the enum default applies during schema resolution (reader schema explicitly supplied and differing from writer's), not to a naive single-schema decode, and users repeatedly hit the difference. Practical takeaway: a reader-side fallback that only fires in an explicitly-two-schema code path will be missed by most users.

132Secondary/practitioner guidance found: add symbol defaults pre-emptively because older Avro versions tolerate-and-ignore them, so they become useful once everyone upgrades. [WF] — surfaced via https://medium.com/expedia-group-tech/safety-considerations-when-using-enums-in-avro-schemas-82e18baaa081 (secondary source, flagged as such).

Q4 — Strictness

133Answer: reader ignores writer's extra fields; reader's extra fields must have defaults or it is a hard error. Avro is tolerant in one direction and strict in the other, and it is the strict direction that bites.

134

"if the writer's record contains a field with a name not present in the reader's record, the writer's value for that field is ignored." [WF] — https://avro.apache.org/docs/1.11.1/specification/

135

"if the reader's record schema has a field that contains a default value, and writer's schema does not have a field with the same name, then the reader should use the default value from its field." [WF] — https://avro.apache.org/docs/1.11.1/specification/

136

"if the reader's record schema has a field with no default value, and writer's schema does not have a field with the same name, an error is signalled." [WF] — https://avro.apache.org/docs/1.11.1/specification/

137Note the word "ignored" in the first rule — not "preserved". See Q7.

Q5 — Version semantics

138Answer: no version number; Avro versions by carrying the WRITER'S SCHEMA WITH THE DATA. This is the most important structural idea in Avro and the one that transfers best to files-at-rest.

139

"a reader of Avro data, whether from an RPC or a file, can always parse that data because the original schema must be provided along with the data" [WF] — https://avro.apache.org/docs/1.12.0/specification/

140

"Binary encoded Avro data does not include type information or field names." [WF] — https://avro.apache.org/docs/1.12.0/specification/

141Evolution is then a pairwise function of two schemas, not a linear version ladder. There is no "version 3"; there is only "resolve writer-S1 against reader-S2". That means compatibility is a relation, not an ordering — a point §4 develops.

Q6 — Field identity

142Answer: NAMES, not numbers — the opposite of protobuf/Thrift/Cap'n Proto/FlatBuffers. Renaming is therefore a breaking change, softened by aliases.

143

"if the writer's record contains a field with a name not present in the reader's record, the writer's value for that field is ignored" [WF] — https://avro.apache.org/docs/1.11.1/specification/ (field matching is by name)

144

"Named types and fields may have aliases. An implementation may optionally use aliases to map a writer's schema to the reader's. This facilitates both schema evolution as well as processing disparate datasets." [WF] — https://avro.apache.org/docs/1.11.1/specification/

145Note "An implementation MAY optionally use aliases" — alias support is not guaranteed. That is a materially weaker guarantee than protobuf's reserved, which is enforced by the compiler.

146There is no reserved mechanism in Avro. Avro reserved equivalent: NOT FOUND — searched the 1.11.1 and 1.12.0 specifications; Avro has no construct for retiring a name so it cannot be reused with different semantics. Name reuse with a changed type is caught only if the reader happens to resolve against the old writer's schema and the types fail to match.

147Practitioner-level rules found (secondary sources, flagged): you cannot rename a field (use aliases); you cannot change a field's data type (add a new field); "A non-union type may be changed to a union that contains only the original type, or vice-versa"; "If you do not provide a default value for a field, you cannot delete that field from your schema." [WF] — https://docs.oracle.com/cd/E26161_02/html/GettingStartedGuide/schemaevolution.html

Q7 — Round-trip preservation of unknown fields

148Answer: NO. Avro DROPS unknown fields — "ignored" is the spec's word — and this is the concrete reason people cited protobuf's unknown-field preservation as the reason to prefer protobuf over Avro (and, in issue #272, the reverse).

149

"if the writer's record contains a field with a name not present in the reader's record, the writer's value for that field is ignored." [WF] — https://avro.apache.org/docs/1.11.1/specification/

150Avro's mitigation is different in kind: because the writer's schema travels with the data (object container files), a reader that resolves against the writer's schema loses nothing — but any reader that projects onto its own reader schema and re-writes has performed a lossy transformation. Avro trades "the reader can preserve what it doesn't understand" for "the data can always be re-read later with full fidelity, given the original file." For files-at-rest that trade is arguably better; for read-modify-write pipelines it is worse.

Q8 — Compatibility taxonomy

151Avro's own spec defines NO backward/forward taxonomy — NOT FOUND in the specification. Searched the 1.11.1 and 1.12.0 specs; they define schema resolution rules only. The backward/forward/full vocabulary that everyone attributes to Avro is Confluent Schema Registry's, layered on top. See §1.7 and §4.

Q1 — Tagged vs untagged unions

152Answer: no untagged unions. Thrift's union is a struct in which at most one field is set, and every field carries a numeric field id in its wire header, so the variant is identified by id. NOT FOUND: an authoritative Apache Thrift statement specifically about union semantics — https://thrift.apache.org/docs/types returned "Unions are not mentioned in this content" and https://thrift.apache.org/docs/idl documents field ids and requiredness but I did not obtain a union section. Searched thrift.apache.org/docs/idl, /docs/types, and the 2007 whitepaper.

Q2 — Absent vs empty vs null for collections

153Answer: YES for scalars via the generated __isset struct; for collections it depends on requiredness. The __isset design is Thrift's explicit-presence mechanism and predates protobuf's rediscovery of the same idea.

154[RAW] from the 2007 Facebook whitepaper, §5:

155

"Essentially, the inner isset object of each Thrift struct contains a boolean value for each field which denotes whether or not that field is present in the struct. When a reader receives a struct, it should check for a field being set before operating directly on it." [RAW] — https://thrift.apache.org/static/files/thrift-20070401.pdf

156The three requiredness levels [WF] — https://thrift.apache.org/docs/idl:

157

"Required fields are always written and are expected to be set." / "Required fields are always read and are expected to be contained in the input stream." "Optional fields are only written when they are set" / "Optional fields may, or may not be part of the input stream." (default/implicit) "In theory, the fields are always written. There are some exceptions to that rule." / "Like optional, the field may, or may not be part of the input stream."

158

"Default requiredness is a good starting point. The desired behaviour is a mix of optional and required." [WF] — https://thrift.apache.org/docs/idl

Q3 — Closed vocabularies (enums)

159Answer: Thrift enums are i32 on the wire; behaviour on unknown values is implementation-dependent and NOT specified authoritatively. NOT FOUND in thrift.apache.org/docs/types (explicitly returned NOT FOUND) or /docs/idl.

160The best statement I found is secondary, from Thrift: The Missing Guide (Diwaker Gupta), §1.3 [RAW, extracted from the PDF]:

161

"a field with an enum type can only have one of a specified set of constants as its value (if you try to provide a different value, the parser will treat it like an unknown field)" [RAW] — https://diwakergupta.github.io/thrift-missing-guide/thrift.pdf

162

"Enumerator constants MUST be in the range of postive 32-bit integers." [RAW, postive sic] — https://diwakergupta.github.io/thrift-missing-guide/thrift.pdf

163Practitioner reports confirm the hazard: adding enum values can break jobs running an older Thrift schema. [WF] — surfaced via https://cwiki.apache.org/confluence/display/FLINK/FLIP-237:+Thrift+Format+Support (secondary).

Q4 — Strictness

164Answer: tolerant reader by design, stated in 2007 and unchanged. [RAW] from the whitepaper §5.3 "Case Analysis" — this is the four-case table the question asks about:

165

"1. Added field, old client, new server. In this case, the old client does not send the new field. The new server recognizes that the field is not set, and implements default behavior for out-of-date requests." [RAW]

166

"2. Removed field, old client, new server. In this case, the old client sends the removed field. The new server simply ignores it." [RAW]

167

"3. Added field, new client, old server. The new client sends a field that the old server does not recognize. The old server simply ignores it and processes as normal." [RAW]

168

"4. Removed field, new client, old server. This is the most dangerous case, as the old server is unlikely to have suitable default behavior implemented for the missing field. It is recommended that in this situation the new server be rolled out prior to the new clients." [RAW]

169— all https://thrift.apache.org/static/files/thrift-20070401.pdf

170Case 4 is Thrift's own naming of the forward-compatibility hazard and its own prescription of an upgrade ordering — thirteen years before Confluent's tables said the same thing. Note it is a deployment-order remedy, not a format remedy.

Q5 — Version semantics

171Answer: no schema version in the data; and Thrift explicitly separates protocol-level versioning from IDL-level versioning. [RAW], whitepaper §5.4:

172

"The TProtocol abstractions are also designed to give protocol implementations the freedom to version themselves in whatever manner they see fit. Specifically, any protocol implementation is free to send whatever it likes in the writeMessageBegin() call. It is entirely up to the implementor how to handle versioning at the protocol level. The key point is that protocol encoding changes are safely isolated from interface definition version changes." [RAW] — https://thrift.apache.org/static/files/thrift-20070401.pdf

173This two-layer split is a genuinely transferable idea: version the envelope/encoding explicitly and let the content evolve by field rules. See §5.

Q6 — Field identity

174Answer: numeric field identifiers, and the whitepaper already recommends always writing them explicitly. [RAW], §5.1:

175

"Versioning in Thrift is implemented via field identifiers. The field header for every member of a struct in Thrift is encoded with a unique field identifier. The combination of this field identifier and its type specifier is used to uniquely identify the field. The Thrift definition language supports automatic assignment of field identifiers, but it is good programming practice to always explicitly specify field identifiers." [RAW] — https://thrift.apache.org/static/files/thrift-20070401.pdf

176Note "The combination of this field identifier and its type specifier" — Thrift's identity is (id, type), so changing a field's type is a distinct field, not a redefinition. That is a subtly different (and arguably safer) identity model than protobuf's id-alone.

177Thrift reserved mechanism: NOT FOUND. Searched thrift.apache.org/docs/idl and the whitepaper. Thrift has no reserved-id construct; the discipline is purely conventional.

Q7 — Round-trip preservation of unknown fields

178Answer: NO — unknown fields are skipped, not retained. Authoritative statement NOT FOUND, but the whitepaper's "simply ignores it" wording (cases 2 and 3 above) plus the absence of any unknown-field-set API in generated code is dispositive in practice. Searched thrift.apache.org/docs/idl, /docs/types, the whitepaper, and the Missing Guide. There is no UnknownFieldSet analogue in Thrift's generated code model.

Q8 — Compatibility taxonomy

179Answer: no named taxonomy; the four-case analysis in §5.3 IS Thrift's taxonomy. Thrift does not use the words backward/forward. NOT FOUND: any published Facebook retrospective on the required/optional decision. Searched for "Thrift retrospective", "Facebook Thrift lessons", "required fields harmful Thrift". The closest published critique is Thrift's own IDL documentation, which is unusually blunt for official docs:

180

"Because of this behaviour, required fields drastically limit the options with regard to soft versioning." [WF] — https://thrift.apache.org/docs/idl

181

"Because they must be present on read, the fields cannot be deprecated." [WF] — https://thrift.apache.org/docs/idl

182

"If a required field would be removed (or changed to optional), the data are no longer compatible between versions." [WF] — https://thrift.apache.org/docs/idl

183This is Thrift's own retrospective on required, in its official docs, reaching the same conclusion protobuf reached — without ever removing the keyword. See §3, R1.

Q1 — Tagged vs untagged unions

185Answer: JTD has ONLY tagged unions. Untagged unions are absent by design, and the RFC gives the rationale twice — once positively (code generation) and once negatively (ambiguity).

186The positive rationale, §1:

187

"JTD's niche is to focus on enabling code generation from schemas; to this end, JTD's expressiveness is intentionally limited to be no more powerful than what can be expressed in the type systems of mainstream programming languages." [RAW]

188

"Enable code generation from JTD schemas. JTD schemas are meant to be easy to convert into data structures idiomatic to mainstream programming languages." [RAW]

189

"JTD is intentionally designed as a rather minimal schema language. Thus, although JTD can describe some categories of JSON, it is not able to describe its own structure... By keeping the expressiveness of the schema language minimal, JTD makes code generation and standardized error indicators easier to implement." [RAW]

190

"A \"discriminator\" form of JSON objects, corresponding to a discriminated (or \"tagged\") union. The \"discriminator\" form of JSON objects is akin to a C++ \"std::variant\"." [RAW]

191

"JTD's feature set is designed to represent common patterns in JSON-using applications, while still having a clear correspondence to programming languages in widespread use." [RAW]

192

"The principle of clear correspondence to common programming languages is why JTD does not support, for example, a data type for integers up to 2**53-1." [RAW]

193The negative rationale — §2.2.8 is an entire section devoted to making tags unambiguous by construction. This is the most directly applicable text in the whole research corpus for a JSON-in-a-repo format:

194

"To prevent ambiguous or unsatisfiable constraints on the \"discriminator\" property of a tagged union, an additional constraint on schemas of the \"discriminator\" form exists." [RAW]

195

"* For each member P of S whose name equals \"properties\" or \"optionalProperties\", P's value, which must be an object, MUST NOT contain any members whose name equals D's value." [RAW]

196

"JTD handles such possible ambiguity by disallowing, at the syntactic level, the possibility of contradictory specifications of discriminator \"tags\". Discriminator \"tags\" cannot be redefined in other parts of the schema." [RAW]

197

"JTD handles such possible ambiguity by disallowing, at the syntactic level, the possibility of contradictory specifications of whether an instance described by a schema of the \"discriminator\" form may be null. The schemas in a discriminator \"mapping\" cannot have \"nullable\" set to \"true\"; only the discriminator itself can use \"nullable\" in this way." [RAW]

198Note the repeated phrase "disallowing, at the syntactic level" — JTD's method is to make the ambiguous schema unwritable, not to define a resolution rule for it. Contrast OpenAPI's discriminator (§1.6), which is a hint layered over a validation model that would work without it.

199Confirmation that JTD refuses type-unions generally, from the Ajv implementation docs:

200

"Unlike JSON Schema, JTD does not allow defining values that can take one of several types, but they can be defined as nullable." [WF] — https://ajv.js.org/json-type-definition.html

Q2 — Absent vs empty vs null for collections

201Answer: YES — JTD distinguishes all three cleanly, and it is the only format here that does so with three distinct, orthogonal, first-class mechanisms.

• 202absent: put the member in optionalProperties rather than properties. §3.3.6: "For every member name in P, a member of the same name in the instance must exist." [RAW] — i.e. properties members are mandatory-present.
• null: nullable: true. §3.3.3 (via [WF]): "If the schema has a member named 'nullable' whose value is the boolean 'true', and the instance is the JSON primitive value 'null', then the schema accepts the instance." — https://www.rfc-editor.org/rfc/rfc8927.html
• empty: the elements form matching [].

203And a sharp asymmetry worth copying: nullable: false is inert.

204

"it is not the case that putting a 'false' value for 'nullable' will ever override a 'nullable' member." [WF] — https://www.rfc-editor.org/rfc/rfc8927.html

Q3 — Closed vocabularies (enums)

205Answer: CLOSED and hard-rejecting. There is no open-enum, no sentinel convention, no fallback.

206

"For a schema of the \"enum\" form to be correct, the value of the member named \"enum\" must be a nonempty array of strings, and that array must not contain duplicate values." [WF, §2.2.4] — https://www.rfc-editor.org/rfc/rfc8927.html

207Validating a value not in the list produces an error with schemaPath pointing to /enum — unknown values are rejected. [WF, §3.3.4] — https://www.rfc-editor.org/rfc/rfc8927.html

208JTD provides no enum-evolution story at all. This is a real limitation for a long-lived at-rest format and JTD does not pretend otherwise.

Q4 — Strictness

209Answer: STRICT BY DEFAULT — unknown members are rejected unless the schema opts in. JTD is the anti-Postel design point in this corpus, and unusually, the RFC acknowledges the disagreement explicitly rather than asserting a winner.

210

"Some users may expect that {\"a\": \"foo\", \"b\": \"bar\"} satisfies the schema in Figure 2. Others may disagree, as \"b\" is not one of the properties described in the schema." [RAW, §3.1]

211

"Evaluation of a schema does not allow additional properties by default, but this can be overridden by having the schema include a member named \"additionalProperties\", where that member has a value of \"true\"." [RAW, §3.1]

212

"briefly, the schema { \"properties\": { \"a\": { \"type\": \"string\" }}} rejects { \"a\": \"foo\", \"b\": \"bar\" }" [RAW, §3.1]

213The one deliberate hole in the strictness — the "discriminator tag exemption" [RAW, §3.3.6]:

214

"If the \"discriminator tag exemption\" is in effect on I (see Section 3.3.8), then ignore I." [RAW]

215That is: the tag property itself is exempted from the variant schema's "no additional properties" rule, because the variant schema doesn't declare the tag (it's declared once, on the union). A small but instructive piece of engineering — the strictness rule and the tagging rule would otherwise collide, and JTD carved the exemption at exactly one point rather than loosening either rule.

216Confirmation from the implementation side:

217

"Unlike JSON Schema, all properties defined in properties schema member are required, the data instance must be JSON object (without using additional type keyword) and by default additional properties are not allowed (with the exception of discriminator tag)." [WF] — https://ajv.js.org/json-type-definition.html

Q5 — Version semantics

218Answer: NONE. JTD has no versioning mechanism whatsoever, and no evolution rules. The only extension point is inert metadata.

219

"Users MAY add metadata members to JTD schemas to convey information that is not pertinent to validation." [WF, §2.3] — https://www.rfc-editor.org/rfc/rfc8927.html

220

"Users SHOULD NOT expect metadata members to be understood by other parties. As a result, if consistent validation with other parties is a requirement, users MUST NOT use metadata members to affect how schema validation...works." [WF, §2.3] — https://www.rfc-editor.org/rfc/rfc8927.html

221JTD schema-evolution rules: NOT FOUND — because there are none. Searched the full RFC text for "evolution", "compatib", "version". The RFC is a validation spec, not an evolution spec. This is a genuine and important gap: JTD gives you a rigorous way to say what a document looks like today and no way at all to say how it may change.

222The RFC is also unusually honest about its own status:

223

"This document does not have IETF consensus and is presented here to facilitate experimentation with the concept of JTD. The purpose of the experiment is to gain experience with JTD and to possibly revise this work accordingly." [RAW, §1]

Q6 — Field identity

224Answer: names only. No numeric tags, no reserved mechanism, no aliases. JTD is a validation language over JSON, and JSON member names are the only identity available. NOT FOUND: any reuse policy. Searched the full RFC.

Q7 — Round-trip preservation

225Answer: N/A for the validator; NEGATIVE for the code generators, which is the point. JTD does not define a codec, so it neither preserves nor drops. But its raison d'être is code generation into structs, and generated structs by construction drop what is not declared. additionalProperties: true permits unknown members to pass validation; it does not create a place to store them.

226Explicit statement about round-trip preservation in JTD: NOT FOUND. Searched RFC 8927 in full and the Ajv JTD docs.

Q8 — Compatibility taxonomy

227NOT FOUND — JTD defines none. Searched RFC 8927 in full.

Author's design writing (requested, partial gap)

228Ulysse Carion's own blog post articulating JTD's design goals vs JSON Schema: NOT FOUND. Searched: "Ulysse Carion JSON Type Definition design why not JSON Schema blog rationale discriminator"; "jsontypedef JSON Type Definition Carion blog tagged unions why no untagged union design goal code generation"; and direct fetches of https://jsontypedef.com/ and https://jsontypedef.com/docs/. Note: jsontypedef.com is no longer under the author's control — the domain now serves unrelated casino-affiliate content. The canonical design rationale surviving in a citable, stable form is RFC 8927 §1 and Appendix A ("Rationale for Omitted Features", §A.1 on 64-bit numbers, §A.2 on non-root definitions) plus the github.com/jsontypedef org. If this rationale matters to the decision, RFC 8927 Appendix A is the artifact to cite, not the website.

Cap'n Proto

229Q1 — unions: explicitly tagged, with a stored discriminant.

230

"A union is two or more fields of a struct which are stored in the same location. Only one of these fields can be set at a time, and a separate tag is maintained to track which one is currently set." [WF] — https://capnproto.org/language.html

231Q2 — absent/empty/null: Cap'n Proto deliberately has no optional and no presence.

232

"Cap'n Proto has no notion of 'optional' fields." [RAW] — https://capnproto.org/faq.html

233

"A primitive field always takes space on the wire whether you set it or not (although default-valued fields will be compressed away if you enable packing). Such a field can be made semantically optional by placing it in a union with a Void field" [RAW] — https://capnproto.org/faq.html

234

"A better approach may be to give the field a bogus default value and interpret that value to mean 'not present'." [RAW] — https://capnproto.org/faq.html

235

"Pointer fields are a bit different. They start out 'null', and you can check for nullness using the hasFoo() accessor. You could use a null pointer to mean 'not present'. Note, though, that calling getFoo() on a null pointer returns the default value, which is indistinguishable from a legitimate value, so checking hasFoo() is in fact the only way to detect nullness." [RAW] — https://capnproto.org/faq.html

236Q3 — enums: new enumerants may be added at the end.

237

"New fields, enumerants, and methods may be added to structs, enums, and interfaces, respectively, as long as each new member's number is larger than all previous members." [WF] — https://capnproto.org/language.html

238Q4/Q7 — strictness and unknown-field retention: Cap'n Proto retains unknown fields, and its author flagged proto3's removal of the same as a mistake at the time:

239

Feature matrix "Unknown field retention" — Cap'n Proto: "yes"; Protobuf: "removed in proto3"; SBE: "no"; FlatBuffers: "no". [WF] — https://capnproto.org/news/2014-06-17-capnproto-flatbuffers-sbe.html

240

"Apparently, version 3 of Protocol Buffers, aka 'proto3', removes this feature. I honestly don't know what they're thinking." [WF] — https://capnproto.org/news/2014-06-17-capnproto-flatbuffers-sbe.html

241

"This feature has been absolutely essential in many of Google's internal systems." [WF] — https://capnproto.org/news/2014-06-17-capnproto-flatbuffers-sbe.html

242Q5 — version semantics: none; ordinal-based field-level rules only.

243Q6 — field identity: ordinals (@N) and type IDs. The allowed/forbidden list [WF] — https://capnproto.org/language.html:

244

Allowed: "New types, constants, and aliases can be added anywhere, since they obviously don't affect the encoding of any existing type." / "New parameters may be added to a method. The new parameters must be added to the end of the parameter list and must have default values." / "Members can be re-arranged in the source code, so long as their numbers stay the same." / "Any symbolic name can be changed, as long as the type ID / ordinal numbers stay the same." / "Type definitions can be moved to different scopes, as long as the type ID is declared explicitly." / "A field can be moved into a group or a union, as long as the group/union and all other fields within it are new."

245

Forbidden: "You cannot change a field, method, or enumerant's number" / "change a field or method parameter's type or default value" / "You cannot change a type's ID" / "move an existing field into or out of an existing union, nor can you form a new union containing more than one existing field."

246Q8 — taxonomy: none named.

247The explicit comparison to protobuf's mistakes (this is the requested item, and it is the single most vivid passage found in this entire research effort — all [RAW], https://capnproto.org/faq.html):

248

"You don't. You may find this surprising, but the 'required' keyword in Protocol Buffers turned out to be a horrible mistake." [RAW]

249

"The problem with this is, validation is sometimes more subtle than that. Sometimes, different applications – or different parts of the same application, or different versions of the same application – place different requirements on the same protocol. An application may want to pass around partially-complete messages internally. A particular field that used to be required might become optional. A new use case might call for almost exactly the same message type, minus one field, at which point it may make more sense to reuse the type than to define a new one." [RAW]

250

"A field declared required, unfortunately, is required everywhere. The validation is baked into the parser, and there's nothing you can do about it. Nothing, that is, except change the field from 'required' to 'optional'. But that's where the real problems start." [RAW]

251

"Imagine a production environment in which two servers, Alice and Bob, exchange messages through a message bus infrastructure running on a big corporate network. The message bus parses each message just to examine the envelope and decide how to route it, without paying attention to any other content. Often, messages from various applications are batched together and then split up again downstream." [RAW]

252

"Now, at some point, Alice's developers decide that one of the fields in a deeply-nested message commonly sent to Bob has become obsolete. To clean things up, they decide to remove it, so they change the field from 'required' to 'optional'. The developers aren't idiots, so they realize that Bob needs to be updated as well. They make the changes to Bob, and just to be thorough they run an integration test with Alice and Bob running in a test environment. The test environment is always running the latest build of the message bus, but that's irrelevant anyway because the message bus doesn't actually care about message contents; it only does routing. Protocols are modified all the time without updating the message bus." [RAW]

253

"Satisfied with their testing, the devs push a new version of Alice to prod. Immediately, everything breaks. And by 'everything' I don't just mean Alice and Bob. Completely unrelated servers are getting strange errors or failing to receive messages. The whole data center has ground to a halt and the sysadmins are running around with their hair on fire." [RAW]

254

"What happened? Well, the message bus running in prod was still an older build from before the protocol change. And even though the message bus doesn't care about message content, it does need to parse every message just to read the envelope. And the protobuf parser checks the entire message for missing required fields. So when Alice stopped sending that newly-optional field, the whole message failed to parse, envelope and all. And to make matters worse, any other messages that happened to be in the same batch also failed to parse, causing errors in seemingly-unrelated systems that share the bus." [RAW]

255

"Things like this have actually happened. At Google. Many times." [RAW]

256

"The right answer is for applications to do validation as-needed in application-level code. If you want to detect when a client fails to set a particular field, give the field an invalid default value and then check for that value on the server. Low-level infrastructure that doesn't care about message content should not validate it at all." [RAW]

257

"Oh, and also, Cap'n Proto doesn't have any parsing step during which to check for required fields. :)" [RAW]

258The generalisable law in that story: validation baked into the PARSER propagates failure to every party that must parse, including parties that do not care about the content. For a JSON-file format read by third-party tools, the analogue is exact: a strict schema validator wired into the load path of a generic tool turns every schema addition into a breakage of tools that never looked at the new field.

FlatBuffers

259Q1 — unions: tagged, with a discriminant, and the discriminant is positional.

260

"New union variants must be appended at the end to prevent discriminant mismatches. Adding variants mid-union causes 'CodeV1' and 'CodeV2' to misinterpret values. Using explicit discriminant values (e.g., A = 1) allows middle insertion safely by overriding positional assignment." [WF] — https://flatbuffers.dev/evolution/

261Q2 — absent/empty: tables encode presence via the vtable; a field not written is absent and reads as its default. Structs cannot omit anything:

262

"structs... are required (so no defaults either), and fields may not be added or be deprecated." [WF] — https://flatbuffers.dev/schema/

263

"Fields do not have to appear in the wire representation, and you can choose to omit fields when constructing an object. You have the flexibility to add fields without fear of bloating your data." [WF] — https://flatbuffers.dev/schema/

264Q3 — enums: append-only, and FlatBuffers explicitly pushes unknown-value handling to the application:

265

"Typically, enum values should only ever be added, never removed (there is no deprecation for enums). This requires code to handle forwards compatibility itself, by handling unknown enum values." [WF] — https://flatbuffers.dev/schema/

266Q4/Q7 — strictness / unknown-field retention: unknown fields are skipped and NOT retained ("no" in Cap'n Proto's matrix above). Confirmed by the absence of any unknown-field API in FlatBuffers' model.

267Q5 — version semantics: none.

268Q6 — field identity: slot order, or explicit id:

269

"New fields MUST be added to the end of the table definition." [WF] — https://flatbuffers.dev/evolution/

270

"You MUST not remove a field from the schema, even if you don't use it anymore." [WF] — https://flatbuffers.dev/evolution/

271

"do not generate accessors for this field anymore, code should stop using this data. Old data may still contain this field, but it won't be accessible anymore by newer code." (the deprecated attribute) [WF] — https://flatbuffers.dev/schema/

272

"If you use this attribute, you must use it on ALL fields of this table, and the numbers must be a contiguous range from 0 onwards... When a new field is added to the schema it must use the next available ID." (the id attribute) [WF] — https://flatbuffers.dev/schema/

273

"You can ignore this rule if you use the id attribute on all the fields of a table." [WF] — https://flatbuffers.dev/evolution/

274Renaming is safe because names are not serialized:

275

"Renaming tables and fields is generally permissible since names aren't serialized." [WF] — https://flatbuffers.dev/evolution/

276Q8 — taxonomy: none named.

277Explicit comparison to protobuf's mistakes: NOT FOUND in FlatBuffers' own docs. The FlatBuffers schema page "contains no explicit Protocol Buffers comparison regarding evolution rules" [WF]. The comparison exists only from the outside (Cap'n Proto's 2014 matrix, above).

278FlatBuffers' deprecated is the cleanest reserved analogue in the corpus: it keeps the slot occupied forever and removes the accessor, so the compiler enforces "you cannot use this and you cannot reuse this" in one attribute. Protobuf's reserved does the second half only; nobody else does either half well.

Q1 — Tagged vs untagged unions

280Answer: JSON Schema's oneOf/anyOf ARE untagged unions — matching is by full structural validation. OpenAPI's discriminator bolts a tag on top, but only as an optimisation hint, not as the semantics. This is the crucial architectural difference from JTD and the reason the OpenAPI discriminator is a persistent source of trouble.

281

"When request bodies or response payloads may be one of a number of different schemas, a discriminator object can be used to aid in serialization, deserialization, and validation. The discriminator is a specific object in a schema which is used to inform the consumer of the specification of an alternative schema based on the value associated with it." [RAW]

282

"The discriminator object is legal only when using one of the composite keywords oneOf, anyOf, allOf." [RAW]

283The "hint" sentence — this is the single most consequential sentence in the OpenAPI discriminator design:

284

"which means the payload MUST, by validation, match exactly one of the schemas described by Cat, Dog, or Lizard. In this case, a discriminator MAY act as a \"hint\" to shortcut validation and selection of the matching schema which may be a costly operation, depending on the complexity of the schema." [RAW]

285Because the discriminator is a hint over an underlying structural oneOf, two tools can legitimately disagree: one dispatches on the tag, the other validates all branches. Where the two disagree — a payload whose tag says Dog but whose shape matches Cat — behaviour is unspecified.

286The required-ness of the tag property, and the exclusion of inline schemas:

287

"When used, the discriminator will be the name of the property that decides which schema definition validates the structure of the model. As such, the discriminator field MUST be a required field." [RAW]

288

"As such, inline schema definitions, which do not have a given id, cannot be used in polymorphism." [RAW]

289

"When using the discriminator, inline schemas will not be considered." [RAW]

290

"propertyName | string | REQUIRED. The name of the property in the payload that will hold the discriminator value." [RAW]

291

"mapping | Map[string, string] | An object to hold mappings between payload values and schema names or references." [RAW]

292Unknown tag value:

293

"If the discriminator value does not match an implicit or explicit mapping, no schema can be determined and validation SHOULD fail. Mapping keys MUST be string values, but tooling MAY convert response values to strings for comparison." [RAW]

294Note SHOULD, not MUST, and "tooling MAY convert response values to strings" — two more places where conforming tools may diverge.

295

"When used in conjunction with the anyOf construct, the use of the discriminator can avoid ambiguity where multiple schemas may satisfy a single payload." [RAW]

296

"In both the oneOf and anyOf use cases, all possible schemas MUST be listed explicitly. To avoid redundancy, the discriminator MAY be added to a parent schema definition, and all schemas comprising the parent schema in an allOf construct may be used as an alternate schema." [RAW]

297"all possible schemas MUST be listed explicitly" is a closed-world requirement: an OpenAPI union cannot be open for extension by a third party. Compare JTD (also closed) and protobuf oneof (also closed). No format in this corpus supports an extensible/open union. That is itself a finding.

Q2 — Absent vs empty vs null

298Answer: YES — JSON Schema distinguishes all three (required for presence, type: "null" for null, [] for empty), but OpenAPI 3.0 famously did NOT have type: null and used a bespoke nullable: true instead; 3.1 realigned with JSON Schema. NOT FOUND: an authoritative verbatim quote on the 3.0→3.1 nullable change — I did not fetch a 3.1 section covering it (the 3.1 HTML fetch truncated before reaching the relevant sections). Flagging as a gap.

Q3 — Closed vocabularies

299JSON Schema enum is closed and rejecting, like JTD. NOT FOUND: a verbatim JSON Schema statement on enum evolution, which does not exist because JSON Schema has no evolution model.

Q4 — Strictness

300Answer: JSON Schema is TOLERANT by default — the opposite of JTD — and the mechanism for tightening it is famously broken under composition.

301

"By default any additional properties are allowed." [WF] — https://json-schema.org/understanding-json-schema/reference/object

302

"additionalProperties only recognizes properties declared in the same subschema as itself. So, additionalProperties can restrict you from 'extending' a schema using combining keywords such as allOf." [WF] — https://json-schema.org/understanding-json-schema/reference/object

303

"Because additionalProperties only recognizes properties declared in the same subschema, it considers anything other than 'street_address', 'city', and 'state' to be additional. Combining the schemas with allOf doesn't change that." [WF] — https://json-schema.org/understanding-json-schema/reference/object

304

unevaluatedProperties is "similar to additionalProperties except that it can recognize properties declared in subschemas." [WF] — https://json-schema.org/understanding-json-schema/reference/object

305This is a design reversal of a sort (§3, R7): additionalProperties proved unusable in the presence of allOf, and rather than change its semantics, JSON Schema added a second, differently-scoped keyword alongside it.

Q5 — Version semantics

306JSON Schema versions the schema language via $schema, not the document. Documents carry no version. NOT FOUND: any JSON Schema statement on document/instance evolution — none exists; JSON Schema is explicitly a validation vocabulary.

Q6 — Field identity

307Names only. No reserved mechanism, no aliases, no reuse policy. NOT FOUND.

Q7 — Round-trip preservation

308N/A (validator, not codec). With additionalProperties unconstrained (the default), unknown members validate; whether they survive depends entirely on the consuming code, which is exactly the ambiguity JTD closed.

Q8 — Compatibility taxonomy

309NOT FOUND in JSON Schema / OpenAPI. Supplied externally by Confluent (§1.7).

Known problems (requested)

310Beyond the "hint" ambiguity and the additionalProperties/allOf scoping failure above, the practitioner literature documents: the spec's ambiguity requiring trial-and-error to produce correct schemas; the interaction requiring both type and required to be present; code-generator divergence on propertyName handling in oneOf with mapping; and the redundancy argument — a oneOf over variants with distinct required sets already discriminates, making the keyword unnecessary in many cases. [WF, secondary sources] — https://github.com/OAI/OpenAPI-Specification/issues/2376 ; https://bump.sh/blog/the-discriminator-in-openapi-is-generally-redundant-and-confusing/ ; https://github.com/OpenAPITools/openapi-generator/issues/20954