Schema Reference

A schema tells EByte what a binary format looks like: the fields in order, how wide each one is, and how they depend on each other. This page covers version 1 of the schema format, the one EByte reads. The repeat type needs EByte 0.2.0 or later. Worked schemas for real formats are in the schema library. The full contract, with every edge case, is schema-v1.md on GitHub.

The File

A schema is a UTF-8 JSON file with exactly four members:

{
  "schema_version": 1,
  "name": "ebyte_packet",
  "endian": "little",
  "fields": [ ... ]
}

endian is little or big, and integers inherit it unless they set their own. Names are ASCII letters, digits and underscores, up to 64 characters, starting with a letter or underscore. EByte is strict about the JSON: unknown members, duplicate members, null values and invalid Unicode are all rejected before any binary input is opened.

A schema’s identity is the SHA-256 of its exact bytes, whitespace included. Documents carry that hash, so a document decoded with one schema can’t be encoded with another by accident. Reformatting a schema file changes its identity.

Field Types

Every field has a name and a type.

Integers are strings in the document so that 64-bit values never pass through floating point.

An integer can set its own endian. A length or count is either a literal number or a reference to an earlier integer in the same object, written as {"field": "data_size"}. Strings are exact-width: EByte doesn’t trim, pad or look for a terminator. If a format pads a field, declare the padding as its own field and it’ll be kept.

Repeating Blocks

Many formats are a run of blocks with no count anywhere: RIFF chunks, ZIP entries, PNG chunks. A repeat field reads them, with until saying when to stop.

"until": "end_of_input" keeps reading elements while any byte is left. It has to be the last field of the schema and can’t have when.

{"name": "chunks", "type": "repeat", "until": "end_of_input", "fields": [
  {"name": "id", "type": "string", "encoding": "ascii", "length": 4},
  {"name": "size", "type": "u32"},
  {"name": "body", "type": "bytes", "length": {"field": "size"}}
]}

"until": "tag_mismatch" keeps reading while the next bytes equal a tag. The element’s first field is the tag: an unconditional bytes field of 1 to 16 bytes with expect. When the next bytes differ, or the input runs out, the repeat ends and the following fields carry on from there.

{"name": "entries", "type": "repeat", "until": "tag_mismatch", "fields": [
  {"name": "signature", "type": "bytes", "length": 4, "expect": "504b0304"},
  ...
]}

In the document a repeat is an array of objects, like an array, and you can add, remove or reorder elements freely. Every element has to take at least one byte. On encode, EByte refuses output whose bytes right after a tag repeat start with the tag, because reading the file back would take them for another element.

Checks, Conditions and Derived Counts

expect checks a fixed value. On an integer it’s a decimal string, on a fixed-length bytes field it’s lowercase hex. A magic number is the usual case:

{"name": "magic", "type": "bytes", "length": 2, "expect": "4542"}

A mismatch is an error on decode and on encode. EByte never corrects a value to make it pass.

when makes a field optional. The field is present only if an earlier integer in the same object equals a value:

{"name": "note", "type": "bytes", "length": 2, "when": {"field": "flags", "equals": "1"}}

Equality is the only test. When it’s false, the field takes no bytes and must be absent from the document.

derive lets the encoder work out a count. An unsigned integer marked {"count_of": "records"} is left out of the editable document, and on encode EByte writes the length of the records array into it. The array’s count must point back at that integer. A count field without derive stays in the document, and encode fails if it doesn’t match the array, rather than guessing which one is right.

Rules That Keep It Bounded

• References only point backwards, to an earlier integer in the same object that has no when. The one forward link allowed is a derived count to its own array.
• Arrays and repeats can nest up to 8 levels deep. A repeat looks ahead at most 16 bytes.
• There are no expressions, no arithmetic between fields, no seeking to an offset, no recursion and no plugins.
• Decoding must use every byte. Data left over after the last field is reported as trailing_data, not ignored.

Limits

These are the default ceilings. A program or a command-line flag can lower any of them, but never raise one.

Sizes are checked before EByte reads or allocates for a field, so a header that claims a 2^64-byte field is rejected after reading just that header.

Not in Version 1

Floating point, bit fields, offsets and seeks, checksums, compression and structural diff between two files. If your format needs one of these, say which: it helps decide what comes next.