EByte Testing Starts From Bytes Written Down Before the Code

A codec has a sneaky way of passing its own tests. If the decoder and the encoder share a mistake, say both read a count from the wrong byte, then decoding a file and encoding it again still gives the original bytes back. The round trip looks perfect and the format is wrong. So EByte’s tests don’t start from the code. They start from bytes that were written down by hand before the decoder or encoder existed.

The main example is a 21-byte packet. Its hex, its decoded values, its field-by-field trace and the 25 bytes it should become after one record is added were all specified first, in the repository’s test data. Three more fixtures cover a wide format, an empty one and a stored count. The real decoder and encoder then had to match those bytes and offsets, and a separate set of manual checks reads the fixed layouts without going through EByte at all.

Breaking Files on Purpose

Most of the tests feed EByte something wrong and check what it says back. The 21-byte packet is cut short at every possible length, and each cut has to report truncated_input with the right field, the right start and the exact byte where the data ran out. There are invalid UTF-8 strings, integers one past their range, magic numbers off by one byte, a stored count that doesn’t match its array, and JSON with duplicate keys, nulls or unknown fields.

Then come the hostile cases. A header that claims a field 2^64 - 1 bytes long has to be rejected after EByte has read only the eight bytes of that header, before it reads or allocates anything for the field. The recorded run shows exactly that: 8 bytes read, one read call, limit_exceeded. Limits are tested right at the edge, once with a ceiling that’s exactly enough and once with one that’s a byte or an entry short.

Fuzzing

Six fuzz targets run on every change. One throws random schemas at the compiler. Two attack the decoder and the document loader. Two check that anything the decoder accepts comes back identical, and one of those generates whole schemas with varying integer widths, byte orders, nested arrays, conditions and fixed strings, then checks the integers it reads against Go’s own encoding/binary. The last one fuzzes the command-line options. Each runs for five seconds per change, which finds a lot but proves nothing about the inputs it didn’t try.

Big Files and What They Cost

The chart above comes from ten resource cases, each run in a fresh process. Raw byte fields from empty to the full 16 MiB ceiling all decoded, turned into JSON, loaded back, encoded and compared, and every one came back identical. The JSON is about twice the size of the binary, because each byte becomes two hex characters. A 1 MiB string of NUL characters is the worst case EByte has on file: it became 6 MB of JSON, since every NUL turns into a six-character escape.

The 16 MiB case took about 0.7 seconds for the whole pipeline on the test machine, in a single sample. That’s an observation, not a benchmark. EByte keeps the whole document in memory, so a large file costs several times its size along the way. The results file records the allocations so the cost is visible rather than guessed.

The Command Line and the Disk

The compiled ebyte binary is tested as a separate process too: round trips through standard input and output, failed traces, every exit code, a broken output pipe, and Ctrl-C while it’s waiting on input. Writing a file has its own tests. EByte writes to a private temporary file and links it into place only if the name is still free, and the tests break that at every step: failed creates, syncs, closes and links, a cleanup that fails after the link, a destination that appears while the file is being written. None of them may replace an existing file.

Where the Results Live

All of this runs in GitHub Actions on Linux and macOS on every push, through one script that saves its output to test/results/latest in the repository. A release can only be built from a commit that passed. Before 0.1.0 went public, its Linux, macOS and Windows archives were unpacked and run on their own systems, and after it went public every download was fetched again and a downloaded binary was run. That evidence is kept next to the code in its own folder for the release.