.tnrd

The first format did the obvious thing. The first line of the file is a JSON header: the format name TNRD_V1, the game protocol, the track, the session type and when recording started. After that, every packet the parser understood is written as one more line of JSON, in the order it arrived: telemetry, lap data, car status, damage, motion and so on.

The whole thing goes through a single gzip stream. Every few seconds the recorder flushes it, so if the app crashes mid-session the file is still readable up to the last flush.

Playback is where it hurts. gzip can only be read from the start, so opening a recording means decompressing the entire file into a temporary file on disk, then scanning that to build an index before anything appears on screen.

Same idea, better compressor. The rows are unchanged, but the stream is now Zstandard at level 3 with a checksum on the frame, which compresses smaller and decompresses much faster than gzip. The header gains a compression: "zstd" field.

It's still one continuous stream, so it still has to be decompressed from the beginning, in full, into a temporary file before playback.

The Analysis page was getting a lap delta chart: how far ahead or behind one lap is against another at every point on track. Working that out needs the length of the circuit, so V3 adds track_length_m to the header. The delta chart only switches on for V3 files and newer.

That's the only change to the format. It's still the same single Zstandard stream as V2, so it still has to be decompressed in full before playback, and a flashback that goes further back than the recorder's in-memory buffer can still force the whole compressed file to be rewritten.

Those costs are what kept V3 from ever shipping. The story is below.

V4 throws away the single stream. The rule is simple: everything needed to find data stays uncompressed, and everything large is compressed in small independent pieces.

The data is cut into chunks. Each chunk holds one family of rows for one lap, for example lap 17's telemetry or lap 17's damage, and is its own Zstandard frame, so it can be decompressed without touching anything else. There are 14 families, including telemetry, car status, damage, lap data, motion, positions, participants, race events and tyre sets.

Two uncompressed tables at the end of the file describe it all. The lap table lists every lap with its start time, end time and lap time. The chunk directory gives every chunk's lap, family, position in the file, sizes, row count and checksum. Opening a file is now a handful of small reads and no decompression. Each page then loads only the laps and families it actually draws.

Chunks are written as you drive, tables last, and the fixed header is patched with their positions when recording stops. A lap stays in memory for 30 seconds after it ends in case of a flashback. Every so often a checkpoint footer is appended, and after a crash the reader scans backwards for the last good one. A flashback into data already on disk writes replacement chunks and points the directory at them, instead of rewriting the file.

V4's directory said where each chunk was, but not when it was. To find which chunk covered a moment in the session, the reader had to decompress chunks and read the timestamp of every row. The recorder knew all of this while writing, and simply threw it away.

V5 keeps it. Each directory entry doubles to 96 bytes and now carries the chunk's first and last session time, its lap distance range, and a pointer into a row index. The row index lists where each row sits inside the decompressed chunk: every row for rarely-changing state such as damage or tyre sets, and one entry per 64 rows for high-rate data such as telemetry. It's written once, when recording finishes.

Flashbacks changed too. Instead of replacement chunks, a flashback now appends a small branch entry: the wall-clock time it happened and the session time it rewound to. Newer branches hide older data from their rewind point onward, so nothing already written is ever decompressed, filtered or recompressed.

Or ... that's what's supposed to happen. Except, I set the threshold too high. And what that caused is, data never got to rewind because the maximum flashback threshold of the game was lower than what was needed to trigger a branched flashback. If it was below the threshold, it did a in memory replacement and didn't need a branch.

Which means ... I wrote and designed the entire system for nothing. The code is there but it has never been tested or triggered.

The result: opening a file decompresses zero chunks, seeking jumps straight to the right one, and finding the latest state at a moment decompresses exactly one.

V5 had been public for about a day when it became clear it couldn't handle comparing your laps against other drivers. So V6 starts over with a different question: what does each chart actually need?

Earlier versions kept only a little data for the other cars, enough for the Session and Standings pages. V6 records full telemetry for every car on the grid, and the file is split three ways: by driver, then by lap, then by data type. There are 24 data types, each matching something a chart or card shows: speed, RPM, gear, throttle, brake, steering, aero, tyre surface and inner temperature, brake temperature, engine temperature, tyre wear, tyre state, damage, fuel, ERS store, harvest and deployment, engine power, brake bias, G-force, ride height, position, and lap timing. Alonso's speed on lap 5 is exactly one chunk.

Each driver has a small header with their name, team, number and whether they've set their telemetry to restricted, plus a lap table. Listing drivers, showing restrictions and drawing lap times needs no decompression. Things that belong to the whole session, such as weather, race events and participant changes, are stored once rather than 24 times.