Bliss format byte controversy

This article refers to the arguments and problems that resulted from a series of events related to Bliss' handling of metadata and the format byte/extra byte in track files and how this is being solved.

Historical background

Around 2006, Cas was developing a track editor call Castunts for DOS that was to become the inspiration for Cas-Stunts 2.0, which later evolved to Bliss. Castunts included just a handful of the features present in Bliss, but one that it did not have was the ability to store metadata in tracks. Cas knew it was possible to embed extra information by overlaying these files, so it was in his plans to do this. He created a few tracks with his editor that he touched manually with a binary editor to add a track title and author's name to the files.

The fields were placed right after the 1802 bytes if the raw track files, but because this overlay could be lost should the track be edited with the internal editor, Cas also modified the last byte of the raw files, that's normally a zero, to make it 149, making this value an ID for what his editor would become. Castunts 1.0 was never finished and never had the ability to insert metadata in track files.

In May 2015, Cas had agreed to post a track to ZakStunts for the race of that month. He chose to submit 4:00am (4AM.TRK), a track he had made near 2006 and that had already been raced in Paleke's World Stunts Championship, but never free-style. It turned out that this happened to be one of the few track files that had an overlay from the times of Castunts 1.0. The track got uploaded to ZakStunts without a problem, but when racers began trying to post their first replays, they weren't able to. Nobody knew what was going on until Dreadnaut saw the file was longer than expected. Then Cas remembered and explained about the overlay; the track file was truncated and re-uploaded. Problem solved.

Technical details on metadata and the extra byte

A track overlay of up to 12000 bytes can be added to track files without conflict with Stunts. There is, however, a conflict with whatever tool that decides to take for granted that the track will be 1802 bytes long and won't work otherwise. This is the case of Track Blaster and, indirectly, of ZakStunts. An overlay was preferred to a parallel file by Cas to store metadata because parallel files tend to get lost along the way and require that one renames them if the track file is renamed.

To recognise that a track was made with Castunts/Bliss, relying on an overlay was a little risky. This is why Cas chose the extra byte, which is unused by Stunts or any tool, yet kept the same after editing. So when Cas-Stunts 2.0 was presented, it did not store a zero. Cas thought that Castunts 1.0 was history, so instead of using 149, he made Cas-Stunts 2.0 use a 150 for new tracks. For some time, nobody noticed and there was no mention on the matter.

The metadata argument

At some point in the development, the time came to add the feature of metadata to the editor. Cas wrote a forum topic asking the folks what they thought would be the best method and format for metadata. To his surprise, nobody seemed very interested in metadata and some even disliked the idea pretty much. In particular, the overlay idea was strongly rejected and the use of a binary format was also frowned upon. To keep the community happy, but also be able to accomplish his wish, Cas created two formats: one that stored metadata in an overlay and one that would place the information in a separate parallel file. Both formats were to be binary. When people were to submit tracks to ZakStunts or just not want the overlay, they could use the split format.

At this time, Cas made a great mistake. He thought it made sense to have two format codes and store them in the extra byte. And besides, he dropped 150 and now issued 151 for the overlay format and 152 for the split format. This turned the extra byte into a format byte, which could change for the same track depending on how it was stored. To make things worse, a bug in the code made the values 2 and 3 also written to the extra byte sometimes!

The conflict

The problems with extra byte proliferation are many: first, sometimes comparing two track files with the same track may fail if all of the 1802 bytes are compared. Besides, when testing that a replay has been made with a certain track, the extra byte is passed to the replay, so the result could also be wrong.

This exact scenario started to come true and caused Dreadnaut a nightmare[1] trying to organise track and replay files. By that time, Cas had already eliminated the bug and the format byte had been dropped to a fixed value, but of course, it couldn't be the same, so now 153 was being used. Yet, the files that had been produced during the previous months had accumulated with all flavours of the extra byte.

Resolution

As of March 2019, the current public version of Bliss (2.5.4) does not modify the extra byte in tracks being edited. It does, for new tracks, still use the value 153. The community would prefer a zero, but at least this being a fixed number and only being applied to new tracks should not cause havoc. It is planned on future versions to switch back to zero

Currently, Bliss allows to save the track file as "raw", besides combined/overlay and split formats. Both metadata containing formats store it identically, in binary form and starting version 2.5.5, there is a text-based split metadata format. The metadata has not caused any problem, as has the extra byte and the problems cause by the extra bytes were due to the mistakes described above. Had it always been just one fixed number, there wouldn't have been any issue.

Potential future problems

At some point shortly after the end of the conflict, ZakStunts was patched so that track uploading would automatically strip anything after the first 1802 bytes and the format-byte was forced to zero. Or perhaps, this is being done manually, since ZakStunts does not allow for regular users to directly post tracks, so it isn't apparent to the racer. This creates the potential for the opposite problem, since tracks made originally with Bliss will have a 153 and if racers have the original track file, they may try to post replays that contain this byte as well, but so far, this has not occured.

Race For Kicks does not rip track files when uploaded and some regular racers are allowed to post tracks. If the posted track did not contain metadata, it is created when the track is posted and the poster gives it a title, but the format-byte is not touched. When replays are posted, they are verified against the first 1801 bytes, not 1802, because Cas interprets that while the format byte is part of the file, it's not part of the track. This passive handling is safer and in Race For Kicks, this is more necessary as the tournament administrator does not have to be an intermediate when posting tracks.