Bliss format byte controversy

This article refers to the arguments and problems that resulted from a series of events related to Bliss' handling of metadata and the format byte/extra byte in track files and how this is being solved.

Historical background

Around 2006, Cas was developing a track editor call Castunts for DOS that was to become the inspiration for Cas-Stunts 2.0, which later evolved to Bliss. Castunts included just a handful of the features present in Bliss, but one that it did not have was the ability to store metadata in tracks. Cas knew it was possible to embed extra information by overlaying these files, so it was in his plans to do this. He created a few tracks with his editor that he touched manually with a binary editor to add a track title and author's name to the files.

The fields were placed right after the 1802 bytes if the raw track files, but because this overlay could be lost should the track be edited with the internal editor, Cas also modified the last byte of the raw files, that's normally a zero, to make it 149, making this value an ID for what his editor would become. Castunts 1.0 was never finished and never had the ability to insert metadata in track files.

In May 2015, Cas has agreed to post a track to ZakStunts for the race of that month. He chose to submit 4:00am (4AM.TRK), a track he had made near 2006 and that had already been raced in Paleke's World Stunts Championship, but never free-style. It turned out that this happened to be one of the few track files that had an overlay from the times of Castunts 1.0. The track got uploaded to ZakStunts without a problem, but when racers began trying to post their first replays, they weren't able to. Nobody knew what was going on until Dreadnaut saw the file was longer than expected. Then Cas remembered and explained about the overlay; the track file was truncated and re-uploaded. Problem solved.

Technical details on metadata and the extra byte

A track overlay of up to 12000 bytes can be added to track files without conflict with Stunts. There is, however, a conflict with whatever tool that decides to take for granted that the track will be 1802 bytes long and won't work otherwise. This is the case of Track Blaster and, indirectly, of ZakStunts. An overlay was preferred to a parallel file by Cas to store metadata because parallel files tend to get lost along the way and require that one renames them if the track file is renamed.

To recognise that a track was made with Castunts/Bliss, relying on an overlay was a little risky. This is why Cas chose the extra byte, which is unused by Stunts or any tool, yet kept the same after editing. So when Cas-Stunts 2.0 was presented, it did not store a zero. Cas thought that Castunts 1.0 was history, so instead of using 149, he made Cas-Stunts 2.0 use a 150 for new tracks. For some time, nobody noticed and there was no mention on the matter.

The metadata argument

At some point in the development, the time came to add the feature of metadata to the editor. Cas wrote a forum topic asking the folks what they tought would be the best method and format for metadata. To his surprise, nobody seemed very interested in metadata and some even disliked the idea pretty much. In particular, the overlay idea was strongly rejected and the use of a binary format was also frowned upon. To keep the community happy, but also be able to accomplish his wish, Cas created two formats: one that stored metadata in an overlay and one that would place the information in a separate parallel file. Both formats were to be binary. When people were to submit tracks to ZakStunts or just not want the overlay, they could use the split format.

At this time, Cas made a great mistake. He thought it made sense to have two format codes and store them in the extra byte. And besides, he dropped 150 and now issued 151 for the overlay format and 152 for the split format. This turned the extra byte into a format byte, which could change for the same track depending on how it was stored. To make things worse, a bug in the code made the values 2 and 3 also written to the extra byte sometimes!

The conflict

The problems with extra byte proliferation are many: first, sometimes comparing two track files with the same track may fail if all of the 1802 bytes are compared. Besides, when testing that a replay has been made with a certain track, the extra byte is passed to the replay, so the result could also be wrong.

This exact scenario started to come true and caused Dreadnaut a nightmare[1] trying to organise track and replay files. By that time, Cas had already eliminated the bug and the format byte had been dropped to a fixed value, but of course, it couldn't be the same, so now 153 was being used. Yet, the files that had been produced during the previous months had accumulated with all flavours of the extra byte.

Resolution

As of March 2019, the current public version of Bliss (2.5.4) does not modify the extra byte in tracks being edited. It does, for new tracks, still use the value 153. The community would prefer a zero, but at least this being a fixed number and only being applied to new tracks should not cause havoc. It is planned on future versions to switch back to zero

Currently, Bliss allows to save the track file as "raw", besides combined/overlay and split formats. Both metadata containing formats store it identically, in binary form. However, there is a plan to change that to text mode for the split format in future versions. The metadata has not caused any problem, as has the extra byte and the problems cause by the extra bytes were due to the mistakes described above. Had it always been just one fixed number, there wouldn't have been any issue.