Land of Seylia Windows, Mac, Linux game

Fair warning: this gets kinda technical at times. But, there's pics of what happens when you turn text into images!

As you may have heard on our Twitter feed, we've been working on a save format for Land of Seylia, and in the process, we came across an annoying technical issue: verbose saves were nice and readable, but also a fair bit larger than we were hoping for. Since we can't very well save less data, we have to be more careful in how we use each byte if we want to reduce file size. One obvious way to achieve this is to employ compression, and we figured out some nifty tricks along the way.

First of all, if we're using compression, then we have to decide what kind of compression. The obvious choice, for data that's more-or-less textual, would be to use something like the zip or gzip formats. However, both of those would have required us to figure out how to include those algorithms in our code, either by finding a pre-existing library or by writing it ourselves. Isn't there an easier yet still effective way to do things? Well, as it turns out, there is: we're currently coding in Javascript and HTML5, which means we have access to the HTML canvas element. Moreover, it's possible to query a canvas element and get its image data in PNG format, a format which is - you guess it - essentially a compressed bitmap. Yes, that means that the vast majority of our compression is handled in just two lines of code:

ctx.putImageData(imdata,0,0);
var d = context.toDataURL("image/png");

But wait, you ask, isn't PNG an image format? And aren't saves more like text? Well, yes, on the surface. But deep down, at the level of the hardware, they're both just streams of bits. One works with pixels, the other works with characters. Figure out some way to map pixel values to sets of characters, and you're in business. The first thing we did is to map groups of 3 ASCII characters onto the RGB values of pixels, and lo and behold, we got images like the following:

These images did give us modest size savings, and had a cool side-effect as well: you can actually see what your save data looks like! For instance, the diagonal striations in these images are bottom half (roughly) of these images represent map tile data. Above that, most of the data is related to units.

However, we still weren't happy with the compression results. So, to solve that, we did something tricky: we stored 4 characters of text in the RGB data of each pixel. Normally, this would be storing 4 bytes of data in the space sufficient for 3, which doesn't really work. We got around that issue by realizing that most of our data came from a set of 60 or 70 characters, and that if we limited ourselves to only 64 characters, we could squeeze exactly 4 of those characters into 3 bytes. Furthermore, if we're careful, we can encode any other character as a special combination of the 64 characters in our restricted alphabet. We chose the upper- and lower-case letters, the numbers, and the symbol %, giving us 63 characters to work with, and one extra character for other purposes (in the special-purpose algorithm used in-game, that last character is the tab character, for delimiting purposes). Moreover, Javascript comes with a built-in function, encodeURIComponent, designed to fit most characters into this subset, with everything else given a special combination like "~". The remaining few characters left over are then handled as special cases in our code.

So, what came out of all this trickery? Quite good results, overall. Our file sizes dropped quite a bit, changing our compression rate from "mediocre" to "fairly good", we gained full Unicode compatibility, and the images even became more vibrant:

Both of these images are produced from the same text, the one on the left being 3 characters per pixel, and the one on the right using our tricky technique of fitting 4 characters in one pixel. Of course, this is from near the start of the game, when file sizes are still fairly low. More impressive is the change you see when you use this improved compression technique on a save file containing a map that has been deliberately expanded to large sizes, as the following pair of images demonstrates:

Notice, by the way, that it's obvious that this save is mostly map - there's a huge map block throughout the bottom three-quarters of the image, and the data related to units and such is a relatively narrow band at the top of the image.

Overall, it was a nice experiment, which led to a nice quick-and-easy method to compress our data, as well as a cool visualization. We plan on sharing more actual game news with you soon, and if you're interested in following along with development, have a look at our Twitter feed! Also, if you're interested in using this text-to-PNG trick in your own HTML5 project (or just playing around with various text and images), we'll be posting the source code for it shortly. In the meantime, we leave you with two last samples: a save from our unit-testing save (which was actually played as the game is meant to be played), and the Universal Declaration of Human Rights in 60x62 PNG form.