Google Summer of Code 2022 - The ENIGMA Team

Week 5: Refactors, bug fixes and Glib, Csv, Zstd layer data support

Contributor: Kartik Shrivastava
Project: Add Tiled compatibility to/from Enigma
Guided by: bjorn, fundies and Josh

Highlights:


A side note

What you can learn from this as a casual reader?

Details

Loading Csv layer data using string streams

TMX files supports storing tile data in CSV format too. Each comma separated value contains global id of the tile. There are total layerWidth*layerHeight such values, arranged in layerHeight number of lines each containing layerWidth number of global ids separated by commas.

The type of data stored is of type unsigned int, which allows us to make a fairly simple csv loader. At first we will load the csv string from child node of layer - data node into a string. Next we iterate over this data and extract global id data string separated by commas.

String containing data is converted into istringstream to be able to iterate line by line. Then each line is converted into stringstream to be able to iterate comma by comma. Finally each sub-string obtained is converted into unsigned int global id. Kindly refer following snippet for more details:



Let's add zstd library support to Enigma - libEGM

Zstd or Zstandard is a fast lossless compression algorithm and TMX files can store tiles using zstd compression, that's why we are going to include this library in enigma.

We will start by first installing/building the library. I'm using Arch linux and it's package manager pacman has a prebuilt zstd package available. Its installed using sudo pacman -Sy zstd command.

Now we have to update couple of files to finally be able to use zstd in libEGM. Following are notable edits, kindly refer to #5270098 commit for complete list of updated files.

Makefile changes: CommandLine/libEGM/Makefile, we will update CXXFLAGS and LDFLAGS like following:

CI changes: Updated pacman installs in "Bootstrap Archlinux" section of azure-pipelines.yml

CMakeLists.txt changes: Add zstd library



Loading zstd compressed layer data

As now we have access to zstd library, so decompressing zstd compressed stream is pretty simple. We first decode the base64 layer - data string and pass it to ZSTD_decompress method of zstd for decompression. Kindly refer to following code snippet for details:



Loading gzip compressed layer data

To decompress gzip compressed data, zlib library can be used and to do so only one small change in current zlib decompress method is required.

Instead of using inflateInit we will use inflateInit2 which allows us to pass extra windowBits and if we pass a special windowBits equal to 32 | MAX_WBITS as its 2nd argument, zlib triggers automatic header detection to properly decompress both zlib abd gzip streams. Refer to following code snip:

So, that's the brief description of feature updates made in Week 5 of this exciting TMX importer. Coming week I'll work mostly on stretch goals like adding hexagonal and isometric map support.