What is the best routine for graphics compression?

The place for codemasters or beginners to talk about programming any language for the Spectrum.
alexeym
Berk
Posts: 3
Joined: Sun Sep 15, 2019 10:37 am

What is the best routine for graphics compression?

Post by alexeym » Sun Sep 15, 2019 10:39 am

Hi. I'm looking for a Mac or Windows tool to compress ZX Spectrum graphics (not only full screen .scr images but also sprites) and routine which would decompress the data and put the decompressed sprite (3x2 chars or 24x16 pixels) to x,y coordinates (1-32, 1-24). I wonder what is the best routine for that?
0 x

User avatar
djnzx48
Manic Miner
Posts: 540
Joined: Wed Dec 06, 2017 2:13 am
Location: New Zealand

Re: What is the best routine for graphics compression?

Post by djnzx48 » Sun Sep 15, 2019 10:55 am

1 x

alexeym
Berk
Posts: 3
Joined: Sun Sep 15, 2019 10:37 am

Re: What is the best routine for graphics compression?

Post by alexeym » Sun Sep 15, 2019 11:08 am

Awesome links. Thank you so much!
0 x

Ralf
Dynamite Dan
Posts: 1287
Joined: Mon Nov 13, 2017 11:59 am
Location: Poland

Re: What is the best routine for graphics compression?

Post by Ralf » Sun Sep 15, 2019 11:27 am

If you ask me about drawing sprites, then the best compression is no compression at all ;)

Any data compression will slow down your drawing routine. And speed is in most cases critical. Make your sprite drawing just a little bit slower
and you'll discover that everything doesn't fit into a frame, that your framerate drop from 25 fps to 17 fps for example, making your game 50% slower.

Both speed and memory are issues on Zx Spectrum but I would say speed is a bigger issue.
0 x

alexeym
Berk
Posts: 3
Joined: Sun Sep 15, 2019 10:37 am

Re: What is the best routine for graphics compression?

Post by alexeym » Mon Sep 16, 2019 10:39 am

The game is kind of static (no animation), so that's not a problem. :)
0 x

User avatar
Einar Saukas
Manic Miner
Posts: 942
Joined: Wed Nov 15, 2017 2:48 pm

Re: What is the best routine for graphics compression?

Post by Einar Saukas » Mon Sep 16, 2019 4:06 pm

Even if performance is not a problem, compressing individual sprites won't provide very good results. Your sprites will compress better if you divide your sprites into small groups (preferably putting more similar sprites together in the same group), then compress each group. Later during the game, you would decompress one of the groups into a temporary area, before copying the required sprites to the screen. Notice it will take some experimentation to find out the best trade-off between group size and compression ratio that works best for you.

For compressing small blocks, you should get good results using ZX7. There are more sophisticated data compressors available for the Spectrum, but they are typically focused on compressing larger data blocks with several Kb each. In comparison, ZX7 is focused on compressing smaller blocks (although my suggestion above still applies!), also it decompresser faster, it doesn't require additional memory, and the decompressor size is smaller than others.

To compress full screen .scr images, I'm sure you will get the best results using RCS+ZX7 combined together. Moreover the "Smart" version of the decompressor can both decompress a full .SCR (compressed with RCS+ZX7) directly to screen, and also a block of sprites (compressed with ZX7) to the temporary area I mentioned.
0 x

User avatar
Einar Saukas
Manic Miner
Posts: 942
Joined: Wed Nov 15, 2017 2:48 pm

Re: What is the best routine for graphics compression?

Post by Einar Saukas » Mon Sep 16, 2019 6:26 pm

djnzx48 wrote:
Sun Sep 15, 2019 10:55 am
You could try these articles:

http://hype.retroscene.org/blog/dev/740.html
http://hype.retroscene.org/blog/933.html
I remember reading the first article a couple years ago, but I didn't know there was a follow-up. These articles are awesome!!!

Both articles are very well thought, well researched, and well written. It was a real pleasure to read an in-depth analysis like this.

My only (constructive) criticism is that I don't think these scenarios are practical enough for the Spectrum. For instance, the first scenario consists of 8 files extracted from the "Calgary" test. However most of these are text files larger than 38Kb. It would never make sense to decompress such a large block of text into the Spectrum memory at once. Especially considering that there's not even enough room to fit both the compressed block and the decompressing memory area in 48Kb! Even in a very large text adventure for the Spectrum 128K, a text block to be compressed would never exceed 16Kb. As a matter of fact, in practice text blocks would be typically much smaller, because whenever a Spectrum program contains a lot of text, it's better to use simpler text compression strategies and avoid data block compression altogether...

However I'm not implying the articles are flawed. I'm just pointing out that the article is oriented towards compression of larger data blocks. The selected set of files in the article contains 77 files, with total size of 1,233,995 bytes, therefore an average of 16,026 bytes per file. But I believe the most typical usage of data compression in Spectrum programs is to compress relatively small data blocks, from a few hundred bytes to a few Kb. At least that's what I always needed in my own Spectrum programs, and that's ZX7 main strength. My point is, the article conclusions make sense, but they don't necessarily apply to compressing smaller blocks of data.

Another point is that data compression specifically for Spectrum screens, using RCS+ZX7, was not measured. The "Graphics" scenario from the article consists of 30 files, including 19 Spectrum screens. Using only ZX7, these 19 Spectrum screens compress to 93,969 bytes and the remaining 11 other files to 78,171 bytes, a total of 172,140 bytes (59.37%). But using RCS+ZX7 for these 19 Spectrum screens instead, they compress to 81,849 bytes, reducing the total to 160,020 bytes (55.19%). That's better than all other results in the article! Although I understand that including RCS was outside the scope of the articles, that's perfectly fine. :)
0 x

User avatar
Sokurah
Microbot
Posts: 114
Joined: Tue Nov 14, 2017 10:38 am

Re: What is the best routine for graphics compression?

Post by Sokurah » Mon Sep 16, 2019 10:01 pm

In Vallation I'm only storing enemies as 16x16 frames, then when you change screen it'll take a few of those and shift them into 24x16 images before the next screen is shown. This way I can pack in more enemies. But as mentioned before, actually compressing the spritedata doesn't seem like an option that would yield much space. And certainly not something you'd want to do for something drawn on the fly ... if you know what I mean.
0 x
Website: Tardis Remakes / Mostly remakes of Arcade and ZX Spectrum games.
My games for the Spectrum: Dingo, The Speccies, The Speccies 2, Vallation & Sqij.
Twitter: Sokurah

catmeows
Berk
Posts: 36
Joined: Tue May 28, 2019 11:02 am

Re: What is the best routine for graphics compression?

Post by catmeows » Wed Sep 18, 2019 3:03 pm

Einar Saukas wrote:As a matter of fact, in practice text blocks would be typically much smaller, because whenever a Spectrum program contains a lot of text, it's better to use simpler text compression strategies and avoid data block compression altogether...
I agree. Typical text message would be just few lines of text long. No LZ method can compete dictionary in such case. I have really good experience with byte pair encoding that can achieve about 50% compression for english.

About ZX7 - I quite agree with the article. The match/literal flag bit is a mistake and elias encoding (probably) too. When you look ať modern byte based LZs like LZ4, LZ5, LZF they are much faster and not much worse or same as ZX7 and their offset size is tuned for rather big files. A modern "8bit" byte based LZ could probably offer more than ZX 7.
0 x

Joefish
Manic Miner
Posts: 622
Joined: Tue Nov 14, 2017 10:26 am

Re: What is the best routine for graphics compression?

Post by Joefish » Wed Sep 18, 2019 4:47 pm

This is the problem with data compression on systems with not a lot memory - you need to compress a large amount to get good efficiency, but also that you can't unpack bits of it at a time because it needs to be able to refer back through previously extracted data to point to repititions, which is where the savings come from.

One option you might consider is compressing up to 6K of data at a time. You can then unpack this to screen memory (with the attributes blanked out to hide it), copy out the sprite or level data you need, then wipe the screen again.

And as Sokurah says, if your sprite data is pre-shifted, undo the pre-shifting before you compress as this will lead to far more repeated bytes and better compression. Up to you whether that means your sprite frames shrink from 24x16 to 16x16, or whether you leave them as 24x16. But you will need an efficient re-shifter routine to set them back after they're extracted, or the saving won't be worth it.

And as Einar says, a routine to re-order screen data to line-by-line can improve screen compression, as it makes the repeated parts of a screen closer together in memory, so the compressor is then more efficient and picking up on repetitions.
0 x

Post Reply