A new data compressor called ZX0

[XoRRoX]

@Einar Saukas Is it somehow possible to use zx0 for data in a column x row square? I'd like to use it for gfx that I'd like to unpack to a certain screen location without touching the other gfx on the screen.

[Einar Saukas]

No. You have to decompress it to a continuous memory area first, before drawing it to the screen.

[XoRRoX]

Too bad, would make it more flexible to use for certain gfx. But I'll try to think of something else to compress them.
Thank you.

[zara6502]

This is how ZX0 uses a single bit to distinguish between 3 different block types:

• Immediately after a literal block, you cannot have another literal block (otherwise you would have merged both into a single block). Therefore after a literal block, bit 0 means "copy from last offset" and bit 1 means "copy from new offset".
• Immediately after any copy block (either from last or new offset), you cannot have a "copy from last offset" block (otherwise you would have merged both into a single block). Therefore after after any copy block, bit 0 means "literal" and bit 1 means "copy from new offset".

Greetings again. It was time to return to compression again, I reread the correspondence, reread the documentation for ZX0 again and could not recreate this part of the compressor operation correctly.
For example, there is data

Code: Select all

ABCDEABCDABCABCABC

Code: Select all

1. len(5) (literals ABCDE, 0 we do not set it as this is the beginning of the stream and the literals will be the first)
2. 1 + len(4) + offset(5) (for ABCD)
3. 1 + len(3) + offset(4) (for ABC)
4. 1 + len(3) + offset(3) (for ABC)
5. 0 + len(3) (last offset for ABC)

In this case, how do we understand in line 5 of the code that we are doing "copy from last offset" and not "literal"?

And the second question is, how often is "copy from last offset" used? I practically do not receive such situations in manual mode for test data, For example, for a test phrase:

Code: Select all

I am Sam

Sam I am

That Sam-I-am!
That Sam-I-am!
I do not like
that Sam-I-am!

Do you like green eggs and ham?

I do not like them, Sam-I-am.
I do not like green eggs and ham.

I get (there may be errors, but in general the meaning does not change) such a sequence of offsets:
3, 1, 4, 2, 7, 5, 16, 12, 24, 2, 33, 16, 56, 3, 36, 61, 47, 77, (31, 31), 7, 8, 110, 91, 112, 68, 37, 67, 115, 99, 68, 35
as you can see, there was only one situation (31, 31) where the previous offset coincided. Maybe it's just a "free" opportunity to cut by a couple of bytes?

[Einar Saukas]

Code: Select all

1. len(5) (literals ABCDE, 0 we do not set it as this is the beginning of the stream and the literals will be the first)
2. 1 + len(4) + offset(5) (for ABCD)
3. 1 + len(3) + offset(4) (for ABC)
4. 1 + len(3) + offset(3) (for ABC)
5. 0 + len(3) (last offset for ABC)

In this case, how do we understand in line 5 of the code that we are doing "copy from last offset" and not "literal"?

This situation will never happen since it's more efficient this way:

Code: Select all

1. len(5) (literals ABCDE, 0 we do not set it as this is the beginning of the stream and the literals will be the first)
2. 1 + len(4) + offset(5) (for ABCD)
3. 1 + len(3) + offset(4) (for ABC)
4. 1 + len(6) + offset(3) (for ABC)

[Einar Saukas]

And the second question is, how often is "copy from last offset" used?

Often enough to be worth it.

Anecdotal evidence otherwise doesn't mean much...

[C.Born]

Hi
how big is the change that zx0 will compress at a z80 machine?
zcc DOES compile
zcc +zx -vn -zorg0x5f00 zx0.c optimize.c compress.c memory.c -ozx0 -DAMALLOC -lndos -create-app

BUT zx0 uses ARG[] in main
if zx0.c was written for a zx memory, will the other parts compres?
# JUST THEORY PSEUDO CODE

Code: Select all

BASIC:
 CLEAR 24319: LET zx0=24320
 LOAD "zx0" CODE zx0
 LET i$="input": LET o$="screen.zx0"
 LET f=0: LET c=0: LET b=0: LET q=0
 DEF FN u(f,c,b,q,i$,o$)=USR zx0
 RANDOMIZE FN u(f,c,b,q,i$,o$)  (= RANDOMIZE USR zx0 )
 SAVE o$ CODE zx0+leng, leng

Code: Select all

zx0start.asm:

DEF_ADR      equ  23563           ; sysvar DEF_ADR, points to needed value's

mode         defb  0
image_start  defw  0
zx0_start    defw  0
length       defw  0

startdef     ld hl,(DEF_ADR)      ; hl = actual RAM location in BASIC listing at position off the DEF FN
             ld ix,(mode)
             ld bc,4
             add hl,bc            ; adres+4 = var f
             srl (hl)
             sra (ix+0)
             ld c,8
             add hl,bc            ; adres+4+8 = var c
             srl (hl)
             sra (ix+0)
             add hl,bc            ; adres+4+8+8 = var b
             srl (hl)
             sra (ix+0)
             add hl,bc            ; adres+4+8+8+8 = var q
             srl (hl)
             sra (ix+0)
               ; (ix=0)= %0000fcbq

; input file OR location
             add hl,bc            ; adres+4+8+8+8+8
                                  ; hl points to STRING or adres
             ld (image_start),hl
; whats EOF token ??
             ld a,token
             CPIR 

             inc hl                 ; at least 1 byte above source
             ld (zx0_start),hl ; destiny of compressed file


...  the ZX0.C routine's


             ld bc,length     ; return with compressed length
             ret              ;

does it have a chance to work and will my zx make a compressed file ?

[ParadigmShifter]

That seems mad compression is way slower than decompression you should do it on a fast PC. I imagine it uses a lot of memory allocations as well? Also you can't compress a big file if you need to do that since you'll run out of memory.

You're not developing only on the ZX are you without using an emulator??? That's a bit bonkers too, but very hardcore, so kudos if you are It's fine to be bonkers though I understand.

Even speccy developers had PCs (or something with a disk drive which they developed on so they didn't lose all their source if the target machine crashed... it's not cheating or anything).

[C.Born]

nah, it aint cheating when ZX can do it

btw, i realized, if zcc is used its better to avoid those IX and IY..

Code: Select all

startdef     ld hl,(DEF_ADR)      ; hl = actual RAM location in BASIC listing at position off the DEF FN
             ld de,(mode)
             ld bc,4
             add hl,bc            ; adres+4 = var f
             rrc (hl)
             rlca
             ld c,8
             add hl,bc            ; adres+4+8 = var c
             rrc (hl)
             rlca
             add hl,bc            ; adres+4+8+8 = var b
             rrc (hl)
             rlca
             add hl,bc            ; adres+4+8+8+8 = var q
             rrc (hl)
             rlca
             ld (de),a            ; a = %0000fcbq

so
i spend many times 5 minutes waiting for a tape loading error, like 500 - 1000 times al together ??
why do you THINK i do mind waiting 28 seconds for a compression that will make me wait 1.2 minutes next time ??
without the loading error, but thats the tape deck ...

[ParadigmShifter]

I understand

As I said, it's fine to be bonkers As you were good Sir

[zara6502]

Often enough to be worth it.

Anecdotal evidence otherwise doesn't mean much...

I hope you are not tired of me
I have completed my own implementation, using a custom implementation of graph traversal, which I create by processing repetitions. But, since I didn't understand exactly how you use "last offset", I couldn't use such a technique myself. In general, I managed to make a working archiver, but it inevitably lags behind in compression ratio by ~ 10-15% on large files. I'll try some more analytics, but I'm not sure that it will make a big difference. In the meantime, I've redone your optimize() implementation in C# (which is what I'm writing in) and do some reverse engineering. I'm not a professional programmer, so it's hard for me to do this, but it was generally nice to see that conceptually I wrote my implementation in a similar style. But the main thing is that I understood the basic principle of the appearance of "last offset" blocks. They appear, for example, in a situation where there is the phrase "AAABBBCC" and then there is the phrase "AAADDDCC", that is, they have common parts in comparable places and some different parts, so the "last offset" block will work in matching parts.

[Einar Saukas]

They appear, for example, in a situation where there is the phrase "AAABBBCC" and then there is the phrase "AAADDDCC", that is, they have common parts in comparable places and some different parts, so the "last offset" block will work in matching parts.

Exactly!