A new data compressor called ZX0
Re: A new data compressor called ZX0
@Einar Saukas Is it somehow possible to use zx0 for data in a column x row square? I'd like to use it for gfx that I'd like to unpack to a certain screen location without touching the other gfx on the screen.
- Einar Saukas
- Bugaboo
- Posts: 3169
- Joined: Wed Nov 15, 2017 2:48 pm
Re: A new data compressor called ZX0
No. You have to decompress it to a continuous memory area first, before drawing it to the screen.
Re: A new data compressor called ZX0
Too bad, would make it more flexible to use for certain gfx. But I'll try to think of something else to compress them.
Thank you.
Thank you.
Re: A new data compressor called ZX0
Greetings again. It was time to return to compression again, I reread the correspondence, reread the documentation for ZX0 again and could not recreate this part of the compressor operation correctly.Einar Saukas wrote: ↑Thu Nov 09, 2023 1:09 pm This is how ZX0 uses a single bit to distinguish between 3 different block types:
- Immediately after a literal block, you cannot have another literal block (otherwise you would have merged both into a single block). Therefore after a literal block, bit 0 means "copy from last offset" and bit 1 means "copy from new offset".
- Immediately after any copy block (either from last or new offset), you cannot have a "copy from last offset" block (otherwise you would have merged both into a single block). Therefore after after any copy block, bit 0 means "literal" and bit 1 means "copy from new offset".
For example, there is data
Code: Select all
ABCDEABCDABCABCABC
Code: Select all
1. len(5) (literals ABCDE, 0 we do not set it as this is the beginning of the stream and the literals will be the first)
2. 1 + len(4) + offset(5) (for ABCD)
3. 1 + len(3) + offset(4) (for ABC)
4. 1 + len(3) + offset(3) (for ABC)
5. 0 + len(3) (last offset for ABC)
And the second question is, how often is "copy from last offset" used? I practically do not receive such situations in manual mode for test data, For example, for a test phrase:
Code: Select all
I am Sam
Sam I am
That Sam-I-am!
That Sam-I-am!
I do not like
that Sam-I-am!
Do you like green eggs and ham?
I do not like them, Sam-I-am.
I do not like green eggs and ham.
3, 1, 4, 2, 7, 5, 16, 12, 24, 2, 33, 16, 56, 3, 36, 61, 47, 77, (31, 31), 7, 8, 110, 91, 112, 68, 37, 67, 115, 99, 68, 35
as you can see, there was only one situation (31, 31) where the previous offset coincided. Maybe it's just a "free" opportunity to cut by a couple of bytes?
- Einar Saukas
- Bugaboo
- Posts: 3169
- Joined: Wed Nov 15, 2017 2:48 pm
Re: A new data compressor called ZX0
This situation will never happen since it's more efficient this way:zara6502 wrote: ↑Fri Mar 22, 2024 7:30 amIn this case, how do we understand in line 5 of the code that we are doing "copy from last offset" and not "literal"?Code: Select all
1. len(5) (literals ABCDE, 0 we do not set it as this is the beginning of the stream and the literals will be the first) 2. 1 + len(4) + offset(5) (for ABCD) 3. 1 + len(3) + offset(4) (for ABC) 4. 1 + len(3) + offset(3) (for ABC) 5. 0 + len(3) (last offset for ABC)
Code: Select all
1. len(5) (literals ABCDE, 0 we do not set it as this is the beginning of the stream and the literals will be the first)
2. 1 + len(4) + offset(5) (for ABCD)
3. 1 + len(3) + offset(4) (for ABC)
4. 1 + len(6) + offset(3) (for ABC)
- Einar Saukas
- Bugaboo
- Posts: 3169
- Joined: Wed Nov 15, 2017 2:48 pm
Re: A new data compressor called ZX0
Often enough to be worth it.
Anecdotal evidence otherwise doesn't mean much...
Re: A new data compressor called ZX0
Hi
how big is the change that zx0 will compress at a z80 machine?
zcc DOES compile
zcc +zx -vn -zorg0x5f00 zx0.c optimize.c compress.c memory.c -ozx0 -DAMALLOC -lndos -create-app
BUT zx0 uses ARG[] in main
if zx0.c was written for a zx memory, will the other parts compres?
# JUST THEORY PSEUDO CODE
does it have a chance to work and will my zx make a compressed file ?
how big is the change that zx0 will compress at a z80 machine?
zcc DOES compile
zcc +zx -vn -zorg0x5f00 zx0.c optimize.c compress.c memory.c -ozx0 -DAMALLOC -lndos -create-app
BUT zx0 uses ARG[] in main
if zx0.c was written for a zx memory, will the other parts compres?
# JUST THEORY PSEUDO CODE
Code: Select all
BASIC:
CLEAR 24319: LET zx0=24320
LOAD "zx0" CODE zx0
LET i$="input": LET o$="screen.zx0"
LET f=0: LET c=0: LET b=0: LET q=0
DEF FN u(f,c,b,q,i$,o$)=USR zx0
RANDOMIZE FN u(f,c,b,q,i$,o$) (= RANDOMIZE USR zx0 )
SAVE o$ CODE zx0+leng, leng
Code: Select all
zx0start.asm:
DEF_ADR equ 23563 ; sysvar DEF_ADR, points to needed value's
mode defb 0
image_start defw 0
zx0_start defw 0
length defw 0
startdef ld hl,(DEF_ADR) ; hl = actual RAM location in BASIC listing at position off the DEF FN
ld ix,(mode)
ld bc,4
add hl,bc ; adres+4 = var f
srl (hl)
sra (ix+0)
ld c,8
add hl,bc ; adres+4+8 = var c
srl (hl)
sra (ix+0)
add hl,bc ; adres+4+8+8 = var b
srl (hl)
sra (ix+0)
add hl,bc ; adres+4+8+8+8 = var q
srl (hl)
sra (ix+0)
; (ix=0)= %0000fcbq
; input file OR location
add hl,bc ; adres+4+8+8+8+8
; hl points to STRING or adres
ld (image_start),hl
; whats EOF token ??
ld a,token
CPIR
inc hl ; at least 1 byte above source
ld (zx0_start),hl ; destiny of compressed file
... the ZX0.C routine's
ld bc,length ; return with compressed length
ret ;
- ParadigmShifter
- Manic Miner
- Posts: 765
- Joined: Sat Sep 09, 2023 4:55 am
Re: A new data compressor called ZX0
That seems mad compression is way slower than decompression you should do it on a fast PC. I imagine it uses a lot of memory allocations as well? Also you can't compress a big file if you need to do that since you'll run out of memory.
You're not developing only on the ZX are you without using an emulator??? That's a bit bonkers too, but very hardcore, so kudos if you are It's fine to be bonkers though I understand.
Even speccy developers had PCs (or something with a disk drive which they developed on so they didn't lose all their source if the target machine crashed... it's not cheating or anything).
You're not developing only on the ZX are you without using an emulator??? That's a bit bonkers too, but very hardcore, so kudos if you are It's fine to be bonkers though I understand.
Even speccy developers had PCs (or something with a disk drive which they developed on so they didn't lose all their source if the target machine crashed... it's not cheating or anything).
Re: A new data compressor called ZX0
nah, it aint cheating when ZX can do it
btw, i realized, if zcc is used its better to avoid those IX and IY..
so
i spend many times 5 minutes waiting for a tape loading error, like 500 - 1000 times al together ??
why do you THINK i do mind waiting 28 seconds for a compression that will make me wait 1.2 minutes next time ??
without the loading error, but thats the tape deck ...
btw, i realized, if zcc is used its better to avoid those IX and IY..
Code: Select all
startdef ld hl,(DEF_ADR) ; hl = actual RAM location in BASIC listing at position off the DEF FN
ld de,(mode)
ld bc,4
add hl,bc ; adres+4 = var f
rrc (hl)
rlca
ld c,8
add hl,bc ; adres+4+8 = var c
rrc (hl)
rlca
add hl,bc ; adres+4+8+8 = var b
rrc (hl)
rlca
add hl,bc ; adres+4+8+8+8 = var q
rrc (hl)
rlca
ld (de),a ; a = %0000fcbq
i spend many times 5 minutes waiting for a tape loading error, like 500 - 1000 times al together ??
why do you THINK i do mind waiting 28 seconds for a compression that will make me wait 1.2 minutes next time ??
without the loading error, but thats the tape deck ...
- ParadigmShifter
- Manic Miner
- Posts: 765
- Joined: Sat Sep 09, 2023 4:55 am
Re: A new data compressor called ZX0
I understand
As I said, it's fine to be bonkers As you were good Sir
As I said, it's fine to be bonkers As you were good Sir
Re: A new data compressor called ZX0
I hope you are not tired of meEinar Saukas wrote: ↑Fri Mar 22, 2024 4:35 pm Often enough to be worth it.
Anecdotal evidence otherwise doesn't mean much...
I have completed my own implementation, using a custom implementation of graph traversal, which I create by processing repetitions. But, since I didn't understand exactly how you use "last offset", I couldn't use such a technique myself. In general, I managed to make a working archiver, but it inevitably lags behind in compression ratio by ~ 10-15% on large files. I'll try some more analytics, but I'm not sure that it will make a big difference. In the meantime, I've redone your optimize() implementation in C# (which is what I'm writing in) and do some reverse engineering. I'm not a professional programmer, so it's hard for me to do this, but it was generally nice to see that conceptually I wrote my implementation in a similar style. But the main thing is that I understood the basic principle of the appearance of "last offset" blocks. They appear, for example, in a situation where there is the phrase "AAABBBCC" and then there is the phrase "AAADDDCC", that is, they have common parts in comparable places and some different parts, so the "last offset" block will work in matching parts.
- Einar Saukas
- Bugaboo
- Posts: 3169
- Joined: Wed Nov 15, 2017 2:48 pm
Re: A new data compressor called ZX0
Exactly!