Making screen layouts in code

[dfzx]

(I know you're joking but your docs are excellent @dfzx! I'm been working on making my tool export sprites in the format sp1 expects and the docs are really clear about how to pad them out, store the mask, how to load them and so on. Really, they've justified my decision to go with z88dk on their own.)

Thank you! Such comments are really appreciated.

[clebin]

Tetting to grips with z88dk slowly and I'm really liking it despite my usual wrangles with C pointers. The 'solid' property on the tilemap is used now, so I can move a sprite around and collide and land on the scenery with a bit of gravity. I've chucked some bouncing circles on the screen to test the speed and it seems ok before I've added their collision detection. The actual enemies will be a mixture of smaller and larger sprites.




Only problem I'm having is my program keeps wiping out different areas of the ASCII character set whenever I add something new! I haven't looked at how to allocate and structure memory usage on the Spectrum yet, let alone bank switching if I need it.

[clebin]

I should probably take this to the z88dk forum I know...

I'm still struggling to understand the Spectrum memory map and how much memory I've got left once you take out the z88dk and sp1 libraries. The outputted .map file is quite hard to parse as it's not in order and I'm not adept at looking at the hex in an emulator.

My tiles are still overwriting the default font and when I go above only 10 screens, I'm getting crashes! I figured that at some point I would need to use compression, but I didn't think I'd be here after 10 screens and a handful of sprites!

Each screen takes 768 bytes, with each of the 32x24 attributes represented as a single-byte tile number. My tap file weighs in at 20k if that's any guide. I'm setting a few pragmas:

#pragma output CRT_ORG_CODE = 24500
#pragma output REGISTER_SP = 0xd000

I'm not sure any of data is actually in the DATA section at the moment - it's all in CODE. What should I be doing to organise my program?

Damn, I've still got a lot to learn!

[bob_fossil]

I've written a couple of games using C and assembly with z88dk though I used my own graphics routines rather than the SP1 library - which does use more memory. I tended to stick my game code at 32768 or above. If I was using IM2 I would put the IM2 table at 32768 and have my game code follow so that my code didn't clash with the table. I had the stack pointer set to before the table / game code address so that the stack wouldn't overwrite my code. I put any graphics and data above the BASIC area (25000 +). With this layout you do have to keep an eye on your typical stack usage to make sure it's not overwriting your data.

[dfzx]

My tiles are still overwriting the default font and when I go above only 10 screens, I'm getting crashes! I figured that at some point I would need to use compression, but I didn't think I'd be here after 10 screens and a handful of sprites!

Each screen takes 768 bytes, with each of the 32x24 attributes represented as a single-byte tile number. My tap file weighs in at 20k if that's any guide. I'm setting a few pragmas:

#pragma output CRT_ORG_CODE = 24500
#pragma output REGISTER_SP = 0xd000

I'm not sure any of data is actually in the DATA section at the moment - it's all in CODE. What should I be doing to organise my program?

I'm assuming you're using SDCC and startup=31...

You need to take a step back, grab a pen and paper, and draw out your memory map. something like this. It's the only way to get a picture of what's ending up where.

Start by thinking about the blocks of code you have. Your CRT_ORG_CODE pragma has moved the C-compiled code down from its normal position at 0x8000 (32768) to just above the BASIC area. I guess you needed more room for the program as it got bigger? Some of it is now in contended memory which might slow things down but won't really hurt for the time being.

You've also got a pragma to move the stack, but you've put it in its default location, so that seems to be OK. The default memory map for an SP1 program is shown here. If you don't have any other pragmas that's probably the layout you've got.

If your stack is the default size of 512 bytes at 0xD000 it'll grow downwards to 0xCE00 (52736). (Truth is there's nothing stopping it growing down any further than that. If any part of your code is a stack-hog there's nothing in the Z80 or Spectrum hardware to raise an exception or take any action.) Right, so the next question is how much heap are you using? The heap sits on top of your program's data and grows upwards. If you're using too much heap space it'll grow upwards and corrupt your stack. SP1 uses a bit of heap, but I don't really understand what for or how much it needs. I'd suggest populating a block of memory at, say, 0xCD00 to 0xCE00, with a known value. Keep an eye on it as your program runs. If the top bit (0xCE00 downwards) gets changed then your stack is growing downwards beyond its limit. If the bottom bit (0xCD00 upwards) changes then your heap is growing up dangerously close to the stack.

You can get more clues from the map file the compiler generates. The trick isn't to try to parse it, instead decide what you want to know and go looking for it. Unfortunately the symbols are cryptic and I don't know what most of them mean. __code_compiler_size is a useful value. That tells you how many bytes your compiled C code is taking. __BSS_END_tail is another useful one. I think that's the highest point in memory which the C code's data will occupy. To be honest, I'm a bit vague on what the map's telling me as well.

I should probably take this to the z88dk forum I know...

Yes, there aren't too many SP1 experts in the world, all those that exist are in that forum. You'll probably get better help on understanding the map file too.

[clebin]

Huge thanks @bob_fossil and @dfzx for taking the time to help and point me in the right direction. I'm still digesting everything in your posts and will do some more reading around what you've said.

CRT_ORG_CODE pragma has moved the C-compiled code down from its normal position at 0x8000 (32768) to just above the BASIC area. I guess you needed more room for the program as it got bigger? Some of it is now in contended memory which might slow things down but won't really hurt for the time being.

I just blindly copied an example to be honest and moving it around crashes the program. I'm not normally such a hacky programmer, but after the quick successes I guess I got ahead of myself. I'll do as you say and step back and plan it all out. I've had my screen data in the code section so one job will be to separate that so I know what I'm dealing with.

Knowing to look for '__code_compiler_size' and '__BSS_END_tail' is immediately useful. Likewise, the tip about placing some known value in memory to check if it's overwritten by the stack or heap. Going down to 48k is quite an adjustment from anything I've done before and your knowledge is invaluable.

[clebin]

I did as you say @dfzx and mapped everything out - the tips you gave were a good starting point for understanding the map. I can see how setting my program to 0x8000 to place it outside contended memory shoves everything up to meet the stack pointer. I've tried putting the data below it, but I haven't managed to find a working arrangement. I'm not sure what goes on between 0xD000 and the UDGs but after reading your link it seems that SP1 is using it.

Code: Select all

+-------------+
|0xFFFF  65535|
|             | User Defined Graphics
+-------------+
|0xD000       | <-- SP
|0xCFCF       | Stack pointer grows to
|-------------| 
|             |
|             | Z88DK heap memory
|             |
|-------------| 
|0xAF86       | __BSS_END_tail
|             | Z88DK BSS section (CRT_ORG_BSS)
|-------------|
|0xADFF       | DATA section ends (size: 0x2A36/10806)
|             | 
|0x83C        | DATA section starts
|-------------| 
|0x835F       | Program code end (size: 0x1527/5414)
|             | 
|             | 
|0x8000       | (Uncontended memory)
|             | 
|0x6000       | Program code start
|             |
|0x5B00       | (Lower RAM)
|-------------|
|0x5AFF  23295|
|             | Display File
|0x4000  16384|
|-------------|
|0x3FFF  16383|
|             | ROM
|0x0000      0|
+-------------+

It looks like my screens are simply using too much RAM as every empty square is using up a byte. The simplest solution would be to compress the screens I'm not using with ZX0. If I bite the bullet and go 128k, I can uncompress the current level on demand and have space for multiple tilesets. Still, that feels like a cop out.

Oh well, it's all learning...

[Sol_HSA]

Just a quick note: stack grows *downwards*.

[dfzx]

Whether to switch to 128K is obviously your choice, but I'd caution you not to be too keen. It opens a whole host of complications about bank switching and other stuff. Besides, you're now bumping up against the limits of the Spectrum, and many would say that's where the fun starts. How much can you get out of the little machine? Just switching to more powerful hardware when the going gets tough is, as you say, maybe bit of a cop out. But of course it's up to you. Andy Dansby has an excellent set of 128K programming tutorials if you go that route.

Assuming your program never returns to BASIC, the UDGs are irrelevant. They're used by the Spectrum's ROM, and as it happens z88dk defaults to a setup which preserves them (in case the C program returns to BASIC which might still need them). If you never return to BASIC you can use that memory for whatever you like.

SP1 is an assembly language graphics library which can be built so that it fits into the Spectrum's memory wherever you like. z88dk, by default, puts it right at the top of memory. So your assumption is correct: the area from 0xD000 to 0xFFFF is indeed used by SP1, its code and its static tables. It also sets up the IM2 interrupt vectors in that area. (Are you using interrupts? Keep them disabled if not, or set up an empty ISR for now.)

So SP1 is a memory hog. By default it takes the top 0x3000 bytes - that's 12K of precious uncontended memory! You get a lot for your "money" as it were, and you can reconfigure it to take less space if you're happy to trade away some of its potential, but that's advanced stuff which I've not yet tinkered with.

It seems you have another memory hog, though. From what you say, your screens aren't compressed in any way. A byte for an empty square? Hmmm. But you don't need to use ZX0 for compression, at least not yet. A trivial RLE compression would probably save you loads of memory, and is easy to implement in C. I'd start there.

That would mean writing some sort of tool for your PC (Windows or Linux, I presume) which compresses your screen data down and packs it into an ASM file as DEFB constants (or something similar). Those files are then assembled into a new section (i.e. z88dk builds them as a binary file separate from your compiled C code). That binary then needs to be loaded into contended memory (between the screen and 0x8000). When your C code needs the next screen it needs to go and find the packed data and unpack it into non-contended memory where it can be given to SP1 to use.

It sounds complicated spelt out like that, but it honestly isn't that hard. Pretty much every game, even BITD, needed an approach like that. We're Spectrum programmers, it's what we do.

If you need help rearranging your memory map I can help with that, either here or on the z88dk forum. But I think you'll find that just implementing simple compression/decompression routines for your static data will free up enough memory to get you going again. You really need everything stable before you start relocating everything because that's not an easy process to get right first time.

[clebin]

Whether to switch to 128K is obviously your choice, but I'd caution you not to be too keen. It opens a whole host of complications about bank switching and other stuff. Besides, you're now bumping up against the limits of the Spectrum, and many would say that's where the fun starts. How much can you get out of the little machine? Just switching to more powerful hardware when the going gets tough is, as you say, maybe bit of a cop out. But of course it's up to you. Andy Dansby has an excellent set of 128K programming tutorials if you go that route.

Yeah, you're right... I'm not thinking like a Spectrum programmer. No more cutting corners.

Assuming your program never returns to BASIC, the UDGs are irrelevant. They're used by the Spectrum's ROM, and as it happens z88dk defaults to a setup which preserves them (in case the C program returns to BASIC which might still need them). If you never return to BASIC you can use that memory for whatever you like.

That's interesting. I'm still not sure why characters get randomly overwritten by my tiles but it's not a big deal for now. I'll come to that eventually.

It seems you have another memory hog, though. From what you say, your screens aren't compressed in any way. A byte for an empty square? Hmmm. But you don't need to use ZX0 for compression, at least not yet. A trivial RLE compression would probably save you loads of memory, and is easy to implement in C. I'd start there.

That would mean writing some sort of tool for your PC (Windows or Linux, I presume) which compresses your screen data down and packs it into an ASM file as DEFB constants (or something similar). Those files are then assembled into a new section (i.e. z88dk builds them as a binary file separate from your compiled C code).

I'll do that. I have my tool that I linked to earlier which converts Tiled files into assembly or C format. I'm slowly refining it so I'll add an extra argument to do RLE compression. I'm on a Mac and using VS Code by the way.

It sounds complicated spelt out like that, but it honestly isn't that hard. Pretty much every game, even BITD, needed an approach like that. We're Spectrum programmers, it's what we do.

I hope to be more of a part of that 'we' some day!

If you need help rearranging your memory map I can help with that, either here or on the z88dk forum. But I think you'll find that just implementing simple compression/decompression routines for your static data will free up enough memory to get you going again. You really need everything stable before you start relocating everything because that's not an easy process to get right first time.

Thanks for the kind offer! I may take you up on that later but I'll keep plugging away for now.

I did had real "doh!" moment earlier. I said I can't get my program to work when I move the DATA section around, no matter what I tried. Even CRT_ORG_DATA=-1 wouldn't work, which I read is supposed to stick it at the end of the code section but in a different bin file.

At that point I checked my file-sizes and realised that "-create-app" doesn't actually stick together the generated *_CODE.bin and *_DATA.bin files in the TAP file. Argh, no wonder it crashes every time!! So now I have to figure out z88dk-appmake to combine them. Slowly, I feel like the mist is clearing...

[clebin]

To keep the motivation up, I started putting together a vertical skyscraper level. It won't feature any flying vans.

[dfzx]

I did had real "doh!" moment earlier. I said I can't get my program to work when I move the DATA section around, no matter what I tried. Even CRT_ORG_DATA=-1 wouldn't work, which I read is supposed to stick it at the end of the code section but in a different bin file.

At that point I checked my file-sizes and realised that "-create-app" doesn't actually stick together the generated *_CODE.bin and *_DATA.bin files in the TAP file. Argh, no wonder it crashes every time!! So now I have to figure out z88dk-appmake to combine them. Slowly, I feel like the mist is clearing...

I'd expect setting CRT_ORG_DATA to -1 to produce the separate DATA bin file. Odd that it didn't.

In my experience the -create-app option is overrated. It only seems capable of doing really basic stuff, and doesn't complain when something harder is put in front of it. It normally gets those wrong, as you've found out. I use individual compile-to-object, link, and app-make commands in any less-than-trivial projects.

[clebin]

I'd expect setting CRT_ORG_DATA to -1 to produce the separate DATA bin file. Odd that it didn't.

Sorry if I wasn't clear. CRT_ORG_DATA =-1 did create a separate DATA bin file, just that -create-app ignores it and I hadn't realised.

[clebin]

I've moved my 'Spectrum Tiled Tool' from Bitbucket to Github:

https://github.com/clebin/spectiledtool

I also added the simple RLE compression that @dfzx suggested (cheers again) as an extra option:

Code: Select all

php SpecTiledTool.php --tileset=tileset.tsj --map=map.tmj --graphics=tiles.gif --compression=rle --format=c

This is only for the tilemap/screen data. If you use --format=asm, it adds a 16-bit number at the beginning to denote the size of the data.

I'm saving 40-60% depending on the screen with no noticeable extra delay in switching screens, so that'll give me enough space to be getting along with for now.

[dfzx]

I'm saving 40-60% depending on the screen with no noticeable extra delay in switching screens, so that'll give me enough space to be getting along with for now.

Has this restored stability?

[clebin]

I'm saving 40-60% depending on the screen with no noticeable extra delay in switching screens, so that'll give me enough space to be getting along with for now.

Has this restored stability?

Yes it's running stable now thanks. I'm not relocating the DATA section anywhere at the moment as I haven't figured out z88dk-appmake yet, but speed is fine so far with the code in contended memory.

[clebin]

Another little screenshot:



It might be obvious now which game I'm remaking... although it's not exactly a well-loved title!

I was on the verge of releasing it, but decided to bit the bullet and revise those screen layouts again. By saving each building, tree, phone-box, etc as an individual tilemap and combining them to build the screen I should be able to save 2-3k total. That's huge in the context of this game as I'd had to strip everything right back to fit into RAM, including the wasp pictured in this shot.