Abersoft fig-Forth Recompiled

[ParadigmShifter]

Ok. I'd just expect something like Update/BeginDraw (tells engine you are about to draw so you have finished the update)/EndDraw (does the sort, then sets flag in ISR telling you are ready to draw), kind of like how openGL works (not a low level engine coder though, so don't believe everything I say)

[ketmar]

ah. i see it more like a "hardware sprite", though: you change the coordinates, and the sprite immediately (well, almost) moves to the new place. that's all you need to do — just tell the engine the new coords and gfx address. it is harder to program (the engine itself, i mean), but much easier to use. ;-)

btw. reading that WoS topic gave me an idea. i can draw the sprites at the bottom first, yay! why chase the ray? render bottom sprites first, and by the time you're finished, you can render top sprites without flickering. and i can sort them without worrying about wasting precious "off-screen" cycles. and i can even call ROM IM1 handler too. and play music. and some sfx.

why haven't i thought of that myself?!

[ParadigmShifter]

I'm sure that's a great way to do it if you do have hardware sprites

I skipped all of that (sprites I mean) (cos speccy broke when 17, only started programming games in C when 23 or so, no computer owned in the meantime, imagine being able to score a games programming job doing that theses days lol) and went straight onto 3D openGL style stuff (original Sony libs were based heavily on openGL I think... certainly by the time the PS3 came along - mostly because of all the backlash they had about the PS2 being ridiculously hard to program for (and it shipped with no libs at first I think, lol)).

I like racing the raster anyway it's more exciting And you can do all your processing for next frame leisurely once you've drawn the previous frame as fast as you possibly can. (I don't like flicker or tearing either!). Probably rules out in game beeper tune though which is a shame (AY tune is easy obvs if I could be bothered to learn 128K or AY stuff)

JC does know what he is doing and has miles more experience with Z80 than me anyway. He just needs to learn which direction the x and y axes point

[ketmar]

yeah, if you're doing a custom engine for your game, chasing the ray is ok. but with a general-purpose engine, which intended to be used by people who know little to nothing about such arcane tricks… remember, it is 1986 now in this topic! ;-) and i am not writing something for Joffa to use. ;-)

[ParadigmShifter]

I'd rather it wasn't 1986 tbh - too much D&D not enough womanising (as in critical fail natural 1 rolled on d20 for that) And I didn't even want to go to the pub then!

EDIT: Anyway the Naughty Dog docu on Youtube about Crash Bandicoot is probs worth a watch (not seen it for ages, I think they may mention LISP in it but I don't think they mention what Sony did after they bought them).

[AndyC]

this will limit the ways to use the system. the idea is that you can move the sprites anywhere in your code, when you want it, not when the system wants you to do it. you command — the system obeys, not vice versa.

Doing this in interrupts will make the code exponentially harder to use. Because, at that point, all your sprite data is effectively shared between two threads, will all the concurrency problems that incurs, but on an 8-bit CPU I'll designed for concurrent code.

It's way, way easier to have one "go do all the sprite management and drawing" function that just has to be called once per game loop. You can still have functions to move sprites as and when you like, it just doesn't reflect the results on screen until a screen redraw happens.

[ketmar]

Doing this in interrupts will make the code exponentially harder to use. Because, at that point, all your sprite data is effectively shared between two threads, will all the concurrency problems that incurs, but on an 8-bit CPU I'll designed for concurrent code.

this is not really a big deal. the user cannot directly modify sprite coords, the only way to do it is to call "SPR-UPDATE" word. and that word takes care of checking if the sprite is not rendering right now, and how to properly modify everything even if it is queued. yep, there are some checks and spinlocks in that code, but nothing the end user should see, know or care about.

for the end user, the only precaution is: "if you try to move too many sprites in one frame, some of them may not be updated instantly. they will eventually move to the new place, but there can be delay for several frames."

i agree that the internal engine code is messy this way (modern multithreading with all its pitfalls, now on Z80, yay! ;-), but i believe that the end result totally worth it. the final result really looks more-or-less like a hardware sprite.

of course, using the explicit "END-FRAME" word is easier to implement. but it is harder to use considering the interactive nature of Forth. it may be ok for other languages, without REPL session, where you have to recompile all your code each time.

but for Forth, i'll add window support to print driver later (it is easy), and you'll be able to have your game with a fully-featured debug console. that console is way more powerful than any of the consoles in modern games, because it is actually a REPL with the language your game is written in. and you can call any word there, or write new code, or… even the sky is not the limit! ;-)

for this, "instant sprites" are much easier to use, i believe.

[ketmar]

p.s.: i mean, my sprites are "hardware" in the meaning that you cannot get back the sprite position or gfx after you updated it. it is like doing OUT to some write-only hardware ports: you put new values there, and that's it. if you need to track any info about your sprites, then do it yourself. so user data is never subject to any race conditions.

[DoctorRad]

thank you! but it still doesn't have a source code (or i am unable to extract it).

I was assuming that all the entry points to the sprite routines for Machine Lightning would be exposed so you could call them from your own assembler or otherwise compiled code… or is that not what you're seeking to do?

[ketmar]

I was assuming that all the entry points to the sprite routines for Machine Lightning would be exposed so you could call them from your own assembler or otherwise compiled code… or is that not what you're seeking to do?

sorry, but i need full disassmbly, so i could re-assemble everything to any address, and conditionally exclude the routines i don't need. White Lightning already has all entry points, even named. but alas, no assembler source code.

[ketmar]

so i decided to draw sprites in batches by 8 sprites per interrupt. let's hope it will be enough. (i may adjust this number after testing, tho.)

interrupt handler will take first 8 sprites from the queue, and sort them in 3 sub-batches: one sub-batch per 1/3 of screen$. it won't sort sub-batches themselves, though.

then it will blit the bottom batch, then the top batch, then the middle batch. the rationale is this: before the handler there may be some other code (input handling, music, etc.). then the sorter. this will not leave enough time to draw the top 1/3 of scr$, so we will start from the bottom one, hoping that the ray is not there yet. after this, the ray is prolly scanned the top part, so we'll render "top" sprites. and then the middle part, in the hope that the ray is at the bottom now.

as most sprites aren't moved too far away from their previous positions, this should reduce both flickering, and tearing.

high-level code will take care of sprite queuing, checking for one sprite to not queued twice, and so on. the user will simply call "SPR-UPDATE ( x y spr-data spr-index )" word, and the engine will do the rest. i will also implement API to check how many sprites are queued yet, if the given sprite is in queue, and if there is enough room for more sprites to be rendered on the next interrupt.

i already started writing the code, so we'll see how well it will go.

ah. i also ditched runtime-shifted sprites, and switched to AGD sprite format: 32 bytes, pre-rotated. this way i won't need to pefrom any costly sprite shifting in runtime, and there is no need for huge buffers to hold shifted sprite data.

after implementing this idea, i will prolly switch to bottom-up rendering for sprites, like Ultimate does in their games. i.e. each sprite will be rendered starting from the bottom row.


i may also look into implementing PSST-like engine later: "OR" sprites, then erase them, and draw again. this time i will not use any interrupts (or will implement "END-FRAME" callback).

[ParadigmShifter]

If you flip them (or work backwards instead to draw them bottom up) it's probably worth reversing the direction you layout the data as well every other line so you can zigzag.

I always write different routines for varying widths of sprites as well, I like to waste memory on code lol Multiples of 2 wide you can use the stack to blit (but you can't zig zag then of course...). Height is ok being a variable though since that's just the outer loop variable.

[DoctorRad]

interrupt handler will take first 8 sprites from the queue, and sort them in 3 sub-batches: one sub-batch per 1/3 of screen$. it won't sort sub-batches themselves, though.

An alternative approach that someone more adept than me could code up fairly easily I think: David Webb's routines from his book Advanced Spectrum Machine Language will print 40 characters with attributes before the main screen starts being drawn. That's ten 16x16 sprites on character boundaries, without erasure of the previous sprite position. Optional OR-printing is also available by utilising the flash bit of the attribute code.

You could then print another 40 characters while the top third of the screen is being drawn, so you'd only need two queues: characters anywhere, and characters in the bottom 2/3 of the screen. Deciding which queue to put the entries in could be done by the game code rather than having to perform a sort, and there should be no flicker.

I think more performance could perhaps be squeezed out of similar code, though, as Webb doesn't use the stack pointer trick to move two bytes at a time, largely because the routines deal with single character squares.

[ParadigmShifter]

I was thinking of maybe having 3 queues, one for each screen third for my stuff but it's fast enough/not drawing enough yet I don't need it atm.

I do scroll columns down 8 pixels one third at a time though (which is a pain, and requires making a copy of the 7th and 15th row of the screen, it's much easier to scroll from bottom to top).

I use an erase list and a draw list which I populate after drawing the previous frame stuff (so I can start drawing immediately following the HALT). And all my graphic drawing routines are aligned to 256 byte code boundary so I can LD H, address : LD L, 0 : JP (HL) to jump to them (I am using stack to copy everything so can't use a CALL). Also saves storing the low byte of the routine to jump to in the drawlist buffers, saving a byte per operation.

[ketmar]

@ParadigmShifter, optimisations will come later. at first, i want the main code up and working. ;-)

David Webb's routines from his book Advanced Spectrum Machine Language will print 40 characters with attributes before the main screen starts being drawn. That's ten 16x16 sprites on character boundaries, without erasure of the previous sprite position.

i can redraw about 4 sprites before the ray hits the first scr$ byte. but the problem is, as i said, that there could be other code executed before sprite printing: sfx, beeper music, AY music, and so on. especially for music, i don't want it to "float".

it is essential, for example, to poll the keyboard when the ray is off-screen, this way keyboard polling is stable.

besides, there can be the case when all sprites are at the top. using 3 batches will deal with this case without much problems. that could happen quite often if the user has gamplay area at the top, and the status at the bottom.

that's why i prefer to go after the ray, if it is possible, instead of trying to run before it.

as for sorting, it's not that hard: i don't need the strict order, i only need to put 8 sprites to 3 lists: "top", "mid", "bottom". this can be done in one pass. i chose 3 lists, because it is easy to compare — just check the high byte of scr$ address.


there is one trick with updates, tho: the curcular buffer with sprites queued for rendering is being modified by both interrupt handler, and by the user code. and i cannot simply disable interrupts, because then i will miss an update completely.

so i used this trick: i have a flag, which when set, tells the interrupt handler to increment some variable and skip sprite updating. i am setting this flag when i need to update lists, and then i check if the interrupt occured while the main code was busy doing updates. if there was an interrupt, i know that i missed one scr$ update, but it is not too late to do it yet, so i manually call the routine to update scr$.

(actually, there are two flags, but let's skip other details for now. i may write a better explanation when i'll have this code working.)

[DoctorRad]

i can redraw about 4 sprites before the ray hits the first scr$ byte. but the problem is, as i said, that there could be other code executed before sprite printing: sfx, beeper music, AY music, and so on. especially for music, i don't want it to "float".

You could do everything else you want to do 50 times a second immediately after the drawing routines, they would still be happening every 1/50 of a second, just ~128 scan lines after the interrupt itself.

it is essential, for example, to poll the keyboard when the ray is off-screen, this way keyboard polling is stable.

Why is it essential to do it when the ray is off-screen if you do it at regular intervals?

besides, there can be the case when all sprites are at the top.

This is the main limitation of my proposal, but could be overcome with game design and logic, perhaps.

[ketmar]

You could do everything else you want to do 50 times a second immediately after the drawing routines, they would still be happening every 1/50 of a second, just ~128 scan lines after the interrupt itself.

yes, you are right. but there can be less than 4 sprites to update. i don't want to waste time for all 4 sprites, if the user have only 1 or 2. the timings will deviate a lot. it's not a big deal, of course, but still something to remember.

Why is it essential to do it when the ray is off-screen if you do it at regular intervals?

due to some quirks of some original hardware. it is recommended to poll the keyboard when ULA is not busy, and the easiest way to do it is to poll it in the interrupt handler. otherwise it is better to poll the kb several times to check if the result is stable.

i don't remember the gory details, but i am absolutely sure that i read the articles with proper explanation of this (down to hw level). it may be false memory, of course, but better be safe than sorry, i think. ;-)

This is the main limitation of my proposal, but could be overcome with game design and logic, perhaps.

only i'm not creating the engine for some particular game. i simply don't know what the user may want, and i want the engine to be as general as possible.

of course, it is possible to document the requirements, and let the user deal with them… but remember, i am creating The Best Game Making Tool of 1986! the less limitations are there — the easier to sell it to Ocean and became a rich man! ;-) it is a kind of roleplaying which adds some fun to the task.


p.s.: and sorry for being negative. i appreciate your proposals, and thought about them hardly. i am just trying to explain why i chose the way i… chose, not to tell that your ideas are wrong.

[ketmar]

so, i wrote the code to queue sprites, to sort the queue to 3 batches, and to blit the batches.

i implemented one additional check: if the bottom batch doesn't have enough sprites (currently "no sprites", but i will prolly change it to "2" or something), i am blitting the mid batch, and then the top batch. otherwise top, then mid. this is for the case when the game field is at the top.

by the way, it is possible to atomically check and reset the bit-flag on Z80, and set another one (i needed it to check if interrupt happened): The Mighty "SCF / RL (HL)" instruction is to the rescue!
bit 7 is "interrupt triggered" flag, and bit 0 is "interrupt blitting enabled" flag. "blitting enabled" is reset here, so i can shift "triggered" flag to carry, and simultaneously set "blitting enabled". yay, who would thought that Z80 is so advanced that it has atomic instructions for hardware multitasking?! ;-)

the code is very straightforward now, no optimisations were done. the only optimisation is unrolled 16 line blitting calls, but that's only because i am out of registers to write a proper loop. ;-)

[ketmar]

also, while i am here: i stole some sprites for tests from All Hallows. of course, i will remove them for the release, just wanted to recomend you a fun game. ;-)

p.s.: btw, i hope you're all enjoying reading this "dev diary" (or at least not annoyed by it too much ;-). sorry if i spam with message notifications too much. i just thought that it would be interesting to read my thoughts, and some progress while you don't have something real to try.

[ketmar]

found bugs in "2SWAP" and "<>". two almost useless words, lol.

and some trivia to make this post bigger: did you know that ZIP Compiler basically translates BASIC to Forth, and then compiles Forth to machine code, with top of the stack cached in HL?

it is possible to write a native ZX Forth compiler which does the same, but i already have too much projects on my hands. so it have to wait… a little, i hope.

[ketmar]

16-sprite PoC tech demo. control code is in Forth, of course. updating 9 sprites per interrupt. the main control loop is exactly:

Code: Select all

: xi
  1 BORDER
  SPR-SETUP
  \ CLS CR CR CR
  ." \cpress SPCACE to pause animation.\nuse kempston to move.\n"
  #XI . ." sprites, and the player."
  SPR-ON
  0 LAST-K C!
  BEGIN
    player ( control) #XI SPR-UPDATE
    #XI FOR I 4* SPI + I xz-update LOOP
    \ HALT \ this can be uncommented, it doesn't really matter
    xi-moving IF
      SPI DUP #XI 4* + SWAP DO I xz-move 4 +LOOP
    ENDIF
    LAST-K C@ 32 - 0IF
      xi-moving 1 XOR TO xi-moving
      0 LAST-K C!
    ENDIF
  ?TERMINAL UNTIL
  #XI FOR I SPR-ERASE LOOP
  HALT HALT HALT HALT
  \ SPR-OFF
  \ SPR-UNSETUP
;

p.s.: now i prolly need to rip off some AGD game as a demo. ;-)

[ketmar]

trying to optimise things a little, and add attribute control. but for this we need 768-byte "screen property map", so the engine could skip overwriting tile attributes. this map can be put in contended memory, i believe, because it is not heavily used.

for now, i can (re)print 11 sprites per interrupt. with attrs it will prolly be 10 — still ok. basically, 8 nasties, player, and one free sprite — enough for most games. even 9 is ok, i think. i will prolly stick with 9, and implement blitters for smaller 8xn sprites, to be used as projectiles. it usually doesn't matter if projectiles will flicker a little.

[ketmar]

implemented sprite attributes, and property map. property map covers the whole screen (32x24), and if bit 7 in property byte is set, it means "don't replace attributes". this map can also be used for other tile properties, like "passable" and such.

currently, attribute mask for sprites is hard-coded, and doesn't allow to change paper color. this is because old sprite attributes are not erased, so changing paper will leave a trail. also, you could use paper to draw something on the background. ;-)

there is a small color fighting if several sprites try to set attribute. alas.

also, it is possible to skip attribute setting completely by using attr $FF. and it is possible to override sprite attribute on update, so you don't need to duplicate bitmap sprite data if you want to use it with another color. ;-)

download demo

[ketmar]

screenshot:

Spoiler

also, sprites are rendered from their borrom line now. and sprite origin is its bottom line: i used to this in XTE, and i believe that it is more useful than "origin at the top" mode.

[ketmar]

as i need to build rooms somehow, let's build the room builder! of course, with kempston mouse, because i refuse to use any kind of such visual editor without a mouse.

lucky me: ZXEmuT has API to read and write host files, so i don't have to mess with snapshots. for others it will be the usual block-based disk i/o.