Understanding BASIC memory map

[jamesh]

“BASIC Programming” by Vickers illustrates BASIC memory map with this graphics:

Code: Select all

---+-----+----+----------+----------+-------+-------+-----+---+---+------------+
...|INPUT| NL | Temporary|Calculator| SPARE |Machine|GOSUB| ? |3Eh|User-Defined|
...| data|    |work space|  stack   |       | stack |stack| ? |   |  Graphics  |
---+-----+----+----------+----------+-------+-------+-----+---+---+------------+
   |                     |          |       |                 |   |            |
 WORKSP                STKBOT     STKEND    sp           RAMTOP  UDG      P_RAMT

Perhaps someone can help me understand it better, please?

The area between WORKSP and RAMTOP, is it actually unused unless BASIC code is running? Well, except the CPU stack, but we’ll get to that later. It looks like after NEW, LOAD “”, and even RUN, WORKSP==STKBOT==STKEND, which means entire area between WORKSP and SP is unused, right? And that’s what BC_SPACES call does, allocates a chunk in that area. Also, BC_SPACES crashes hard (exists to BASIC) if you request more than available. And somehow adds 80 bytes to the number your requesting, for whatever reason. Considering all that, how do I estimate the amount of “available” memory there? Basically RAMTOP-STKEND-something would give a rough estimate, but how do I measure/estimate that something? A few bytes, kilobyte, several Kbytes? That something is Z80 stack + GOSUB stack + that mysterious “?”.
There is “FREE MEMORY” routine at 0x1f1a, is that the one I am looking for?

And a few questions to understand the map better:

1. What’s that “?” between “GOSUB stack” and RAMTOP?
2. GOSUB stack, it’s above Z80 stack? Growing it should be expensive… But, anyway, unless BASIC is in the middle of a SUBroutine, GOSUB stack should be 0, right?
3. What would be a safe estimate for the maximum Z80 stack size, used by BASIC? Using FUSE’s built in debugger I can see that SP points to 12 bytes below RAMTOP, when sitting at the prompt, and 18 bytes below when entering code via USR addr.

Similar questions about the area above RAMTOP:
1. What’s the purpose of that “3Eh” between RAMTOP and UDG area?
2. UDG area can only be moved explicitly, right? Unless something modifies UDG sys var, it just stays 8*21 bytes below P_RAMT, right?

Thanks!

P.S. The goal is to allocate a large chunk of memory in your machine code without asking user to explicitly set RAMTOP.

[catmeows]

P.S. The goal is to allocate a large chunk of memory in your machine code without asking user to explicitly set RAMTOP.

But RAMTOP is there exactly to limit memory used by Basic. Anything below it is not guaranteed.

UDG area can only be moved explicitly, right? Unless something modifies UDG sys var, it just stays 8*21 bytes below P_RAMT, right?

Yes.

1. What’s the purpose of that “3Eh” between RAMTOP and UDG area?

Byte 0x3E is put by ROM on the bottom of stack. AFAIK, it is not used for anything, but I may be wrong.

3. What would be a safe estimate for the maximum Z80 stack size, used by BASIC? Using FUSE’s built in debugger I can see that SP points to 12 bytes below RAMTOP, when sitting at the prompt, and 18 bytes below when entering code via USR addr.

This question does not make sense, the space is allocated and deallocated from both directions, stack from top, working space from bottom. The ROM itself, whenever it wants to reserve more working space, checks there is at least 80 bytes between top of stack and basic stack STKEND -> see ROM routine 'TEST-ROOM'. That may be little bit conservative but a) either you plan to never return to BASIC and then you don't care or b) you will return to Basic or at least use some of ROM routines and in that case this test will throw error very soon after you return to Basic.

GOSUB stack, it’s above Z80 stack? Growing it should be expensive… But, anyway, unless BASIC is in the middle of a SUBroutine, GOSUB stack should be 0, right?

GOSUB stack is just glorified name for the fact that GOSUB will do four operation befor continuing to GOTO code: pick up ERROR handler from Z80 stack, put number of current line on Z80 stack, put number of current command on Z80 stack, put ERROR handler back on the top.
Therefore Z80 and GOSUB stack are interleaved and for all practical purposes it is just Z80 stack.

What’s that “?” between “GOSUB stack” and RAMTOP?

no idea, I'm not author of that diagram

The area between WORKSP and RAMTOP, is it actually unused unless BASIC code is running?

Well, define 'unless BASIC is running'. ROM is either interpretting a line of basic program or is in input mode. In both cases, it allocates space above WORKSPC. For example, you type a letter into input line, a single byte is allocated somewhere above WORKSPC.

Actually, you answered yourself already: BC_SPACES (and the other memory magament routines) actively manage memory between top of z80 stack and start of BASIC area. I repeat: between start of BASIC and top of Z80 stack. Whenever you need room for a line of basic, buffer for INPUT command, space for tape header, space for new variable, space for old string that is now longer, space for calculator stack, these routines kick in, give you room and collect garbage when you are done. That is why, in general, trying to allocate safe space below RAMTOP is good example of bad idea.
Still, people are trying to do crazy thing - if I would do something like that, I would use string variable, if I don't mind that its location change frequently and I need search through variable area all the time. Another posibility is to use space directly in program line, behind REM or DATA statement, as both are ignored by interpret. Still, if you do something with channels, start of Basic will be moved up anyway.

[catmeows]

Edit:
1) that 3E byte is mark for bottom of GOSUB stack
2) yes, FREE MEMORY is that one you want - use PRINT 65536-USR 7962

3) I would really use variable area:
3a) read value of VARS to get start of variable area
3b) reserve space for a dummy variable by calling MAKE-ROOM with HL=pointing to address you got by step 3a (start of variable area) and BC=needed space +3
3c) put string mark in the first byte: 0x40 | (ascii code - 0x60), put 2 bytes of length in the second and third byte, use rest of reserved space

note that there are 5 bits reserved for letter but there is only 26 letters that could be used by Basic as string variable name. that means it is perfectly possible to have string named '{$' or '|$' in variable area. Such string will not be accessible from Basic and will be completely transparent to any Basic program. It will be always available at the start of variable area. Of course, NEW or CLEAR will destroy it.

[jamesh]

that 3E byte is mark for bottom of GOSUB stack

Well, thanks a lot! That helps already, especially GOSUB and UDG parts. Perhaps, I should have said “TEMPORARILY allocate a chunk of memory” to avoid confusion.

I must admit, I forgot to mention the context – dot command, and what I meant by “not running BASIC” hopefully gets better defined in that context. Original ZX Spectrum ROM is not multitasking in any way. So at any given moment when CPU us executing my code [written in assembly language] I am 100% confident that “BASIC” (either prompt or interpreter) is not running. Sans interrupt handling intricacies. Hence, if my code can do something (e.g. allocating a chunk of memory) that BASIC would not “notice” once it resumes execution, it should be perfectly fine. Am I wrong?

Re: 3Eh at the top of the stack. That "?" may be a placeholder for the MSB of that 3E record, stack records are 2 bytes long, aren't they?

The code we ended up with, if someone is curious.

[catmeows]

IMHO, you can use all space between STKEND and Z80 stack. Still, remember that USR is just Basic function and it should work even when it is part of more complex expression so you should not touch anything bellow STKEND.
But why not to reserve space directly on Z80 stack ?

[ParadigmShifter]

Only the interrupt service routine (ISR) can run in the middle of executing machine code.

If you use the default interrupt handler all you need to do is make sure IY remains the same since it updates FRAMES and the keyboard scanning stuff. Also the stack pointer must be pointing to the actual stack.

So it should be safe to run anything (that does not corrupt basic stuff of course) as long as you keep the default ISR (i.e. don't go into IM2) and you don't mess with IY (you can do that if you change IY within a DI/EI block that sets IY to the correct value before turning interrupts back on). If you change SP you need to do that inside a DI/EI block too.

EDIT: H'L' needs to be preserved when you return to basic as well though.

[catmeows]

If you change SP you need to do that inside a DI/EI block too.

Why you should not change stack pointer, Basic does that all thé time.

[ParadigmShifter]

Why you should not change stack pointer, Basic does that all thé time.

PUSH and POP is fine... (EDIT: because they are atomic i.e. an interrupt can't go off in the middle of a PUSH or POP instruction).

What you can't do is save the SP, set it to something else (e.g. to do a fast copy routine), then use that inside a block with interrupts enabled before setting it back.

EDIT2: ADD HL, SP and ADD IX, SP are atomic too that should be ok as well. ADD IY, SP would be ok as long as you don't modify IY (so not very useful really).

EDIT3: But that's only because the ISR saves and restores registers if it uses them so this can't happen

LD HL, somevalue
; ISR goes off now
ADD HL, SP ; if ISR changed value of HL without saving and restoring it, this would be bad

[catmeows]

Any ISR that would destroy HL would be most likely useless.

What I mean than you can reserve space on stack like this:

Code: Select all

ld hl, -1000  ; reserve 1000 bytes
add hl, sp
ld sp, hl

and free reserved memory like this

Code: Select all

ld hl, 1000
add hl, sp
ld sp, hl

basically any high level compiler supporting stack frame would do that this way, perhaps using IX instead HL to have easy access to reserved space.

[ParadigmShifter]

Yup that is ok. EDIT: Because even if an interrupt occurs SP is changed atomically

What isn't ok is stuff like this if an interrupt goes off in the middle (it uses DI/EI to prevent that though so it is safe)

Code: Select all

	; A -> attrib to set when clearing screen
cls:
	di                  ;disable interrupt
	ld (.stack+1), sp	;store current stack pointer
	ld sp, 16384 + 6144
	ld hl, 0
	ld b, l             ;set B to 0. it causes that DJNZ will repeat 256 times
.loop1
	push hl             ;store hl on stack
	push hl             ;next
	push hl             ;these four push instruction stores 8 bytes on stack
	push hl
	push hl             ;store hl on stack
	push hl             ;next
	push hl             ;these four push instruction stores 8 bytes on stack
	push hl
	push hl             ;store hl on stack
	push hl             ;next
	push hl             ;these four push instruction stores 8 bytes on stack
	push hl
	djnz .loop1			;repeat for next 12*2 bytes
	ld sp, 16384 + 6144 + 768
	ld b, 128			; clear attribs in 128 * 3 pushes
	ld h, a
	ld l, a
.attribloop
	push hl
	push hl
	push hl
	djnz .attribloop
.stack
	ld sp, 0            ;parameter will be overwritten
	ei
	ret

[1024MAK]

Is there any reason why RAMTOP is not being moved down? Also, if UDG are not being used, that area can be used.

Mark

[jamesh]

That's how it works right now. Code checks RAMTOP and prints "please do CLEAR XYZ" if there is no room above the RAMTOP.

However, given the nature of the tool (a dot command) and most frequent use case (fresh after reset) I'd like to remove that manual step. In that most frequent scenario there is enough memory it's just a matter of choosing the right area. Of course, with "fresh after reset" state I can simply use 0x8000 as I "only" need 16K. But I'd like to have a universal solution, which may work and co-exist with some BASIC code of an arbitrary size.

[AndyC]

Couldn't you just lower RAMTOP automatically? Surely it's not that difficult to find the entry point to the CLEAR routine?

[1024MAK]

When your own machine code is in control, you can move the entire area around RAMTOP down, as long as your code updates the relevant system variables and the stack pointer to the new values.

Mark

[1024MAK]

Also, just a reminder, CLEAR can be used in a running BASIC program, like so:

Code: Select all

10 PRINT "one"
20 PAUSE 150
30 CLEAR 60000
40 PRINT "two"

But keep in mind that CLEAR also clears the screen and clears any current variables in use. Hence if you have any data in variables, you need to store those values somewhere outside BASICs area.

I'm sure that a machine code program could replicate similar functionality

Mark

[jamesh]

About CLEAR'ing from my machine code. I could not get CLEAR_1 working, mostly because I am not sure how that STMT_RET and 'error address' should be defined.

[C.Born]

Also, just a reminder, CLEAR can be used in a running BASIC program, like so:

Code: Select all

10 PRINT "one"
20 PAUSE 150
30 CLEAR 60000
40 PRINT "two"

But keep in mind that CLEAR also clears the screen and clears any current variables in use. Hence if you have any data in variables, you need to store those values somewhere outside BASICs area.

I'm sure that a machine code program could replicate similar functionality

Mark

like in a REM statement, that will be save for any Clear OR Run.
besure you have the correct ADRESS
system var 23637 hold NEXTLINE wich i lower with 4 bytes STEP BACK
this is what i use now and MM points to the MINUS SIGN '-':

Code: Select all

  10 DEF FN p(a)= PEEK a+ 256 * PEEK (a+1)
  12 LET mm=FN p(23637)-4 : REM -1x

cheers