Spectrum Next dotn files

[markusr]

I'm trying to port Specasm to the Spectrum Next. The 48kb version of Specasm consists of four binaries, two tap files (specasm and salink) and two dotx files (saimport and saexport). .launcher is used to allow the tap files to be launched from the command line via a dot command, e.g., typing .specasm launches specasm.tap, the assembler/editor. The tap files are both pretty big. In the last release, specasm.tap was about 40kb and salink.tap was around 37kb.

As .launcher is not supported on the Next, as the addition of the Next opcodes to the Specasm binary will require some sort of paging to get everything to fit and as dotx files require a CLEAR to use, I'd like to convert all the existing binaries to dotn files for use on the Next. I've been able to rebuild the two smaller dotx files (saimport and saexport) as dotn files and these build and run fine on the Next. Unfortunately, I can't figure out how to convert the two larger binaries to dotn files. When I compile them I get an error that looks something like this

Code: Select all

Error: Section MAIN exceeds 64k [0x8000,0x10329]
zxn: Aborting... one or more binaries overlap
Building application code failed

Here's the command line that generates that error

Code: Select all

zcc +zxn -SO3 --opt-code-size --max-allocs-per-node200000 -Cs "--disable-warning 85" -clib=sdcc_iy -m -startup=31 -o SALINK salink.o error.o expression.o state_base.o peer_file_next.o util_print_zx.o peer_next.o -subtype=dotn -Cz"--clean" -create-app

I'm not able to find much documentation on dotn files but what I can find suggests that dotn files have no memory limits. However, I can't figure out what I need to do to take advantage of this feature. I've tried messing around with a few settings I don't understand like #pragma output DOTN_LAST_PAGE = 11, all to no avail.

Can anyone explain how to write large dotn files or point me to some existing examples? Do I need to implement some manual paging mechanism to get this to work? How would I go about doing that?

Any help or advice would be greatly appreciated.

[_dw]

I don't know anything about it. So I can comment on it... .)

According to what you write, you are trying to compile a program that should run on a 48kb ZX Spectrum, but you are running out of 33577 bytes.
That's an awful lot, I feel that z88dk starts the program at address 0x8000 by default, and I don't know how to reduce it to, for example, 0x60C0.

https://www.z88dk.org/forum/viewtopic.php?t=11567
From my point of view, the z88dk is terribly confusing.

But you are overflowing by much more bytes than moving the start could help you. The question is whether you should compile all 4 files together, because there must be some space for the source(user data) as well. If it is not, then the program uses different parts gradually. Editor, linker, compilation.

Which are recorded and used separately from the tape together with the data written by the user.

PS: It should contain the compilation script directly.
https://github.com/markdryan/specasm/bl ... /README.md

Building Specasm
Specasm is built with z88dk and GNU Make. Install a recent version and then

cd build
make -j
And then wait. All the tap and dotx files will be created in the build directory.

To create a zip file with all the files that need to be copied onto the spectrum, type

make release
from the same directory. The specasm.zip file can be found in the build/release folder.

The saexport, saimport and salink commands can be built and run on POSIX compatible systems. Simply type make from the main Specasm directory. The saimport is essentially the assembler without the editor, so can be used in conjunction with salink to assemble and build Spectrum programs directly on a modern machine, but where's the fun in that?

Spoiler

Code: Select all

VPATH = src

BASE =\
	error.c \
	state_base.c

COMMON =\
	line_common.c

SRCS =\
	editor.c \
	editor_tests.c \
	editor_test_content.c \
	line_parse.c \
	line_dump.c \
	peer_unit.c \
	peer_posix_screen.c \
	peer_text_screen.c \
	state_dump.c \
	state_parse.c \
	test_content.c \
	unittests.c

POSIX = \
	peer_posix.c \
	peer_file_posix.c \
	peer_posix_screen.c \
	peer_text_screen.c

SAIMPORT =\
	line_parse.c \
	saimport.c \
	state_parse.c 

SAEXPORT =\
	line_dump.c \
	state_dump.c \
	saexport.c

SALINK =\
	salink.c \
	expression.c

TEST_CONTENT_ZX =\
	test_content.c \
	test_content_zx.c

CFLAGS += -Wall -MMD -DUNITTESTS -Isrc

all: unittests saimport saexport salink

unittests: $(BASE:%.c=%.o) $(COMMON:%.c=%.o) $(SRCS:%.c=%.o)
	$(CC) $(CFLAGS) -o $@ $^

saimport: $(BASE:%.c=%.o) $(COMMON:%.c=%.o) $(POSIX:%.c=%.o) $(SAIMPORT:%.c=%.o)
	$(CC) $(CFLAGS) -o $@ $^

saexport: $(BASE:%.c=%.o) $(COMMON:%.c=%.o) $(POSIX:%.c=%.o) $(SAEXPORT:%.c=%.o)
	$(CC) $(CFLAGS) -o $@ $^

salink: $(BASE:%.c=%.o) $(POSIX:%.c=%.o) $(SALINK:%.c=%.o)
	$(CC) $(CFLAGS) -o $@ $^

test_content_zx: $(TEST_CONTENT_ZX:%.c=%.o)
	$(CC) $(CFLAGS) -o $@ $^

clean:
	- rm *.d *.o unittests saimport saexport salink

-include $(BASE:%.c=%.d)
-include $(COMMON:%.c=%.d)
-include $(SRCS:%.c=%.d)
-include $(POSIX:%.c=%.d)
-include $(SAIMPORT:%.c=%.d)
-include $(SAEXPORT:%.c=%.d)
-include $(SALINK:%.c=%.d)
-include $(TEST_CONTENT_ZX:%.c=%.d)

[Seven.FFF]

DOTN is not a separate file type at all, it’s just a dot command. That’s probably why you’re not finding much info.

The main thing to know is that, unlike standard esxDOS which only allows dot commands to be max 8KB (filling one page of divMMC RAM mapped between $2000..3FFF), NextZXOS allows them to be of unlimited length.

The first 8KB is loaded at $2000, and the file handle is left open with the read position pointing to the first byte of the remainder. NextZXOS has an API call to find out the open handle number, and API calls to allocate and deallocate 8KB banks of unused main SRAM. On the Next any 8K SRAM bank can be mapped into any of the eight 8KB slots of CPU address space.

A DOTN dot command is one that allocates as many banks as it needs, loads the rest of the command (which could be code or data) into those banks, maps the banks as it sees fit during the rest of execution, then before exiting deallocates the banks again and restores the original paging configuration.

In this way it’s possible to run a large complex program and still return to BASIC leaving the main 48K RAM completely untouched. The only consideration is that BASIC programs that want to call such large dot commands and also use extra banks themselves (NextBASIC can natively use up to 1.75MB of main SRAM to hold programs and data, unlike 128K BASIC where less than 48K is directly usable), will need to allocate their extra banks dynamically (there’s a BASIC statement to do that) rather than hardcoding unallocated bank numbers that might get stomped on.

However… DOTN is really only a different C runtime in z88dk, set up for convenience to make large dot commands easy to create from standard template code. The CRT does a lot of the heavy lifting of the setup and exit cleanup code. They’re not distinct file types at all, and the OS treats all dot commands equally, without caring what code is inside.

When I write large dot commands in asm, I do all the allocation, loading, mapping, deallocation and bank restoring myself manually. I use sjasmplus or Zeus-ish assemblers, and all they need to support is directives to ORG and DISPlace assembled code to the addresses it is going to run at, and to emit a binary file of the final dot command.

If you’re supporting assembling dot commands then that’s really all you need to do, and the rest would be up to the dev writing the command. If you want to do something similar to the heavy lifting that the Z88dk CRTs do, that’s much more complicated and you should probably study the CRT sources.

[Seven.FFF]

Oh lol, I went back and reread your question this morning. You're not trying to make your assembler generate DOTN commands, you're trying to build your assembler as a DOTN command! While my explanation of how they work under the hood might be interesting, it doesn't help answer your question. I'll see if I can rustle up some specific help or example code.

[Seven.FFF]

@markusr I'm posting this on behalf of Xalior who doesn't post here.

He says he wrote https://github.com/StalePixels/trampoline_examples with examples of how to use "unlimited executable memory" for DOTN and NEX files, building from the same codebase even.

If you want more direct 1:1 help, he's hanging out on the Next Discord in the #c-and-z88dk channel, and would be happy to assist and talk through it further. Invite is here: https://discord.gg/xgG7xyNY56

[Xalior]

Thanks @Seven.FFF (I found my password, go me!)

@markusr That code is a wee bit old, built for a slightly older version of z88dk than the current, so it you get build problems (they changed some defaults, and the commandline switches around them) please do come poke me on the link shared above, I'm active most days, and will happily work with you to help you understand that, if it's an issue, and to help get your assembler on the Next.

(Feel free to reply to this, too, but I won't see it, unless Robin comes and finds me, and drags me in to see it... like he did this post )

-Dx

[markusr]

Thanks @Seven.FFF and @Xalior . That github repo was exactly what I was looking for. I'll have a good study of the code and makefiles and will ask any further questions on discord. I'll report back here once I get it all to work, with a summary of what it was I did.

[markusr]

Well, I've got it all to work, mostly. I managed to locate three blocks of code in both the linker (salink) and the editor (specasm) that were self-contained, in that they had only one or two entry points. I compiled these separately associating each of them with a different 8 KB bank. I then created trampoline functions for all the entry points which mapped in the required bank before calling the real function. I had to make a few other changes to the text plotting code and to the keyboard handling to reflect the fact that the code is now part of a dot file and is running at a higher clock speed, but that was mostly it.

I still need to do some more testing (just realised this evening that I'm encoding push imm16 incorrectly) and update the documentation, so the new release is still a couple of weeks out, but it is coming.

[Luzie]

...As .launcher is not supported on the Next

Hi. There´s already a .launcher DOT-Command for esxDOS as can be read here: https://board.esxdos.org/viewtopic.php?pid=233#p233
Maybe you could rename yours to e.g. .launch to separate them a little bit?

[markusr]

Hi. There´s already a .launcher DOT-Command for esxDOS as can be read here: https://board.esxdos.org/viewtopic.php?pid=233#p233
Maybe you could rename yours to e.g. .launch to separate them a little bit?

I think maybe there's some misunderstanding here. I don't have a .launcher dot command. Specasm uses the esxDOS .launcher program to allow its main binaries, specasm.tap and salink.tap, to be launched from the command line, as dot commands, e.g., .specasm. In the Next version of Specasm, which I'm currently preparing, these two tap files will become .dotn files and so .launcher won't be used at all.

[Luzie]

I think maybe there's some misunderstanding here. I don't have a .launcher dot command. Specasm uses the esxDOS .launcher program to allow its main binaries, specasm.tap and salink.tap, to be launched from the command line, as dot commands, e.g., .specasm. In the Next version of Specasm, which I'm currently preparing, these two tap files will become .dotn files and so .launcher won't be used at all.

Oooh. Sorry. I not read the entire posts and from the little I´ve read, I thought you´ve created a .launcher DOTN-Command. So all good here and now

ps. There´s infos on .DOTN-Commands at SpecNext Forum: "Dot Commands - the final word": https://www.specnext.com/forum/viewtopic.php?t=1257