dsForth/TR-DOS

Show us what you're working on, (preferably with screenshots).
User avatar
ketmar
Manic Miner
Posts: 207
Joined: Tue Jun 16, 2020 5:25 pm

dsForth/TR-DOS

Post by ketmar »

this is recompiled and slightly optimised AberSoft FIG-Forth compiler. actually, it is not FIG anymore, but something inbetween of FIG and Forth-83. it also has words to read/write TR-DOS disks (using sectors as blocks), and simple text file reader from normal TR-DOS disk.

i did most of the work ~15 years ago, using custom decompiler to get the source code, and custom assembler to recompile it. then i forgot about it all, and only found it several days ago. so i took the source, added some code to my UrAsm assembler, and made the whole thing working again.

why it may be interesting: first, it comes with full source code, so you can customize it for your needs. second, the original compiler was located too close to BASIC, so there was no room for TR-DOS buffers. third, it is using custom 6px wide char font (i.e. 42 chars per line). i also did some small optimisations, added some useful words, and replaced simple line input from the original with much better line editor.

this thing is not finished (it is fully working, but TR-DOS interface is very simplistic, and i planned to add more extensions), but i don't think that i will develop it further. so instead of letting it rot on my HDD forever, i decided to make it public. maybe somebody will use it as a base for ZXNext forth compiler, for example. ;-)

in the archive you will find full source code, win32 UrAsm assembler, and compiled .TAP/.SCL (so you can try it in your favorite emulator).

dsForth v0.0.1.
6 x

azesmbog
Berk
Posts: 47
Joined: Sat May 16, 2020 8:43 am

Re: AberForth/TR-DOS

Post by azesmbog »

Good job!
Jupiter Ace has a built-in Forth.
How compatible is this Forth with that?))
0 x

User avatar
ketmar
Manic Miner
Posts: 207
Joined: Tue Jun 16, 2020 5:25 pm

Re: AberForth/TR-DOS

Post by ketmar »

dunno. ;-)

this one is close to Forth-83 standard, with some FIG remnants. there were plans to add menu system, fullscreen block editor, sprite system, better 128K support, and so on, but you prolly know how it usually goes with most good plans... ;-)

i decided to publish it because it contains full (and relocatable) source code for AberSoft Forth system. it is quite good, and was used as the base for White Lightning, btw (which in turn is a predecessor of Laser Basic ;-). i spent alot of time decompiling it, and cleaning it up, so i just cannot stand the fact that nobody will know how i used to wasting my youth. ;-)
1 x

User avatar
ketmar
Manic Miner
Posts: 207
Joined: Tue Jun 16, 2020 5:25 pm

Re: AberForth/TR-DOS

Post by ketmar »

and if you think that it is very easy to simply dump the project and forget about it... NO! here's dsForth v0.1.0!
download: dsForth v0.1.0

brief changelog
SpoilerShow
v0.1.0
* added name hashing, so compiler is MUCH faster now (~3.3 times faster)
* removed USER variable area (nobody needs it anyway)
* removed `WIDTH` variable (who cares)
* made numeric comparison code slightly faster, and added some handy comparisons
* renamed "U*" to "UM*", "U/MOD" to "UM/MOD", added "U*", "U/", "U/MOD", "UMOD"
* added `(DI)`, `(EI)` and `(HALT)` words
* added DRAWCIRCLE
* added `DEFER` and `TO`
* added "VALUE" word to create immediate values (like constants) which can be changed with "TO"
* added 8x8 printing driver
* added `SET-EMIT6` and `SET-EMIT8` words to switch between print drivers
* 6x8 printing driver can set attributes now
* added INK/PAPER/BRIGHT/FLASH words
* added USQRT (16-bit sqrt)
* added RANDOM (and other PRNG supporting words)
* added "stick reading" words to read configurable "control stick" (opqam, kempston, etc.)
* added some other words ;-)
* added textual error messages
* made alot of things configurable (see "main.zas")
* added sample game ;-)
* added "UPINPUT?" system flag (default is true, "WORD" will upcase space-delimited words)
* added "RECURSE" word
* added very simple superinstruction optimiser (for now, only for `<num> <@|!|+|->`, and only if words are on the same line)

to play the sample game, load "main.scl" into your favorite emulator, and type
" GAME" TLOAD
be patient, loading is quite slow (it takes around 20 seconds).
in the previous (original) compiler version it was 70 seconds, tho.

to see some preliminary sprite engine demo, type
" REX" TLOAD
then type
REX
please, note that this is a WIP!

type
" LIME" TLOAD
to load random limerick generator
the first dsForth version was almost unmodified fig-Forth system. i had huge plans for it, but it didn't happened in 2005. dsForth v0.1.0 is almost what i envisioned back then. some things just had to be done. better late than never, i guess. ;-)

the most noticeable change in compiler is adding a simple hashtable for dictionary words. the system has almost 500 words, and simple linear list just cannot work good with that. note that the interpreter has to search the whole list before it can decide that the given word is a number (because our dictionary can easily have words named like `0`, for example). another full search is done on defining a new word, because the system wants to warn the user about word redifinition.

also, most low-level and common words are defined early, so they reside near the end of the dictionary (as we are searching the dictionary from the last defined word).

so the innocent-looking numerical data array is slowest to compile, and simple words with standard math and stack manipulation are second to that. the sample game from the Mastertronic distrubution can take as much as 80 seconds to compile -- several times slower than loading the source code from the tape!

in dsForth v0.1.0 i added a very simple hash table with 64 buckets. this way i wasted 128 bytes for the table itself, and ~2*480 bytes for bucket links. but that 1KB allowed me to speed up the compiler by ~3.3 times! now that sample game compiles in ~21 seconds, which is much better.

the hash function is very simple rrca/add. surprisingly, it gives quite good distribution, with average 7 words per bucket. that is, instead of checking all 480 words, the system only checks about 7-10. bigger hash tables were only marginally faster, so i decided on 6 bit hash.

i also optimised multiplication and division by unrolling them, and rewrote almost all 16-bit comparisons as assembler primitives.

another change is switching from relative jump destinations to absolute. i don't know why FIG system choose relative addressing: it never moves the words around, and absolute addressing is slightly faster.

i also removed "USER" variables. this is good thing for multitasking, and especially for supporting several terminals with independent input/output, but has a little use for Speccy. IX register was used to keep user area address, so it becomes free, and i used it as return stack pointer instead. slightly more speed, slightly less code in several places.

after adding "DEFER" word, i got an easy way to vectorize various subsystems. think of it as a kind of "virtual function": each time the defered word is called, it reroutes execution to the word it was set to. this is slightly slower than the normal word, but the effect is neglible.

after getting such powerful tool, i converted printing subsystem to DEFERed, and now you can easily switch between 32 and 42 chars per line modes with "SET-EMIT6" and "SET-EMIT8". what is even better, the system only have to change around 7 defered words. all other code is using defered primitives like "print a character", and doesn't need to know the low-level details.

i also added "VALUE" variables. you don't have to use "@" word to get a value of a "VALUE" ;-), i.e. they're look like CONSTANTs. but you can use "TO" word to assign a new value to such variable.

another addition was USQRT (fast integer square root), and PRNG (Marsaglia one, as implemented by Patrik Rak).

i added some words to read "joysticks" (keyboard, sinclair, kempston), with the ability to easily redefine keyboard keys, and some words to work with sprites. note that you can use the ordinary 32x24 driver to print sprites too, and it supports three printing modes: overwrite, OR, XOR.

original FIG system expects to have error message texts on an external storage, and without such storage the only thing you had was a non-descriptive error number. i decided to waste some memory for error texts.

internally, the source code splitted to alot of modules, and you can control which modules you want in your system with defines in the "main.zas" file. this way you can build the system with slower, but smaller multiplication and division routines, omit 32 or 42 printing drivers, avoid compiling sprite, stick, or some other words. you can also turn off dictionary hashtable, to trade memory for speed.

of course, you can select default printing driver with compiler options, and completely omit the driver you don't need. (but you have to have at least one! ;-)

i also impletented very simple text file reader for TR-DOS disk system. it's not the best code for now, but it does its work. it should be easy to replace it with +3DOS, for example, but i never had Speccy with +3DOS support, and my emulator cannot even emulate it (althrough FDC emulator is there). maybe one day i'll take a look at +3DOS and will implement the corresponding interface.

one of the latest additions is superinstruction optimiser. it is very simple, but hey, why not? it optimises things like "32 +".

internally, numbers are compiled as the word "LIT", followed by the number. i created several superinstructions like "LIT+", and the compiler is able to use them. so our "32 +" will be compiled to "LIT+ 32" instead of "LIT 32 +". this is both smaller, and faster.

currently, only "+", "-", "@", "!", "=", and "<>" are optimised in this way.


i fully understand that dsForth won't have any users (it is late by several decades), but i wanted to finish what i started 15 years ago. ;-)

i don't know yet how far this project will go. working on dsForth is fun, and i am pretty sure that there will be at least one more vestion, but don't expect this to become a "commercial-grade development system". ;-)

please, note that the code is not optimised to my usual speed and size standards. it is "good enough" for what it does, but there's always a way to optimize any routine by at least one byte. ;-)

i used my UrAsm to build the sources because this is the only Z80 cross-assembler i ever used, and it was easier to extend it instead of learning yet another one. besides, i love to write my own development tools. that's why i have my own cross-assembler, my own emulator, and so on. ;-)

don't worry, win32 UrAsm version is included in the distribution. the source code is here. this is the combined repository for UrAsm and dsForth.


p.s.: althrough i used commercial AberSoft Forth system as a starting point, i don't think that there's alot of the original code left. i mean, most of the code was taken by AberSoft from public domain fig-Forth anyway, and i rewrote almost all complex assembler primitives. i put the whole thing under GPL, and i'm not making any money with it ;-), so i think that it's ok. if you'll write some software with this system, i'll be happy to see your sources GPLed too, but i won't go after you if you will decide it other way. i put GPL mostly to make my intentions clear, and not to force it on you (in this particular case ;-).
2 x

User avatar
ketmar
Manic Miner
Posts: 207
Joined: Tue Jun 16, 2020 5:25 pm

Re: dsForth/TR-DOS

Post by ketmar »

eh. expect at least two new versions. very rough roadplan: bugfixes (as usual); improving Forth-83 compatibility (but don't expect full 83 or ANS standards support); z80 asm+disasm (disasm is already there, asm is in progress); some way to save the current image as a new executable (with the option to start it with a custom word); more code for graphics; maybe some way to dynamically load and unload parts of the system (like asm, for example); unlikely, but possible -- creating "turnkey apps" by removing unused words and word headers (will need 128K to work, but the resulting app would be able to work in 48K mode); VERY unlikely -- improved documentation. ;-)

the secret is that i had little oportunities to use Forth, and i like writing Forth code. of course, the joy may wear off much sooner than i'll finish all those planned things, so no promises here.

p.s.: i found some use for dsForth! (at least for me). it is ideal for trying various game ideas and writing prototypes. doing so in assembler distracts me with various details and microoptimising, basic is out of question due to its speed, zdcc is boring (i have enough C on other machines! ;-), but Forth seems to be the ideal fit. it is much faster than basic (it is completely possible to write a decent platformer fully in Forth, i guess -- will try it ;-), i can easily write some low-level words in asm if needed, the code is still high-level (some liars keep saying that Forth is write-only language; don't listen to them! ;-)... i mean, i found an excuse to keep improving dsForth. especially now, when my 16K game project collapsed (boring to play, no memory to make it more fun).
0 x

Nienn Heskil
Dizzy
Posts: 61
Joined: Tue Jun 09, 2020 6:14 am
Location: Russia
Contact:

Re: AberForth/TR-DOS

Post by Nienn Heskil »

doing so in assembler distracts me with various details and microoptimising
What about using 'macro-fu'? If you're good enough with it, C is literally unnecessary[1], as you can have nearly that level of convenience, but without having to bring in its whole stupid infrastructure and whatnot (and with full advantages of ASM, of course). I really dig the 'Lua+macro' combo these days (sjasmplus), the possibilities are downright limitless.

[1] Although it is unnecessary either way. :)
0 x

User avatar
ketmar
Manic Miner
Posts: 207
Joined: Tue Jun 16, 2020 5:25 pm

Re: AberForth/TR-DOS

Post by ketmar »

Nienn Heskil wrote:
Mon Jul 27, 2020 7:44 am
doing so in assembler distracts me with various details and microoptimising
What about using 'macro-fu'?
UrAsm can do quite advanced macros, but for me it defeats the reason to use asm in the first place. i don't feel like i'm doing asm anymore. so i limited my macro usage to replacing copypasta like "downhl" with a macro call. using macros to emulate some high-level structures in asm is simply taking out all the fun (at least for me).

on the other side, Forth is a high-level language. and it is still "close to the metal" enough to be fun.

a little bit of history: my Forth adventures actually started with that Speccy AberForth system. i got it with some other utilities, and couldn't made heads or tails of it. it looked like interesting language, but the only thing i could get from it was "? error". then i found a russian translation of excellent Leo Brodie book "Starting Forth", and it was The WOW! Moment. that's why dsForth is based on AberForth.
0 x

Nienn Heskil
Dizzy
Posts: 61
Joined: Tue Jun 09, 2020 6:14 am
Location: Russia
Contact:

Re: AberForth/TR-DOS

Post by Nienn Heskil »

ketmar wrote:
Mon Jul 27, 2020 7:59 am
UrAsm can do quite advanced macros, but for me it defeats the reason to use asm in the first place. i don't feel like i'm doing asm anymore. so i limited my macro usage to replacing copypasta like "downhl" with a macro call. using macros to emulate some high-level structures in asm is simply taking out all the fun (at least for me).
OK, different approaches/priorities, I suppose.
For me, there are bits where performance isn't too important, and where, by the time I get to it, I sure wouldn't want to be thinking in terms of 'which bit of IX does what', 'how many ADD A I put here' and so on. For instance, game logic. This is where using some sort of HLL substitute makes perfect sense. Doesn't have to be ideal, just something that brings the amount of typing/written text (more or less the main problem with ASM) any closer to the way the thing you're after is expressed in our natural language.

Anyway, was just being curious.
0 x

User avatar
ketmar
Manic Miner
Posts: 207
Joined: Tue Jun 16, 2020 5:25 pm

Re: AberForth/TR-DOS

Post by ketmar »

Nienn Heskil wrote:
Mon Jul 27, 2020 8:30 am
OK, different approaches/priorities, I suppose.
yeah. ;-) i am having most fun from "doing it the hard (or unusual) way". getting the thing really done is not the main priority. ;-) but if i can find a way to make the process "fun in ketmar style", the chances of actually finishing something are much higher.

as for game logic... i am usually creating "micro-DSLs" anyway, sometimes with external compiler for them. i think that including full-featured programming language in UrAsm is overkill, it is much easier to create external DSL compiler from scratch (yeah, i love to reinventing the wheel! ;-). Forth looks like a perfect fit here. and i can translate subset of Forth code to (optimised) machine code later. or even left it as is, with its stack machine, why not.
0 x

User avatar
ketmar
Manic Miner
Posts: 207
Joined: Tue Jun 16, 2020 5:25 pm

Re: dsForth/TR-DOS

Post by ketmar »

i heavily reworked startup sequence and some parts of the system. it is now possible to build it with support for TR-DOS, +3DOS, or no DOS at all. ;-) also, "simple file reading" layer splitted to several parts, abstracting DOS functions away. it is now possible to call +3DOS API, but "middle layer" for file reading is not there yet.

tl;dr: next version should have support for loading files from +3DOS disks.

p.s.: maybe i'll create special version for +3 "RAM only" mode too. i don't really need any 48K ROM routines anyway.
1 x

Post Reply