Abersoft fig-Forth Recompiled

[ketmar]

as i played with AberForth, i also created Yet Another decompiled version of it. there were other decompiled versions, but mine has something new to offer. ;-)

* rebased to $6000, so it can be used with TR-DOS (no TR-DOS support included yet).
* removed some unused FIG-Forth parts ("MON", some constants, some unused data).
* converted from ITC to DTC (it is slightly faster now).
* optimised some words.
* added some words i feel necessary (mostly small utility words).
* created several .TAP versions (minimal, with some additional words, and with EDITOR).
* the system will not bomb to BASIC on tape or print error anymore.
* other small improvements.

the recompiled version should be fully compatible with the original AberForth.

also, contrary to other decompiled versions, this one is using the fully featured target compiler, and Forth code is written in Forth, not in some pseudoasm. just look at "figsrc/" — this is the real Forth code now.

42-column print driver is not included, as i wanted to keep this close to the original.

download poc, version 001

WARNING! there may be bugs in decompiled code (my decompiler is far from perfect ;-). this is proof-of-concept, done in 2 days. ;-)

the archive includes full source code to the system ("figsrc/*"), target compiler source code ("tcom-*.f"), x86 GNU/Linux UrForth binary, and prebuilt .TAP files ("tap/*").

[ketmar]

i am preparing poc v2, btw, with many decompilation bugs fixed.

i managed to cut demo game compiling time from ~40 seconds to ~20 seconds. the change is very simple: i put LFA field before NFA.

i don't know why Bill Ragsdale decided to organize word headers this way, but the effect is that "(FIND)" has to scan each word name until a good one is found. this is because "LATEST" points to NFA, and "LFA" is put after "NFA". so to get to the next word in the list, the code has to skip NFA field, and it does this using byte scanning.

tl;dr: a lot of time was wasted for nothing. now compiling is MUCH faster.

DTC doesn't give a huge speedup on that test, by the way (less than a second). so i changed default to ITC — it is smaller. of course, it's possible to use "--dtc" CLI arg to build DTC version (and there is CLI arg to use old headers format too).

i also added 42 columns driver, selectable via CLI arg.

before publishing v2, i want to try STC first. i don't expect a huge speedup (and the code will be ~1/3 bigger, i believe), but it is worth trying anyway.

maybe i should extract White Lightning routines and make them optional. because WL is built on top of the same fig-Forth model, i only need to decompile their gfx routines (and IDEAL). hey, why not?

i may also add TR-DOS and +3DOS support, while i am at it. this will make the system usable on real hardware (because let's be honest, default ramdisc system sux ;-). this may very well become the next dsForth version (it was built on top of fig-Forth model too). and for +3 i could put the system in the first 16K (prolly with Z80 asm included), leaving the rest free for user code.

[ketmar]

i had to look at (and edit) the sample game. standard line editor sux, so i wrote the fullscreen one. and implemented 64 columns print driver for it. ;-)

screenshot

the editor works in any mode, though (32, 42, 64). it has line yank/insert, and nice incremental search.

by the way, it seems that sample game wasn't created on Speccy: it is tightly packed for 64-char lines. the source, of course, is completely unreadable even in 64-cols mode. ;-) also, it was definitely written by someone not expirienced in Forth, and it is good "how to NOT write Forth code" example. i prolly should write a better example game instead.

i also bumped i/o buffer size to 256 bytes in preparation for TR-DOS interface. of course, it wasn't as easy as set new "B/BUF" value: there were some hardcoded numbers in block subsystem. it was "blksize+2" in the original fig-Forth asm source, but that info was lost.

sadly, editor source cannot be made loadable on Speccy itself: it is using some headerless system words. i removed headers from some rarely used internal words to save some RAM (and the editor itself is mostly headerless too). but it is only 2.9KB, so i guess i can include it instead of the old line-oriented one.

[Wall_Axe]

It'll be cool to see the example game.
So if i read the manual to fig forth I would be able to use this?

[ketmar]

It'll be cool to see the example game.

it is the one supplied with Mastertronic re-release. that tape contains RAM-DISC image, you can load it with "LOADT", and then use "1 LOAD" to load the game. but beware: the game completely sux. ;-)

So if i read the manual to fig forth I would be able to use this?

yes. i am making the system faster, and adding some words, but keeping all fig-specific things intact. so you should be able to use either standard Abersoft manual, or any other fig-Forth description.

p.s.: there are some very minor changes (word header format, buffer size, and such). but if you are not modifying the system directly by poking bytes here and there, all fig-Forth code should work. just use the recommended words/constants instead of relying on magic numbers.

[ketmar]

ported ROM floating point library from dsForth. for ~1KB of code you'll have full floating point support, backed by ROM calculator. because why not? ;-)

i know, i know, Chuck Moore said that floating point math is for wimps. but hey, i don't have to write my own FP code in asm anyway! ;-)

[ketmar]

fun fact: for some reason White Lightning has around 40 hidden words. they are not linked to any vocabulary, but have valid headers and names (and they are referenced from other, visible words).

it looks like that system wasn't cross-compiled from some asm source code, but created in several stages. first, the standard fig-Forth nucleus and high-level words were done. and then WL-specific words were loaded on top of this system from some source code — because hidden words have valid link fields, and it looks like they were simply "unlinked" from the word list by some command.

or maybe WL was created with the target compiler which does word hiding in such a funny way.

other interesting thing: WL fig model doesn't have "EDITOR" vocabulary: editor words simply put into "FORTH" vocab. dunno why.

now i have to improve my decompiler to get the proper sources out of the snapshot, because i am interested in this fig variant.

btw, WL line editor sux. it is so unreliable (at least on my emulator) that it's really hard to type anything there. i wonder if it was such bad on the real hardware too. if it is, then i can understand why WL never took off. ;-)

p.s.: Abersoft fig-Forth has several properly hidden words (without headers), mostly related to kb input and screen output. it looks like in WL fig model those words (with different code, though) have their headers intact.

[ketmar]

hm. i decompiled Forth part of White Lightning, and it looks quite interesting. the usual fig model code is intact (of course), along with several standard fig inefficiencies.

but the added code looks way more interesting. for exampe: it contains two "AT" implementations, absolutely identical, yet the earlier one is hidden for some reason. it looks like they at least partially hotpatched updates over the existing system.

there are also some remnants of disk access (only several variables). those variables are hidden too.

sadly, most of the "laser library" is one blob of machine code, without any headers. i vaguely remember that there was a Laser Basic disassembly around. if somebody knows where to download it, it would be great. i believe that it is using the same "laser library" (even the commands are the same! ;-), and it would be interesting to compare different "laser lib" versions.

[ketmar]

just4fun, some current system stats.

Spoiler

build options:
threaded code type: ITC
header format: new
branch offsets: absolute
optimise by: speed
print driver: 64 columns
CLIT/BLIT: enabled
16-bit mul/div: slow
bit shifts: disabled
extra block words: disabled
editor words: disabled
fullscreen editor: enabled
floating point lib: enabled
.HEX2/.HEX4: disabled
Forth R0: $FB40
Forth S0: $FAA0
ZX I/O BUFFER FIRST: $FB40
ZX I/O BUFFER LIMIT: $FF50
ZX I/O BUFFER: 256 bytes.
ZX I/O BUFFER FULL: 260 bytes.
ZX I/O BUFFER COUNT: 4
ZX I/O B/SCR: 4
print driver size: 2,445 bytes (~2.4 kb)
FP library size: 964 bytes (~0.9 kb)
cflow words size: 629 bytes (~0.7 kb)
FS editor size: 2,982 bytes (~2.9 kb)
MAIN SYSTEM size: 16,466 bytes (~16.0 kb)
statistics:
total word count: 669
header count: 462
headers size: 3,394 bytes (~3.3 kb)
CFAs size : 1,354 bytes (~1.3 kb)
code words : 159
mcode size : 2,877 bytes (~2.9 kb)
colon words: 361
tcode size : 7,895 bytes (~7.8 kb)
variable words: 0
constant words: 49
uservar words : 9
quan words : 14
vect words : 9
alias words : 14
saved by CLIT/BLIT: 174 bytes (~0.1 kb)

[Wall_Axe]

So it takes up about 26k?

[ketmar]

nope, "main system size" is total size (bad description, i know ;-). 16 kb with all bells and whistles. can be much smaller if various optional parts are turned off.

p.s.: note that even with RAM-DISC you'll have ~13 kb for your code (this is about as much as the original Aberforth had). and with proper disk system (or 128K version with RAM-DISC in another page) about 23 kb of RAM is free.

p.p.s.: i may create +3 version too, which is using the whole 64KB of memory, putting the nucleus into the first 16 KB, so you'll have all 48 KB free. ;-)

[ketmar]

also, there will be a target cross-compiler which creates "turnkey" apps, omiting all headers, and all unused words. but you'll need x86 GNU/Linux to use it.

[ketmar]

by the way. WIP version to play with:
download .TAPs.

WARNING! this is NOT final! it should work, but there may (and will) be some bugs.

there are 3 .TAP versions, all with demo game. WARNING! demo game works only in 32cols mode.
to load demo game, use: "LOADT 1 LOAD"

source code will be published later, it is VERY WIP yet.

brief changelog:

Spoiler

some notable changes from the original Abersoft fig-Forth system:

* added some useful words, like "-ROT", "NIP", and such.

* optional case insensitivity.

* "ok" prompt will show stack depth and current base (if it is
not decimal). input prompt also indicates current mode: ">" for
interpretation, and ">>" for compilation.

* implemented 42 and 64 columns versions. all drivers support standard
control codes to chage attrs, set cursor position, and such.

* all print drivers are using all 24 text lines now.

* 3 cursor types for "KEY" are supported now, controlled by "KCUR" var.
0 KCUR ! -- no cursor at all
1 KCUR ! -- the usual L/C cursor
$80 KCUR ! -- blinking bar

* keyboard input now emits clicks, as in BASIC.

* PLOT and POINT are able to use all 192 vertical lines now.

* implemented direct input of hex, octal and binary numbers:
$4000 -- hex
@667 -- octal
%10011001 -- binary

* added floating point library (using ROM calculator). FP words
are prefixed with "F" (i.e. "F+" is floating point "add").
FP has its own stack (but it is quite small, so beware!).
to enter FP literal use "F# 6.42" (can be used in interpretation
or in compilation mode).
to print floating point number, use "F.".
to push integer constants from the data stack, use "S>F" or "U>F".
to pop integer constant to the data stack, use "F>S" or "F>U".
full documentation will be provided later.

* it is now possible to use several "WHILE"s, and finish any BEGIN
loop with "UNTIL". i.e. this is valid now:
BEGIN ... WHILE ... WHILE ... UNTIL

* BEGIN loops support "BREAK" and "CONTINUE". you can also use "BREAK"
in "DO" loop.

* added "0IF", "0WHILE", "0UNTIL" words (slightly faster than "NOT xxx").

* added "UU* ( ua ub -- ua*ub )" -- fast 16-bit multiplication with
16-bit result.

* added "U/" and "UMOD" -- fast 16-bit division.

* "UU/MOD" is there too, but note that it is "UU/MOD ( ua ub -- ua/ub ua%ub)"

* included fast and quite good 16-bit PRNG with 32-bit seed: "URND".
seeds are "URND-S0" and "URND-S1". use "URND8" for 8-bit result.

[flange]

I'm enjoying this thread quite a lot as Forth is my thing for computation, Forth79 in my case on the ZX81 and it's been fun comparing the two.
Computation wise this is a fair bit slower than the ZX81 and not as fully featured and this latest version has introduced one bug at least which I suspect lies around the addition of the conditional operator <=. in your previous version I had to create a word for it and it worked well however I see it's now included but causes a crash when using large numbers in something I use to speed test 8-bit Forths. (Josephus problem. )

I look forward to the completion of this work and congratulations on working with a language not too popular universally you have my full support.

[ketmar]

@flange, thank you for your support!

yep, "advanced" comparisons are not properly tested yet, thank you. they weren't in the original, and i wrote the code in "blind mode", just to have something there. ;-)

as for not popular language… i am using x86 Forth system on daily basis to write GNU/Linux software (the same UrForth included with the first download). so it is popular amongst myself at least. ;-)

i have several Z80 Forth systems in development, including cross-compiler which generates mildly optimised native code. they all will be published sooner or later (with full sources, of course). for this one, i simply want to stick to fig model, so people could use Abersoft manual with it. others are deviated from any standard heavily.

p.s.: i'd like to see the code which crashes, if you please. will help me to test some corner cases i missed.

[flange]

When I say not popular what I mean is that when ever I bring the subject up people leave the room!

It will be nice to have a FIG model and as you say the manual is readily available.

Will this ever be available to use on real speccy or are you concentrating on emulator only?

The floating point is a nice useful feature and would say a full doubles extension would also be useful, but whatever you decide I will follow and use this with much pleasure.

[flange]

Just seen your PS so will post the code that crashes with explanation.

[flange]

The Josephus problem if you are not familiar with it can be found here...

https://en.wikipedia.org/wiki/Josephus_problem

Not a great problem but with large enough numbers a good speed test I have found.


Usually in this example a smaller number is used usually in double figures. This code worked in your first version however large numbers such as the example cause a reset after a short time in the second version.

Code: Select all

: josephus  0 1 begin dup 30000 <= while  swap 3 + over mod swap  1+ repeat drop  . ;

[flange]

Forgot to add that my compilation for <= was : <= > not ;

[ketmar]

funny, that's exactly how they are defined:

Code: Select all

: <=  ( a b -- a<=b? )  > NOT ;
: >=  ( a b -- a>=b? )  < NOT ;

[flange]

Then it must be something else causing a reset when using large integers in the Josephus problem, sadly I don't have the skills to solve this.

[flange]

Just to make certain I reloaded a fresh copy of the emulator (Fuse) and now it all works perfectly with large integers!!

I also tried two other emulators and all ok with them also so can only assume I had a corruption in the emulator as all is ok and to confirm I have just run a primes routine to find the first 30,000 or so primes and that too works ok. Phew!!

[_dw]

I rewrote the N-body problem on floating point numbers. So it is possible to test whether the results match.

The Z-variant is made into a separate stack for floating point numbers. I tested it on a floating point ZX ROM 48kb.

The F-variant is done if the number occupies exactly one cell like a normal number (probably 16-bit) and does not have a separate stack.

https://codeberg.org/DW0RKiN/M4_FORTH/s ... body_z.fth
https://codeberg.org/DW0RKiN/M4_FORTH/s ... body_f.fth

[ketmar]

Will this ever be available to use on real speccy or are you concentrating on emulator only?

it should work on the real hw, why not. i don't have real machine anymore, though, so any bug reports from reals are welcome!

The floating point is a nice useful feature and would say a full doubles extension would also be useful, but whatever you decide I will follow and use this with much pleasure. :D

i am simply using ROM calculator for FP, it costs me nothing. writing a full-featured floating point library from the scratch is a very different thing. but if you have some ready-to-use FP library in mind, i can add it as optional module, why not.

i prolly need to implement transient vocabularies, to allow loading such extensions on the Speccy itself. that is, Z80 asm is quite big (around 5 kb, i guess), but you only need it while compiling, and it can be thrown away later. so there should be a way to load it into another memory area, then throw it away. this way i can turn most extensions to loadable source code.

a small problem is that fig (or any other Forth system except UrForth ;-) is not well suited for the task. UrForth Z80 assembler, for example, allows declaring labels simply as "label-name:" (simply end it with colon), and use labels before declaring them. just like Real Asm, you know. ;-) this is done via standard (for UrForth) vocabulary extension mechanics. but fig doesn't have that. it doesn't even have a vocabulary stack (which i am heavily using). of course, it is easy to add, but then it won't be fig system anymore…

[ketmar]

by the way, contented memory sux! Erathosthenes test is ~1.7 seconds faster when the system is uncontended.

but i am not ready to waste 8KB of RAM by moving the system to $8000 yet.