dsForth/TR-DOS

[ketmar]

now UrAsm can create .SCL/.DSK files without external utilities (including direct copying of PC binary data files). and dsForth is able to load +3DOS forth sources from disk. here i'm usually writing "expect a new release soon", but each time i'm doing that, new releases are inevitably delayed. so i won't write that. ;-)

[ketmar]

and now UrAsm has simple, but working Forth system with Forth cross-compiler. you're now able to run native Forth code while compiling dsForth sources, or do something like this:

Code: Select all

" ./rex.x" ZX-VARBIN REX-SPRS
0 ZX-VARIABLE REX_SPR_TABLE  4 2U* ZX-ALLOT

[ZX-VARS]
: RR-BUILD-SPRITE-TABLE ( -- )
  ." building sprite table...\n"

  ( prepare sprite table )
  REX-SPRS REX_SPR_TABLE
  4 0 DO
    2DUP zx!  ( addr rspt ) ( [REX_SPR_TABLE] = addr )
    2+        ( addr rspt )
    ( skip sprite )
    SWAP      ( rspt addr )
    DUP zxC@ OVER 1+ zxC@  ( rspt addr cw ch )
    8 U* U* 2+ +           ( rspt addr )
    SWAP      ( addr rspt )
  LOOP
  2DROP
;
[NATIVE]
RR-BUILD-SPRITE-TABLE

here, "ZX-VARBIN" loads binary data to Speccy memory (the same as INCBIN asm directive), and creates Spectrum-side Forth variable "REX-SPRS" to hold the address of the loaded data. "REX_SPR_TABLE" is Spectrum-side too. "RR-BUILD-SPRITE-TABLE" runs as native assembler macro, tho, but works with speccy memory ("zxC@", etc.). as you can see, in "[ZX-VARS]" mode you can access Speccy Forth variables simply by names. "zxC@" is a hack, i will later tag addresses (will "mmap" Speccy memory to Forth VM memory), so the usual "@" will work.

but that's not all.

Code: Select all

ZX: HELLO REX_SPR_TABLE @ . CR " hi there!" CR ;

this new word starts with "ZX:", therefore it is cross-compiled to Speccy memory, using already compiled dsForth!

due to Forth simplicity, it took me ~1 day to write a working cross-compiler prototype. now dsForth can really be used to prototype my Speccy games. ;-)

ah, i also added "turnkey" compiler option. turning it on will create a stripped-down Forth kernel, without word headers, and omiting most of the compiler code. this shrinks the kernel from ~20 kb to ~10 kb, and it can be made even smaller. for now you have to manually assign "NOTURNKEY" flag to words, but in the future i may implement automatic tagging, so "turnkey image" will include only words really used by the app.


p.s.: yeah, theoretically that Forth cross-compiler can be developed to "full-featured solution", with build-in emulator and various tools, so you'll be able to debug your code on the PC, and in the same time profile it with the Speccy emu, for example. dunno if i'll take it that far, though -- right now i don't need such integrated solution, i'm ok with the simple cross-compiler. nobody's writing in Forth anyway, so there is little reason to develop something i myself don't need (yet). ;-)

[ketmar]

so i rewritten half of the thing (as usual), made it shorter, added conditional compilation and includes. i also cleaned it up, got rid of "zx@" and such (zx memory is "mmaped" onto VM memory now); made access to ZX forth variables transparent (as you don't need special "@" and "!" words anymore, there is no reason to use special modes for that); made zx/native mode switches affect many words (VARIABLE, VALUE, :, and so on), so even some interactive zx forth code can be compiled with minimal changes now. i am really happy that this thing have no users, tbh, because of its "organic design". i mean, it just naturally grows, throwing away old parts (and API/code compatibility with them ;-).

[mention]Nienn Heskil[/mention] remember me saying "i think that including full-featured programming language in UrAsm is overkill"? as usual, the moment i say "no, i won't do it", you can bet it will be done sooner or later. ;-)

[Nienn Heskil]

[mention]ketmar[/mention] Mmm sounds like a useful characteristic Perhaps there's some merit in reviewing my unreasonable demand lists...

It's nice when people are passionate about their projects, that much is clear.

[ketmar]

tbh, i never intended to develop dsForth further. so i was doomed to improve it the moment i wrote "but i don't think that i will develop it further" in the opening post. ;-) now i am really planning to use it for prototyping my games (i still have two ideas for 16K cassette; trying 'em with UrForth/dsForth should be much easier than with a pure asm).

[ketmar]

more progress: it is now possible to create "minimal" and "minified" dsForth builds. the difference is that in minimal build most of the kernel is there, the only things dropped are the parser and the compiler, making the kernel ~8 kb smaller than the full one. and in minified mode the main word is traced, and only most necessary parts of the kernel are included (~12/14 kb smaller than the full one).

it is also possible to exclude word headers from both special builds, saving about 2kb, but in this case it is impossible to properly trace the execution with ZXEmuT tools. with word headers, it is possible to print all executing words with ZXEmuT Tcl script, and without the headers there is simply nothing to print. ;-)

also, +3 DSK version now has an option to create a bootsector, for slightly faster startup time.

UrForth cross-compiler now performs simple superinstriction optimisations on ZX words (think of it as a kind of peephole optimiser). i am also planning to add more optimisations in the future (like dead code elimination, inlining, and so forth).

another possibility is subroutine threaded code kernel, and the ability to dynamically switch between two kernels. this needs more investigation, though.

[ketmar]

just for fun i wrote Z80 disassembler in Forth. it compiles from exactly the same source both in "native" mode (i.e. you can disassemble you assembled code while you're assembling it ;-), and in "zx" mode (i.e. for dsForth).

ok, i lied a little, there is one ifdef:

Code: Select all

$IFNATIVE
  [0x] 00 C,
$ENDIF

it can be removed too, i just didn't bothered to do it yet.

the disasm is opcode-fields-driven (exactly as Zymosis). of coruse, i could use a huge table instead, but it is not fun.


p.s.: Speccy code size is 4485 bytes. it can be made smaller, but i prefer to keep the code readable.

[+3code]

I think your Forth and that other

https://github.com/programandala-net/solo-forth

are the only one that will run on +3, that is good (sadly I'm not a Forth programmer my level is more for Logo or so). Good job and greetings

[ketmar]

thank you! yay, direct threaded code... something i planned to do too. i should take a look.

also, dsForth can work on 48K too. i am the best! ;-)

sadly I'm not a Forth programmer :( my level is more for Logo or so).

Forth is even easier than Logo. no, really, i still can't learn Logo, it is too complicated.

there is excellent Leo Brodie book Starting Forth. it is literally the best textbook you can get, written with humor and great skill, and it is free! give it a try, and chances are you'll become Forth programmer, and quite good Forth programmer. i know it, because this is the book i used to learn Forth! ;-) i bought Brodie book exactly to learn what to do with that funny AberForth Speccy program i had.

please, note that PDF version is for Forth-79 standard, html version is fixed for ANS (afair), dsForth was FIG, but now it is totally non-standard. not that it matters much, tho: the differences are quite small, and i am always ready to help.


p.s.: oops. i read "direct threaded code" as "subroutine threaded code". dsForth is DTC too, of course, so it should be of the same speed (there are only so much ways to write DTC interpreter ;-).

p.p.s.: Forth is so simple, that any (no joking, literally any) Forth programmer can implement Forth system from scratch on a new platform in several days. contrary to other languages, if you know how to program in Forth, you know how to write a Forth complier. because there is no magic it all, only very simple building blocks you will use to create your own programs too.

p.p.p.s.: yay! "Solo Forth started from the code of Abersoft Forth (a tape-based fig-Forth 1.1 for ZX Spectrum 48K), which was first studied and much improved (http://programandala.net/en.program.afera.html) and finally fully disassembled". looks like we had the same start. ;-)

[ketmar]

oops. i could never remember which one is direct threaded code, and which one is indirect. ;-) AberForth is using inderect variant, and so does dsForth. DTC is slightly faster, so i'm converting dsForth to DTC. it will prolly cost additional 100-300 bytes for the whole system. not a big deal, and simplified interpreter/better speed worth it.

expect a new release once i finish the transision.

[ketmar]

sorry for spamming with posts (this topic starting to look more like a developer blog, lol). so i finished converting dsForth to DTC. with DTC, each Forth word is one byte bigger, but each code word is two bytes smaller. it looks like they compenstates each other, so the image size is roughly the same with DTC. but the whole thing is ~1.2/1.5 times faster now. this is roughly the same speedup that Solo Forth got from the same transition. i believe that dsForth and Solo Forth are in the same speed league now (no benchmarking was done, though!).

p.s.: [mention]PeterJ[/mention] maybe we should create a subforum for "WIP projects"? "Brand new software" is good for announces, but the things like dsForth, with more-or-less constant progress reports only creates a useless noise there. and with "WIP projects" authors will be able to announce their projects, and give a link to development thread, if necessary. this way, "Brand new" will have only real announces and the corresponding discussion.

[PeterJ]

[mention]ketmar[/mention], the 'Showcase your work!" Subforum is the best place.

[ketmar]

@ketmar,the 'Showcase your work!" Subforum is the best place.

oh, my bad. then move this topic there, if you please, so i'll stop spamming "brand new" with (mostly useless) updates. ;-)

[PeterJ]

No problem at all [mention]ketmar[/mention]. The topic has been moved.

[ketmar]

thank you alot, [mention]PeterJ[/mention]! now i can post my hourly updates, and won't irritate people looking for a fancy new game to play! ;-)

so, DTC is working, both optimiser and tracer were updated to the new code format, and the cross-compiler is fully functional again. i also implemented autocompletion in line editor, so you can type some letters, and press SS+ENT to ask the system to finish the word for you. i'm so used to autocompletion feature that i always tried to use it in dsForth, but alas, it wasn't there... until now! ;-)

i believe that we're close to the new release now. i'm also planning to write several simple demo games to show you all that Forth (and dsForth) is not a toy, and it can do much more that that awful "game" from Melbourne House tape. i'm not expecting people going mad and start writing new games in Forth, but hey, it is worth trying anyway! ;-)

if only i had such system 20 years ago...

anyway, don't adjust your sets! ;-)

[ketmar]

added floating point support (with ROM calculator), including full-featured float<->string conversions (using calculator routines, so "42E18" can be parsed, and "4.2E+19" will be printed. i guess we're very close to 0.2.0 public release. ;-)

p.s.: don't expect the current development speed to stay the same after 0.2.0 release, tho. the system is more-or-less feature complete now, and for 0.3.0 release i need to complete various libraries, write documentation, and so on. the kind of work i don't really love to do.

[ketmar]

the huge delay with new release is due to... oh... i spent last week writing x86 forth from scratch, in asm and forth. ;-) UrAsm already has UrForth built in, but it becomes more and more limiting to have forth built into assembler, and not vice versa. ;-) the plan is to create a full-featured cross-compiler/emulator. yeah, it prolly means porting Zymosis to Forth (or maybe to x86 asm). ;-)

anyway, 0.2.0 will still be UrAsm-based. and i will backport new loops implementation, with full break/continue support. oh, the joy of Forth: the code from quite big 32-bit system can be taken to little Speccy as-is, because UrForth was made mostly compatible with dsForth.

p.s.: UrForth Native is fully working now (and passes ANS tests where i think it worth support ANS standard). but i want it to be self-hosted, so i wrote (ahem... borro... wrote!) a huge part of built-in x86 assembler already.

and if you want to know why i'm doing all that, considering that there are prolly at least a billion of really good native x86 forth systems... because i can, of course.

[ketmar]

so UrForth is self-hosting system now, which means that i have a working metacompiler framework. now i'm slowly doing natural-syntax Z80 assembler in it, and will use its metacompiler to cross-compile dsForth sources. all that means that dsForth project is not dead, and instead of being a mess of urasm and custom "almost-forth" syntax, it will be almost completely normal Forth, properly cross-compiled (with optimisiations, and better turnkey).

UrForth took slightly longer to write than i expected (3 weeks; i'm simply fine-tuning it now), due to writing natural-syntax x86 assembler in it first. that is, usual Forth assemblers are quite ugly, often looking like "4 [ebp] ecx mov". it is very easy to write the asm engine like that, but it is a PITA to use. so i wrote a custom parser and a "natural-syntax" asm, which accepts normal things, like "lea eax,[esi*4+edx+8]" instead.

i will do the same for Z80 assembler, and then will port current dsForth sources to the new system.

it means that there will be no 0.2.0 dsForth release, i'll jump staight to 0.4.0.

[pandoraefretum]

Hello can I take a look at this dsForth, please ?
Could I have a link please ? The previous links I clicked do not work...
Thanks, Roland

[ketmar]

Hello can I take a look at this dsForth, please ?
Could I have a link please ? The previous links I clicked do not work...
Thanks, Roland

sure, no problems!
here is the latest sources i have (and built binaries too). you will need my UrAsm project if you will want to rebuild it.

sorry for the state of the code: i was in the middle of rewriting the whole system on top of my new 32-bit x86 UrForth (which would be used as a testbed, and as a cross-compiler), but somewhat… ran out of steam. i'll eventually return to the project, but don't know when and how, especially now (i'm living in UA).

there is also experimental +3DOS version too (.dsk binary should work on +3).

p.s.: if you want to fork the project, or use some of its parts for your own, be my guest! ;-)

[Wall_Axe]

So you have cross compilation from PC to x86 dos system and zx spectrum?

Any .tap files of demos available of the games?

[ketmar]

nope, not yet. for various reasons i stopped working on it after creating a barely working proof-of-concept. basically, lack of time. but with new UrForth/C i may revisit this once again. the idea is still interesting, and optimising cross-compiler could be quite fast.

tbh, the main reason why i moved this project to "do it sometimes… later" is lack of users. i myself don't really need it in the finished form (i can use the pieces i already have), and most other ZX/Z80 Forth users seem to already have their own Forth systems (which is expected ;-). so now this is something i would do when i'll have some free time left from my free time. ;-)

[PeterJ]

@ketmar,

I'm very slowly working my way through the Melbourne House Abersoft Forth book, and this example works fine in the original Abersoft Forth



However, in your version there doesn't seem to be the graphics routines:



Am I doing anything wrong?

[ketmar]

dsForth is not really compatible with Aberforth anymore. this word is called "DRAWLINE" there, if i remember it right. ;-)

that is, instead of "sx sy plot ex ey draw" use "sx sy ex ey drawline".

"125 0 PLOT I 99 DRAW" should be "125 0 I 99 DRAWLINE"

[ketmar]

@PeterJ, also, please, note that dsForth is not fully FIG-Forth compatible too. for example, instead of "<BUILDS … DOES>" you should use "CREATE … DOES>". it also has some modern words too, like "VALUE" and "DEFER".

but "<BUILDS … DOES>" will return in the next version, removing it was a design mistake. ;-)