Forth compiler for ZX

[_dw]

that you can throw all kinds of socks to the wall, and watch which will stuck.

From which language is this idiom and forgive my stupid question, but how should I understand it.


But that standard is something that makes it easier to switch to M4 FORTH, so I should stick to it on the other hand if possible.

I came across a web page with UNCONTROLLED FORTH-83 STANDARD where the word 1+ is! and 1-!.

Code: Select all

          1+!          addr --                            "one-plus-store" 
               Add one to the 16-bit contents at addr.

          1-!          addr --                           "one-minus-store" 
               Subtract one from the 16-bit contents at addr.

I have not found these words on other FORTH-83 STANDARD pages.

In the "modern" standard there is only the word +!. I didn't see anything in OPTIONAL either. But I would also like -!. Which will answer "swap negate swap -!". So the alternative is 1-!. I originally introduced only +! because it wasn't entirely clear what was to be subtracted from what, but practically you want to have an analogy with the 1-!. It's logical and clear to understand, and it saves code size.

I wrote something about the comparison between C and FORTH and in C I add a boolean value to the address in memory. Just a 16 bit variable stored in memory. But C has TRUE as 1 and FORTH has TRUE as -1, so it is useful to use -!.

By the way, my first entry was something like: "addr @ swap - addr !"

Code: Select all

dworkin@dw-A15:~/Programovani/ZX/Forth/M4$ ../check_word.sh 'PUSH(addr) FETCH SWAP SUB PUSH(addr) STORE'
    ld    A, [addr]     ; 3:13      addr @ swap -   ( x1 -- x2 ) x2: [addr]-x1
    sub   L             ; 1:4       addr @ swap -
    ld    L, A          ; 1:4       addr @ swap -
    ld    A, [1+addr]   ; 3:13      addr @ swap -
    sbc   A, H          ; 1:4       addr @ swap -
    ld    H, A          ; 1:4       addr @ swap -
                        ;[5:30]     addr !   ( x -- )  addr=addr
    ld  [addr], HL      ; 3:16      addr !
    ex   DE, HL         ; 1:4       addr !
    pop  DE             ; 1:10      addr !
; seconds: 0           ;[15:72]

But I didn't liked it, because it was worse than the original recording with +!. Where I simply stores it as a negative value.

Code: Select all

dworkin@dw-A15:~/Programovani/ZX/Forth/M4$ ../check_word.sh 'PUSH(addr) ADDSTORE'
    ld   BC,[addr]      ; 4:20      addr +!   ( x1 -- x1 ) [addr] += x1
    add  HL, BC         ; 1:11      addr +!
    ld  [addr], HL      ; 3:16      addr +!
    ex   DE, HL         ; 1:4       addr +!
    pop  DE             ; 1:10      addr +!   ( a -- )
; seconds: 0           ;[10:61]

This is what a record with negation would look like. I had to chop it up with an assembler insert token so it wouldn't optimize from "swap negate swap +!" on "-!"

Code: Select all

dworkin@dw-A15:~/Programovani/ZX/Forth/M4$ ../check_word.sh 'NEGATE __ASM PUSH(addr) ADDSTORE'
    xor   A             ; 1:4       negate
    sub   L             ; 1:4       negate
    ld    L, A          ; 1:4       negate
    sbc   A, H          ; 1:4       negate
    sub   L             ; 1:4       negate
    ld    H, A          ; 1:4       negate

    ld   BC,[addr]      ; 4:20      addr +!   ( x1 -- x1 ) [addr] += x1
    add  HL, BC         ; 1:11      addr +!
    ld  [addr], HL      ; 3:16      addr +!
    ex   DE, HL         ; 1:4       addr +!
    pop  DE             ; 1:10      addr +!   ( a -- )
; seconds: 0           ;[16:85]

And this is how the code for "-!" if we know one parameter.

Code: Select all

dworkin@dw-A15:~/Programovani/ZX/Forth/M4$ ../check_word.sh 'PUSH(addr) SUBSTORE'
    ld   BC, addr       ; 3:10      addr -!   ( x1 -- x1 ) [addr] -= x1
    ld    A, [BC]       ; 1:7       addr -!
    sub   L             ; 1:4       addr -!   lo
    ld  [BC],A          ; 1:7       addr -!
    inc  BC             ; 1:6       addr -!
    ld    A, [BC]       ; 1:7       addr -!
    sbc   A, H          ; 1:4       addr -!   hi
    ld  [BC],A          ; 1:7       addr -!
    ex   DE, HL         ; 1:4       addr -!
    pop  DE             ; 1:10      addr -!   ( a -- )
; seconds: 0           ;[12:66]
dworkin@dw-A15:~/Programovani/ZX/Forth/M4$

+! is still more efficient than -!, but there is some saving in bytes and clocks.

I don't really understand the drafters of the standard.
Why only have a quote? It's like in the programming language Karel, who only knows LEFT-SIDE. And you have to program RIGHT-SIDE (RIGHT-FACE I'm not sure what the expression is in English for yelling at the soldiers to turn 90 degrees to the left) via 3x LEFT-SIDE.

Code: Select all

: -! swap negate swap +! ;

Easy, but did everyone do it the same way? Wouldn't it be better to have -! in extended standard?

PS: In reality, those commands are more complicated to interpret, because the PUSH_SUBSTORE connection breaks down, I have DUP_PUSH_SUBSTORE and DROP. Something like in the physics of small particles... .)
And DROP can easily be connected to the next word that comes up.
If I turn on detailed listing via VERBOSE(1)

Code: Select all

dworkin@dw-A15:~/Programovani/ZX/Forth/M4$ ../check_word.sh 'VERBOSE(1) PUSH(addr) SUBSTORE'
   ...new __TOKEN_PUSHS(addr) "{addr}" --> token(1) __TOKEN_PUSHS(addr) "{{addr}}"
   ...new __TOKEN_SUBSTORE() "-!" --> token(2) __TOKEN_DROP "__dtto"

   ...check all tokens
   ...second pass(1) __TOKEN_DUP_PUSH_SUBSTORE(addr) "{addr} -!" --> __TOKEN_DUP_PUSH_SUBSTORE(addr) "{{addr}} -!"
   ...second pass(2) __TOKEN_DROP() "__dtto" --> same

   ...check all tokens2
   ...third pass(1) __TOKEN_DUP_PUSH_SUBSTORE(addr) "{addr} -!" --> __TOKEN_DUP_PUSH_SUBSTORE(addr) "{{addr}} -!"
   ...third pass(2) __TOKEN_DROP() "__dtto" --> same

   ...create(1) __TOKEN_DUP_PUSH_SUBSTORE(addr) "{{addr}} -!"
   ...create(2) __TOKEN_DROP "__dtto"
    ld   BC, addr       ; 3:10      addr -!   ( x1 -- x1 ) [addr] -= x1
    ld    A, [BC]       ; 1:7       addr -!
    sub   L             ; 1:4       addr -!   lo
    ld  [BC],A          ; 1:7       addr -!
    inc  BC             ; 1:6       addr -!
    ld    A, [BC]       ; 1:7       addr -!
    sbc   A, H          ; 1:4       addr -!   hi
    ld  [BC],A          ; 1:7       addr -!
    ex   DE, HL         ; 1:4       addr -!
    pop  DE             ; 1:10      addr -!   ( a -- )
; seconds: 0           ;[12:66]
dworkin@dw-A15:~/Programovani/ZX/Forth/M4$ ../check_word.sh 'VERBOSE(1) PUSH(addr) SUBSTORE PUSH([index])'
   ...new __TOKEN_PUSHS(addr) "{addr}" --> token(1) __TOKEN_PUSHS(addr) "{{addr}}"
   ...new __TOKEN_SUBSTORE() "-!" --> token(2) __TOKEN_DROP "__dtto"
   ...new __TOKEN_PUSHS([index]) "{[index]}" --> token(3) __TOKEN_PUSHS([index]) "{{[index]}}"

   ...check all tokens
   ...second pass(1) __TOKEN_DUP_PUSH_SUBSTORE(addr) "{addr} -!" --> __TOKEN_DUP_PUSH_SUBSTORE(addr) "{{addr}} -!"
   ...second pass(2) __TOKEN_DROP() "__dtto" --> same
         ...token(2) --> __TOKEN_NOPE
   ...second pass(3) __TOKEN_PUSHS([index]) "{[index]}" --> __TOKEN_DROP_PUSHS([index]) "{{[index]}}"

   ...check all tokens2
   ...third pass(1) __TOKEN_DUP_PUSH_SUBSTORE(addr) "{addr} -!" --> __TOKEN_DUP_PUSH_SUBSTORE(addr) "{{addr}} -!"
   ...third pass(2) __TOKEN_NOPE() "" --> same
        ...token(2) --> __TOKEN_NOPE
   ...third pass(3) __TOKEN_DROP_PUSHS([index]) "{[index]}" --> __TOKEN_DROP_PUSHS([index]) "{{[index]}}"

   ...create(1) __TOKEN_DUP_PUSH_SUBSTORE(addr) "{{addr}} -!"
   ...create(2) __TOKEN_NOPE ""
   ...create(3) __TOKEN_DROP_PUSHS([index]) "{{[index]}}"
    ld   BC, addr       ; 3:10      addr -!   ( x1 -- x1 ) [addr] -= x1
    ld    A, [BC]       ; 1:7       addr -!
    sub   L             ; 1:4       addr -!   lo
    ld  [BC],A          ; 1:7       addr -!
    inc  BC             ; 1:6       addr -!
    ld    A, [BC]       ; 1:7       addr -!
    sbc   A, H          ; 1:4       addr -!   hi
    ld  [BC],A          ; 1:7       addr -!
    ld   HL,[index]     ; 3:16      [index]
; seconds: 0           ;[13:68]

This keeps the code efficient even when I have TOS and NOS pulled into HL and DE. I minimize the movement of the data tray up and down by the correct combination of words so that the number of items in the tray remains unchanged.

[ketmar]

From which language is this idiom and forgive my stupid question, but how should I understand it.

i'm sorry. it roughly means "try various things, including wild ones, and see which fit your needs best".

But that standard is something that makes it easier to switch to M4 FORTH, so I should stick to it on the other hand if possible.

honestly, don't bother. it is absolutely impossible to write anything using any standard. also, sticking to F83 will not give you bonus points, because F2012 is told to be a New Hot Thing (and it is even more idiotic than F83 and F92). and they're trying to produce F2022, which is even more idiotic than F2012…

i mean, read standards to learn common word names, but don't feel that your system should conform to any of them. Forth is about efficient using of hardware resources (among other things), and to achieve that, you have to throw away standards and "portability" BS.

just document each your word, it is more than enough for any Forth programmer to catch up. give us stack diagrams, and at least one-line description for your words, and devote several paragraphs to overal system overview. we, forthers, are used to seeing wildly different systems, and to look into documentation first. ;-)

In the "modern" standard there is only the word +!. I didn't see anything in OPTIONAL either. But I would also like -!.

this "modern standard" is a complete bullsh*t. it is made by losers, who either never used Forth for doing real-word apps, or stopped using Forth IRL long time ago. also, it is targeting "big systems", not constrained environments. been there, done that. Gforth is unusable bloatware, BigForth is dead as a doornail. and other people in "standard commetee" don't even have their own Forth systems. meh.

the moment i started to be REALLY productive with Forth was the moment i decided to ignore all and any "forth standards" out there.

But C has TRUE as 1 and FORTH has TRUE as -1

this is not something set in stone too. some of my systems used "1" as TRUE. it all depends of your goals, of the underlying hardware, and the code you're writing. F79 and FIG had it as "1", for example. "-1" is useful when your "OR" and "AND" words are bitwise. but you can introduce words like "LOGOR" and "LOGAND", for logical operations. then "4 8 AND" will be "0", but "4 8 LOGAND" will be "1" (or "-1", or any other non-zero value of your choice, "8", for example).

i also have words "0-AND" and "T-OR" in my system. they are very useful in cases when you need to return a boolean result, dropping some working values first. like, if your word ends with something like "DROP FALSE", it can be replaced with "0-AND", and "DROP TRUE" with "T-OR". this way i am avoiding useless stack manipulations, reusing TOS directly.

also, my system optimises things like "LIT n @" to "LIT:@" and such. i.e. to superinstructions with direct parameter in threaded code. it is done on the fly in "\," word (this word compiles CFA to threaded code). it tracks using of "LIT", and if it sees "LIT" followed by various operations like "@", it combines them in one superinstruction. just don't forget to zero remembered LIT address in "IF", "BEGIN" and such, so things like "$4000 BEGIN @" won't be optimised. ;-)

also, i am heavily using things like "?exit< … >?". this is "IF … EXIT ENDIF". it is useful to check some conditions and exit early, and slightly more efficiend than wrapping the whole word in "IF". like:

Code: Select all

: DO-SMTH-WITH-STR  ( addr count — flag )
  dup not?exit< drop 0-and >?
  ( now really process the string ) ;

I don't really understand the drafters of the standard.

neither do i.

Code: Select all

: -! swap negate swap +! ;

Easy, but did everyone do it the same way? Wouldn't it be better to have -! in extended standard?

because they assume that you are using one of the (two left, lol) commercial optimising compilers, which will inline all definitions, and then heavily optimise the code. this "standard" is basically a common ground for SwiftForth and VFX Forth, a desperate attempt to keep dying "forth vendors" alive. meh. let dead things die already! ;-)

[_dw]

i also have words "0-AND" and "T-OR" in my system. they are very useful in cases when you need to return a boolean result, dropping some working values first. like, if your word ends with something like "DROP FALSE", it can be replaced with "0-AND", and "DROP TRUE" with "T-OR". this way i am avoiding useless stack manipulations, reusing TOS directly.

Personally, I find it clearer to name a word like DROP_TRUE or DROP_FALSE. But I don't know if the length of the word has any greater meaning for Forth.

also, my system optimises things like "LIT n @" to "LIT:@" and such. i.e. to superinstructions with direct parameter in threaded code. it is done on the fly in "\," word (this word compiles CFA to threaded code). it tracks using of "LIT", and if it sees "LIT" followed by various operations like "@", it combines them in one superinstruction. just don't forget to zero remembered LIT address in "IF", "BEGIN" and such, so things like "$4000 BEGIN @" won't be optimised.

also, i am heavily using things like "?exit< … >?". this is "IF … EXIT ENDIF". it is useful to check some conditions and exit early, and slightly more efficiend than wrapping the whole word in "IF". like:

I honestly don't understand this. I'm still a beginner to Forth, even though I'm doing its compiler.
Some things just don't concern me at all, because I look at it as a source for compilation and not as an interpreter and how it works there.
If I don't use something, I forget it. I'm not the youngest anymore.

[_dw]

I have been working on improving the support for the logical comparison >,>=,<,<= for different cell sizes. 8 bit, 16 bit and 32 bit. Both with and without a sign.
And that's for the special case when one value is a constant known at compile time.
Because it creates quite a large space for different optimizations depending on the value of the constant.
And because thanks to this, I dug up the original code a lot, I had to write some test to check correctness and detect errors.

Because I keep making bugs, no matter how hard I try. This is just a fact.

The test lasted so long that I immediately used it as a benchmark.
One side of the condition is always a constant and the other side is the index of the loop that runs over all values.
Except for the 32-bit variety, which only has a 16-bit loop and stretches the number from 0xABCD to 0xABBBBBCD.

Code: Select all

|             System            |               Forth / C              |  Benchmark  |  Time       |           bin size          |
| :---------------------------: | :----------------------------------: | :---------: | :---------- | :-------------------------: |
| ZX Spectrum Fuse 1.6.0 Ubuntu | z88dk.zcc v22110-51889e5300-20231219 | -O2 u8cond  |  0m  1.80s  | 11 615 B (include 4x8 font)
| ZX Spectrum Fuse 1.6.0 Ubuntu | z88dk.zcc v22110-51889e5300-20231219 | -O2 s8cond  |  0m  3.42s  | 11 562 B (include 4x8 font)
| ZX Spectrum Fuse 1.6.0 Ubuntu | z88dk.zcc v22110-51889e5300-20231219 | -O2 u16cond |  9m 55.88s  | 12 311 B (include 4x8 font)
| ZX Spectrum Fuse 1.6.0 Ubuntu | z88dk.zcc v22110-51889e5300-20231219 | -O2 s16cond | 12m  1.65s  | 12 249 B (include 4x8 font)
| ZX Spectrum Fuse 1.6.0 Ubuntu | z88dk.zcc v22110-51889e5300-20231219 | -O2 u32cond | 24m 15.85s  | 12 609 B (include 4x8 font)
| ZX Spectrum Fuse 1.6.0 Ubuntu | z88dk.zcc v22110-51889e5300-20231219 | -O2 s32cond | result fail | 12 575 B (include 4x8 font)
| ZX Spectrum Fuse 1.6.0 Ubuntu | M4_FORTH                             | u8cond      |  0m  0.62s  |  4 823 B
| ZX Spectrum Fuse 1.6.0 Ubuntu | M4_FORTH                             | s8cond      |  0m  0.68s  |  5 005 B
| ZX Spectrum Fuse 1.6.0 Ubuntu | M4_FORTH                             | u16cond     |  3m 36.13s  |  5 502 B
| ZX Spectrum Fuse 1.6.0 Ubuntu | M4_FORTH                             | s16cond     |  3m 57.53s  |  5 884 B
| ZX Spectrum Fuse 1.6.0 Ubuntu | M4_FORTH                             | u32cond     |  5m 20.29s  |  6 487 B
| ZX Spectrum Fuse 1.6.0 Ubuntu | M4_FORTH                             | s32cond     |  5m 43.50s  |  7 587 B

The 32 bit signed variant in C failed. I still don't know what's going on, I then tested to see if it was my fault, but I didn't find anything.

PS: The M4 FORTH compiler initially failed too before I caught the bugs.

[ketmar]

Personally, I find it clearer to name a word like DROP_TRUE or DROP_FALSE. But I don't know if the length of the word has any greater meaning for Forth.

it doesn't for the compiler (most Forth systems using hashed vocabularies anyway), but it surely does for me. ;-) my idea is that faster actions should have shorter names. just a personal preference. besides, that's what my peephole optimiser is able to optimise. i mean, "true or"/"false and" are automatically optimised to "t-or"/"0-and", because "true" and "false" are numeric literals. but my optimiser cannot optimise arbitrary word sequences, so it cannot do anything with "drop true", for example.

I honestly don't understand this. I'm still a beginner to Forth, even though I'm doing its compiler.
Some things just don't concern me at all, because I look at it as a source for compilation and not as an interpreter and how it works there.
If I don't use something, I forget it. I'm not the youngest anymore.

ah. it is mostly for Forth compilers written in Forth itself, yeah. it prolly can be used in Forth compilers written in other languages, but i don't have a lot of expirience with such (because why use another language when i already have Forth? ;-).

the basic idea is this: the only word you need to implement Forth compiler is "comple CFA to the threaded code". i called this word "\," in my system. in many systems such word simply does "," (i.e. just puts the address to the dictionary). but it is the perfect place to plug in peephole optimiser: you can track last compiled words, and perform simple optimisations transparently to all other code. my compiler is able to fold things like "2 +" to one word "2+" and such (i.e. literal followed by some operation). it also does tail call optimisation: when EXIT is compiled, optimiser tries to replace last call with a branch, and omit EXIT. in some other, more complex situations "\," also does word inlining, and even translates call sequences to slightly optimised machine code.

the great thing is that it doesn't matter what i do in "\," — other code still works without any changes (mostly). so i can experiment with various optimisation techniques having a live, working, fully featured Forth system. UrForth/Beast, for example, can be built as DTC-FAST, DTC-SMALL and ITC from the same source code, with and without optimisations and TCO. if i'll ever need to implement optimising STC compiler, i will simply change "\,", but most of the other Beast code will not require any changes.

[_dw]

in many systems such word simply does "," (i.e. just puts the address to the dictionary).

When I typed the compiler like ", " it said "HERE !" and increments HERE by 2, where HERE is a virtual variable that only exists at compile time, so if someone uses it in a loop it will fail. If it is in some initialization or word/function, it will simply store it at some address in memory and it will not be "duplicated". No dictionary is used, nor do I have one, except to use the stack rename macro in M4 if someone redefines an already existing word and then returns it again.

I am now solving the problem that, when implementing the BEGIN-AGAIN/UNTIL/REPEAT loop, I had no idea that it was connected to the IF-ELSE-THEN branching structure.
​

Code: Select all

BEGIN ... condition WHILE  ... condition WHILE  ... REPEAT

Where the word BEGIN creates a new unique ID_BEGIN and the previous one is stored on the stack. So each WHILE is associated with the current ID_BEGIN and thus refers to the same REPEAT. The REPEAT word removes the current ID_BEGIN and pops the previous ID_BEGIN from the stack.

The same for IF, it creates a unique ID_IF for branching constructions, it stores the original one on its own stack. So ELSE, which compiles as just a jump, knows the address of the end of THEN. And the THEN word cancels the last ID_IF and pops the previous ID_IF from its stack.


Original Forth use:

Code: Select all

BEGIN ... condition WHILE  ... condition WHILE ... REPEAT  ... THEN ...

Code: Select all

BEGIN ... condition WHILE  ... condition WHILE ... AGAIN ." never" THEN  ... THEN ...

Spoiler

Code: Select all

: test
begin
." a"
while 
." b"
while 
." c"
again 
." never"
then 
." B"
then
." A" cr ;


0 test
0 1 test
0 1 1 test
0 1 1 1 test
0 1 1 1 1 test
0 1 1 1 1 1 test
0 1 1 1 1 1 1 test

Code: Select all

aA
abBA
abcaA
abcabBA
abcabcaA
abcabcabBA
abcabcabcaA

Where it behaves:
WHILE as IF
REPEAT as AGAIN and THEN together

So the alternative in M4 FORTH would be

Code: Select all

BEGIN ... condition IF  ... condition WHILE ... REPEAT  ... THEN ...
or
BEGIN ... condition IF  ... condition WHILE ... AGAIN THEN  ... THEN ...

Code: Select all

BEGIN ... condition IF  ... condition IF ... AGAIN ." never" THEN  ... THEN ...

but this record fails again in the original FORTH, so it seems that the connection is only between REPEAT and AGAIN+THEN and every WHILE creates a new THEN where it jumps if the condition does not apply.

The easiest way for me will be to remake it when I understand exactly how it behaves in the M4.

My problem now is that i created the word BREAK which is an alternative to the word LEAVE for a DO .. LOOP loop.

For me it proved useful with efficient code, it's just a jump instruction for AGAIN (I didn't have to deal with the fact that it could destroy the effectiveness of interpreted Forth).

So I have the code:

Code: Select all

  s\" How many acres of LAND will you +BUY or -SELL?" input
  begin
    negative? if
       ...  condition if  ...   s\" Again, how much land will you SELL?" input
      else  ...  break
      then
    else
      condition if ...  s\" Again, how much land will you BUY?" input
      else  ... break
      then 
    then
  again

Spoiler

Code: Select all

  s\" How many acres of LAND will you +BUY or -SELL?" input
(    123456789abcdef0123456789abcdef0123456789abcdef0123)
  begin
    dup 0 < if
      negate 
      land @ 2dup u> if
( ? [land])
        .\"  * O great Khammurapi,\n"
        .\"    we have but " 
        u. drop .\" acres of land!\n"
        s\" Again, how much land will you SELL?" input
      else
        over - land !
        cost_per_acre @ * storage +!
        exit
      then
    else
      dup cost_per_acre @ * storage @ 2dup u> if
( ? ?*[cost_per_acre] [storage])
        .\"  * O great Khammurapi,\n"
        .\"    we have but " 
        dup u. 
        .\" bushels of grain!\n"
        .\"    The maximum is " cost_per_acre @ u/ u. 2drop .\" acres.\n"
        s\" Again, how much land will you BUY?" input
      else
        swap - storage !
        land +!
        exit
      then 
    then
  again

And I can't write this as efficiently in pure Forth... The most similar solution is to put the BEGIN AGAIN loop in a new word and use EXIT instead of BREAK.
A much less efficient way is to use a BEGIN UNTIL loop and insert a BOOLEAN value at the end of each branch.

The BEGIN WHILE WHILE ... AGAIN loop is simply not suitable for this. You can easily leave the loop from that one, but I have the exact opposite, a lot of conditions that must apply in certain combinations in order for it to leave. And combining those conditions into one for BEGIN UNTIL is not that efficient because the conditions are resolved differently depending on other conditions (whether the value is positive or negative).

It is simply more efficient to have the word CONTINUE or BREAK in this. I would even support heresy for the CONTINUE(x) and BREAK(x) parameters. Where x == 0 is a normal C-like CONTINUE/BREAK and each additional number adds a jump from the next nesting. To me it makes more sense than using LABEL.

PS: I came across this while working on better string support in M4 FORTH. So I converted the HAMMURABI game to FORTH.

Other hurdles I've run into is that FORTH doesn't seem to know any #IF #IFDEF etc. So I can't easily deal with an incompatibility between Forths somewhere in the beginning.
I also don't know how to redefine the word TYPE and ./" so that it wraps after 51 characters. It just emulated a 51 line terminal. Because the texts are made for the ZX Spectrum and with a 5x8 font, I have exactly 51 characters, and if the line comes out to me in such a way that I don't have room for a break with \n, it is assumed that it will break by itself. With a different width, the words that are divided will be combined into one...

... .... ... ... xxx
yyy ... ... .... ....

... .... ... ... xxxyyy ... ... .... ....

https://codeberg.org/DW0RKiN/M4_FORTH/s ... khammurapi

Now the source can be run in gforth with small modifications such as adding begin..again to the words and replacing break with exit. Deletion of definitions at the beginning and cancellation of commented code that defines words used only in M4 FORTH.

Code: Select all

define({USE_FONT_5x8})
(
require random.fs
: -! dup @ rot - swap ! ;
: u/ swap abs swap abs / ;
: setseed 255 and for rnd drop next ;
)

Or it can be played on the ZX pres tap file created after compiling the M4 FORTH variant.

[ketmar]

When I typed the compiler like ", " it said "HERE !" and increments HERE by 2, where HERE is a virtual variable that only exists at compile time, so if someone uses it in a loop it will fail. If it is in some initialization or word/function, it will simply store it at some address in memory and it will not be "duplicated". No dictionary is used, nor do I have one, except to use the stack rename macro in M4 if someone redefines an already existing word and then returns it again.

you basically wrote a target compiler. there is an old tradition of having target compilers for Forth, they used both to rebuild the system itself, and to build the system for some different arch. UrForth/Beast have target compiler too — that's how she rebuilds herself after all. ;-)

the problem with having a target compiler written in some other language, not in Forth, is that you now have TWO languages to deal with. it is way easier to write target compiler in Forth too. at least for me. ;-)

Original Forth use:

Code: Select all

BEGIN ... condition WHILE  ... condition WHILE ... REPEAT  ... THEN ...

Code: Select all

BEGIN ... condition WHILE  ... condition WHILE ... AGAIN ." never" THEN  ... THEN ...

it is (and always was) a freakin' nightmare. the mere abomination of everything good programming is about. UrForth supported several WHILEs from the very beginning, but you never need to put any ugly "THEN" after the loop. you can even write things like: BEGIN … WHILE … WHILE … UNTIL", and it will work as expected.

but tbh, i abandoned BEGIN loops some time ago. they are too limited. i am using Parnas' generalized loop/iteration thingy instead. it replaces all kinds of loops, and in addition it can replace CASE too. it looks like this:

Code: Select all

<<
  cond ?v| … |?
  cond ?^| … |?
  …and so on…
else| … >>

"?v|" executes the "…" body and exits the loop if "cond" is true, and "?^|" does the same, but restarts the loop instead of exiting. easy to use, and no need for "break/continue". it also supports things like "cond CAND cond CAND … ?v| … |?" "CAND" is short-circuit boolean. "COR" is there too. but i am almost never using them.

My problem now is that i created the word BREAK which is an alternative to the word LEAVE for a DO .. LOOP loop.

again, UrForth has both BREAK and CONTINUE (leftovers from the ancient times), and they work both for BEGIN and for DO loops.

the basic idea is easy: "BREAK" simply creates a chain of branches to be resolved later, and "REPEAT/UNTIL/LOOP" resolves the whole chain. until resolved, branch destination points to the previous branch to resolve.

Code: Select all

  s\" How many acres of LAND will you +BUY or -SELL?" input
  begin
    negative? if
       ...  condition if  ...   s\" Again, how much land will you SELL?" input
      else  ...  break
      then
    else
      condition if ...  s\" Again, how much land will you BUY?" input
      else  ... break
      then 
    then
  again

Spoiler

Code: Select all

  s\" How many acres of LAND will you +BUY or -SELL?" input
(    123456789abcdef0123456789abcdef0123456789abcdef0123)
  begin
    dup 0 < if
      negate 
      land @ 2dup u> if
( ? [land])
        .\"  * O great Khammurapi,\n"
        .\"    we have but " 
        u. drop .\" acres of land!\n"
        s\" Again, how much land will you SELL?" input
      else
        over - land !
        cost_per_acre @ * storage +!
        exit
      then
    else
      dup cost_per_acre @ * storage @ 2dup u> if
( ? ?*[cost_per_acre] [storage])
        .\"  * O great Khammurapi,\n"
        .\"    we have but " 
        dup u. 
        .\" bushels of grain!\n"
        .\"    The maximum is " cost_per_acre @ u/ u. 2drop .\" acres.\n"
        s\" Again, how much land will you BUY?" input
      else
        swap - storage !
        land +!
        exit
      then 
    then
  again

And I can't write this as efficiently in pure Forth...

it shouldn't be done like this. this is not Forth, this is C, disguised as Forth. such code is unmaintainable, and write-only.

yes, it should be factored to many small words. that is the only way to write a maintainable Forth code. and then if you care about speed, let the compiler inline your words back, and optimise. that's what UrForth/Beast: Destroyer does, for example: the compiler aggressively inlines, and then performs peephole optimisations on the inlined code. the result is ~2.5 times slower than gcc -O2. considering that the whole optimiser is ~38 kb of source code, and i wrote it in 2 weeks, i believe that the result is very impressive. ;-) or, to compare apples with apples: Devastator+Succubus is ~1.5 times slower than SP-Forth, which has sophisticated optimizer (110+ kb of source code).

i will later do something like this for Z80 cross-compiler too. i will prolly add more peephole patterns, though.

basically, the First Rule In The Book Of Writing Good Forth Code is: "if you feel that you need a comment to explain what your program is doing, then factor that part into properly named word, and use it instead of comment." and then you will have high-level code like this:

Spoiler

Code: Select all

: inline-cfa
  current-cfa cfa>optinfo code-@ cci-optinfo^:!

  debug-inline-started
  prepare-opt-ilen
  prepare-icode
  trace-jumps
  remove-ret
  cci-ret-count ?<
    inline-force-word? not?exit<
        debug-inliner-failed
        cc\,-noinline
        >? >? ;; alas, there are some "ret"s there
  cci-wlen ?not-exit  ;; empty word, nothing to do
  sanity-check

  ;; ok, we have something to copy
  setup-stacks

  ;; order matters!
  [ debug-disable-inliner-peepopt ] [IFNOT]
  low:stacks-swapped? ?<
    << was-any-stitch-optim?:!f
       remove-end-pop-ebx-start-push-ebx
       remove-end-push-eax-start-pop-eax
       remove-end-push-ebx-start-pop-eax
       remove-end-push-ebx-mov-ebx-eax-start-pop-eax
       remove-end-push-ebx-mov-ebx-lit-start-pop-eax -- most important for benchmark ;-)
       remove-end-push-ebx-mov-ebx-mod-r/m-start-pop-eax
       remove-end-push-[esp+4]-start-pop-eax
       remove-end-mov-ebx-start-pop-ebx
       remove-end-push-ebx-start-pop-ebx
       remove-end-lit-load-start-[ebx]-load  -- this may remove the only instruction in "@"
       ;; it must be here
       remove-end-push-lit-nonpop-start-pop-eax
    was-any-stitch-optim? cci-wlen logand ?^|| else| >>
  >?
  [ENDIF]

  cci-wlen ?<
    copy-code
    check-final-swap-stacks
    setup-new-codeblock
  >?
  \ TODO: do the same for iwords. i.e. inlined forth word can have "add/mul/etc" with literal too!
  low:stacks-swapped? ?<
    << was-any-stitch-optim?:!f
      optim-addr-store
      optim-rstack-store
      optim-add-lit-fold
      optim-sub-lit-fold
      optim-add-lit-inc/dec
      optim-add-lit-add-lit
      optim-mov-lit-add-lit
      optim-mov-lit-umul
      optim-mov-lit-imul
    was-any-stitch-optim? ?^|| else| >>
  >?

  debug-inline-finished
  stat-words-inlined:1+! ;

this is the heart (or should i say "guts"? ;-) of Succubus optimising inliner. it should look like a pseudocode. ;-)

by the way, Devastator can rebuild herself in ~40 msecs on my core2duo/3.5GHz. ~600 kb of source code. Succubus does such a good job that i don't need to use asm to speed up the code anymore. for example, i removed all asm code from input file parsing, and lost only ~1.5 msecs on rebuilds (the loss was ~12-15 msecs without the optimiser).

Other hurdles I've run into is that FORTH doesn't seem to know any #IF #IFDEF etc. So I can't easily deal with an incompatibility between Forths somewhere in the beginning.

oh, it can do that! see "[IF]/[ELSE]/[THEN]" words, they are in ANS standard, and supported by most Forthes out there.

I also don't know how to redefine the word TYPE and ./" so that it wraps after 51 characters.

something like this: first, redefine TYPE and use EMIT to print the string char by char, keeping track of the current horizontal position, and inserting newlines when necessary. then:

Code: Select all

: ." POSTPONE S" POSTPONE TYPE ; IMMEDIATE

if i remember it right, this should work in most ANS Forthes.

p.s.: UrForth/Beast: Devastator + Succubus source code, if you are interested. but beware: it is TOTALLY non-standard. at all. ;-)

[_dw]

it shouldn't be done like this. this is not Forth, this is C, disguised as Forth. such code is unmaintainable, and write-only.
yes, it should be factored to many small words. that is the only way to write a maintainable Forth code. and then if you care about speed, let the compiler inline your words back, and optimise.

I try to make the output assembler code as close as possible to what a human would write.
So if there is an effective method. so I try to use it. I just don't like using a "function" just because I can use EXIT instead of BREAK when the function is only called once, and the only difference in the code is simply the work to call the function and its return overhead.

In this, the Forth language simply goes against me, when I could get into a situation where even for a single Z80 instruction such as "ex DE, HL" I will call a function, because that's what the word SWAP is for.

Maybe it could be solved that for each new word I could have a usage counter and if it is exactly one I would use the INLINE variant. But that has some other consequences for the treatment, so I'll leave it as it is for now.

I don't think the code is write only, maybe it's more C style than FORTh, but it's still structured and understandable to me. On the contrary, I consider CONTINUE A BREAK to be a clearly transparent code that I could solve the classics "spaghetti code" like in ZX Basic or in assembler, which allows absolutely everything. This is how the jump instruction is clearly defined, what it does and where it points.

it shoul
oh, it can do that! see "[IF]/[ELSE]/[THEN]" words, they are in ANS standard, and supported by most Forthes out there.

Nice, nice, nice!

When I studied it and added the word [DEFINED] to [IF] [ELSE] and [ENDIF], I am now able to write code at the beginning or anywhere so it will run in gForth and can also be compiled in M4 FORTH for ZX Spectrum 48kb
So I created the word M4_FORTH so...

[DEFINED] M4_FORTH [IF] ... [ELSE] ... [ENDIF]

So from then on, khammurapi.fth is written so that not a single character has to be changed.

Unfortunately, these words have to be solved by an awk script converting pure forth to M4 FORTH, because then it is too late... if there is, for example, a word definition in the branch, it would escape outside [IF] [THEN], because the word definition changes to a function definition and these are stored after the "main" section/main program. So the first ORG always points to the entry point.

I never remember how I wrote it in that awk, and these branches are even more important than determining whether we are in a text chain or a comment, but somehow I managed to pull it together. I already have them ready bascin single variant. embedded/nested variant too (this word was probably translated wrong, I mean IF in IF and that in the next...) [IF], but they will be in the repository from tomorrow (if I don't die/the laptop will fail/the internet will work/. ..).

[ketmar]

I try to make the output assembler code as close as possible to what a human would write.

In this, the Forth language simply goes against me, when I could get into a situation where even for a single Z80 instruction such as "ex DE, HL" I will call a function, because that's what the word SWAP is for.

that's something for optimiser to do, not for the programmer. ;-) i mean, if one need to write "un-Forthy" code, then what is the reason of using Forth at all?

So if there is an effective method. so I try to use it. I just don't like using a "function" just because I can use EXIT instead of BREAK when the function is only called once, and the only difference in the code is simply the work to call the function and its return overhead.

but this is EXACTLY how good Forth code is written. ;-) yes, it is hard to not think about machine instructions, because Forth is so close to the metal. but we're throwing away the whole Forth this way. many Forth books talk about "Forth phisolophy", "Forth way" and so on. it sounds like a usual bullsh*t, but… no, it isn't. sadly, it is hard to explain: it took me decades (and several developed Forth systems) to grasp.

maybe try to look at Forth like this: Forth is here to make programmers work effective, NOT programs. effective programs is a side-effect. and no good Forth programmer thinks about effective code first. they simply writing it The Natural Forth Way.

side note: in your Khammurapi program i can clearly see a database, and a state machine on top of it. that's prolly how i would do it: db + state machine + general interpreter. it requires to manually reverse-engineer the code first, of course, to build a state diagram. but the final result would be better this way, i believe. ;-)

my Forth code usually works the first time i wrote it (sans hidden logic errors in algorithms themselves, of couse). this is not because i am The Great Mind, or something, no. that's because my Forth words are small (5-7 calls to other Forth words, usually 2 lines of code at max; except the very low-level words sometimes, where i need something like CASE, and it is too small to create a table yet). i can debug such word right after a wrote it. (read: simply try interactively if it works. with compile times counting in milliseconds, recompiling and running the project may be counted as interactive shell too. ;-) but most of the time i don't need to debug anything, because the word is so small that i can keep the whole thing in my head while i writing it.

and when i wrote and tested it, i can completely throw it away from my brain. i don't care how it works anymore, it Just Works. now i can write another small word. and so on.

there is a reason why Chuck never added "BREAK" and "CONTINUE" to Forth. and he regrets adding DO loops (and FOR too). most people coming to Forth from other languages immediately want to add those things "back" (including yours truly ;-). it is easy to do… and it is useless. and long words are useless too. let me show you some Really Bad Forth code:

Code: Select all

|: (FIND-IN-ONE-VOCID-NOHASH)  ( addr count skip-hidden? vocid -- cfa-xt TRUE // FALSE )
  2over hash-name >r vocid>latest  ( addr count skiphid? lfa | hash )
  << @ dup not?v| rdrop 4drop false |?
     dup lfa>hfa @ r@ <> ?^||
     over cand dup lfa>ffa @ wflag-private and ?^||
     dup >r 2over r> lfa>nfa name= not?^||
  else| rdrop >r 3drop true r> lfa>cfa swap >> ;

and what this sh*t does? go figure… i wrote it, and now i barely understand wtf is going on there. mostly because this is at least 4 or 5 words "manually inlined". and 4 stack arguments? 'cmon! i should be using module globals instead. impenetrable, undebuggable and unreadable pile of crap.

and yes, it was written this way because early Beast compiler couldn't inline. this is the part of the main loop (searching words in dictionary), so it should be fast. i can't mentally stand overhead of many CALLs too. ;-) that's why i summoned Succubus: now i know that she will do inlining better than me doing it manually, so i can stop worrying and be happy. ;-)

I don't think the code is write only, maybe it's more C style than FORTh, but it's still structured and understandable to me.

actually, using "BREAK" or "CONTINUE" is a clear sign of UNstructured code.

On the contrary, I consider CONTINUE A BREAK to be a clearly transparent code that I could solve the classics "spaghetti code" like in ZX Basic or in assembler, which allows absolutely everything. This is how the jump instruction is clearly defined, what it does and where it points.

from the PoV of structured programming, it's all the same thing. the mere fact that you removed labels from your GOTOs didn't made your code structured. ;-)

structured programming is not actually about using "proper loop structures", it is about mathematically proved correctness of the code. which is almost impossible to do if your code has abrupt control flow breaks. Dijkstra hadn't used "BREAK" in his works for a reason. Wirth didn't added "BREAK" to Modula and Oberon for a reason. Forth doesn't have that for a reason. and it is the same reason in all cases. ;-) the first pages of Parnas' paper on his generalised iteration structure is full of math, proving that it doesn't break Dijkstra's principles of structured programming.

[_dw]

Redefining .\" to .\" with TYPE together is probably beyond me.

At https://forth-standard.org/standard/core/PARSE it says it should be able to do this:

: .( [CHAR] ) PARSE TYPE ; IMMEDIATELY

alias

: .\" [CHAR] " PARSE TYPE ; IMMEDIATELY

It does not work...

This will go through the entire source file during compilation and print all the text in words .\"
but it does not save them... so they are no longer there when the program is running.

: .\" POSTPONE .\S POSTPONE TYPE ; IMMEDIATELY

also not working

I've tried randomly adding or removing POSTPONE IMMEDIATELY >DOES because I have a vague feeling that it might help, but since I'm not sure how exactly it works, it didn't work. It will either print immediately or it will scream that the first word of the string is an unknown/undefined word.

Code: Select all

[DEFINED] M4_FORTH [IF]
  define({USE_FONT_5x8})
[ELSE]
  51 constant max_x
  variable x 0 x !
  : type
    tuck + swap
    dup 0= if 2drop exit then
    negate
    begin 
      dup while 
        2dup + 
        1 x +! 
        x @ max_x > if
          0 x !
          cr
        then
        @ 
        dup 255 and 10 = if
          0 x !
        then
        emit 1+ 
    repeat 
    2drop ;
  : .\" [char] " PARSE TYPE ; immediate
  : cr 0 x ! cr ;
  require random.fs
  : -! dup @ rot - swap ! ;
  : u/ swap abs swap abs / ;
  : setseed 255 and for rnd drop next ;
  cr
[ENDIF]

[_dw]

actually, using "BREAK" or "CONTINUE" is a clear sign of UNstructured code.


from the PoV of structured programming, it's all the same thing. the mere fact that you removed labels from your GOTOs didn't made your code structured.

structured programming is not actually about using "proper loop structures", it is about mathematically proved correctness of the code. which is almost impossible to do if your code has abrupt control flow breaks. Dijkstra hadn't used "BREAK" in his works for a reason. Wirth didn't added "BREAK" to Modula and Oberon for a reason. Forth doesn't have that for a reason. and it is the same reason in all cases. the first pages of Parnas' paper on his generalised iteration structure is full of math, proving that it doesn't break Dijkstra's principles of structured programming.

I'm not a computer scientist, so I won't argue about the meaning of STRUCTURED CODE.

I take it that there are situations where it is effective to change the code flow, you can use goto/jp/jr.

If it is clear to me where the flow will change, it is a well-written code for me.

For example, the type code
IF
...
ELSE
...
ENDIF

It is a GOTO type situation with clearly defined behavior. The same situation written only with the help of a jump may not be so clear. Even if it's the same code.

JP condition, ELSE
...
jp ENDIF
ELSE:
...
ENDIF:

BREAK and CONTINUE are similar for me.

There is no equally effective substitute for them...

For BREAK I would have to wrap it in a function and use RETURN/EXIT.

Another way would be to create a helper boolean variable and set it to TRUE. In the case of BREAK, set it to FALSE and have this EVERYWHERE in the code...

IF TEMP and .. THEN ... ENDIF
IF TEMP and .. THEN ... ENDIF

Same for CONTINUE.

This would be insensitive and terrible code for me. Written just to achieve the same behavior.

When diving into the WHILE part of the code in C, I get into a situation where, due to the non-existence of BREAK(x) and CONTINUE(x), I will need a LABEL, or the already described less effective alternatives.

Rewriting the code will not make it more efficient. No matter what we call it.

[ketmar]

I'm not a computer scientist, so I won't argue about the meaning of STRUCTURED CODE.

i am not a scientist too. i am an engineer, doing practical things. most of the software i am using now written by myself, and it works. i am rarely shutting down my PC (half-year uptime is not that much ;-), and i want my software to work all this time without restarting and crashes. and it does.

i started programming more than 20 years ago, and never read any theory books back then. i only started to read them 5-7 years back, and i found that they really help. now i understand what Dijkstra was talking about, and why it matters.

but don't take me wrong, please. there is nothing wrong in ignoring those books. i'm just trying to say that they may (and prolly will) help you to write much better code, if you'll read them. it's not some "theoretical science never used in practice". Dijkstra wrote one of the first Algol compilers, for example, and he knew what he was talking about. actually, his book is about the expirience he gathered doing purely practical things.

I take it that there are situations where it is effective to change the code flow, you can use goto/jp/jr.

If it is clear to me where the flow will change, it is a well-written code for me.

sure. there is no rule which cannot be broken. the question is "why?". i mean, first, write the most straightforward code, and prove its correctness. then you can optimise it if you need to. if your code is proven, then you can transform it as you like, keeping all the invariants intact.

Rewriting the code will not make it more efficient. No matter what we call it.

it's not about code efficiency, it's about programmer efficiency. if you can mathematically prove your code, you don't need 1000+ tests to make sure it works: it Just Works. after i learned the apparatus Dijkstra was taliking about, most of the time my code Just Work. in the end of the day, i am spending less time both writing and debugging. and it helps with everything, including Z80 asm.

and i found that most of the time manual microoptimisations don't matter at all, even for 8-bit CPUs. yes, you need fast sprite routine, and some other low-level gfx thingys. but game logic, for example, can be very straightforward, and without any tricks. manual microoptimisation can speed things up by 5-10%. good algorithm can speed everyting up by orders of magnitude.

[ketmar]

Redefining .\" to .\" with TYPE together is probably beyond me.

just checked this in gforth:

Code: Select all

: type  ( addr count )  33 emit type 33 emit ;
: ." postpone s" postpone type ; immediate
: a ." hi" cr ;
a
bye

it works.

At https://forth-standard.org/standard/core/PARSE it says it should be able to do this:

: .( [CHAR] ) PARSE TYPE ; IMMEDIATELY

alias

: .\" [CHAR] " PARSE TYPE ; IMMEDIATELY

now, this is wrong. ;-)
".(" is used to print the string, not to compile the code to print it. your dot-quote is the same as ".(" — it prints the string immediately, not compiling the code. that's why "postpone" is there: it actually compiles the things.


also, your "type" is written in C, not in Forth. ;-)

Code: Select all

variable e-col
0 e-col !

: cr 13 emit 10 emit ;
: wrap  e-col @ 51 = if cr 0 e-col ! then ;
: advance  ( char )  13 = if 0 else e-col @ 1+ then e-col ! ;
: emit  ( char )  dup advance emit wrap ;
: cr  13 emit ;
: type  ( addr count )  for dup c@ emit 1+ next drop ;
: ." postpone s" postpone type ; immediate

: a 256 for ." abc" next cr ;

a
bye

just tested this in gforth: it works.

p.s.: standards sux. ;-)

[_dw]

just checked this in gforth:

Code: Select all

: type  ( addr count )  33 emit type 33 emit ;
: ." postpone s" postpone type ; immediate
: a ." hi" cr ;
a
bye

it works.


now, this is wrong.
".(" is used to print the string, not to compile the code to print it. your dot-quote is the same as ".(" — it prints the string immediately, not compiling the code. that's why "postpone" is there: it actually compiles the things.


also, your "type" is written in C, not in Forth.

Code: Select all

variable e-col
0 e-col !

: cr 13 emit 10 emit ;
: wrap  e-col @ 51 = if cr 0 e-col ! then ;
: advance  ( char )  13 = if 0 else e-col @ 1+ then e-col ! ;
: emit  ( char )  dup advance emit wrap ;
: cr  13 emit ;
: type  ( addr count )  for dup c@ emit 1+ next drop ;
: ." postpone s" postpone type ; immediate

: a 256 for ." abc" next cr ;

a
bye

just tested this in gforth: it works.

p.s.: standards sux.

at first glance it looks like your code won't work

Code: Select all

: type ( addr count ) for dup c@ emit 1+ next drop ;

because FOR ends at ZERO INCLUDING and that was the reason why I rejected the FOR loop at the very beginning, because it would still have to treat the empty string and reduce the index by 1...
but after editing it really works and I really don't know what I did wrong before, right?

Code: Select all

: ." postpone s" postpone type ; immediate

It didn't work for me and I tested it, of course, also in gforth, which is a regular package in my distribution.

Last night I solved it so that I'm all .\" manually rewritten to s\" and added TYPE after them. This also revealed one bug in M4 FORTH, where the combination of DROP together with s\" containing dangerous text like ...C O N G R A T U L A T I ..." activated the letter I as an index.

The idea to move the counter to EMIT is nice!

I ended up with this one

Code: Select all

[DEFINED] M4_FORTH [IF]
  define({USE_FONT_5x8})
[ELSE]
  51 constant max_x
  variable x 0 x !
  : cr 0 x ! cr ;
  : plus_x ( -- )
    x @ 1+ dup x ! 
    max_x >= if
      cr
    then
  ;
  : emit ( char -- )
    dup emit 
    255 and dup 10 = if 
      0 x !
    then
    32 >= if 
      plus_x
    then
  ;
  : type
    dup 0= if 2drop exit then
    1-
    for 
      dup @ emit 1+
    next 
    drop 
  ;
  : .\" POSTPONE s\" POSTPONE TYPE ; IMMEDIATE
  require random.fs
  : -! dup @ rot - swap ! ;
  : u/ swap abs swap abs / ;
  : setseed 255 and for rnd drop next ;
  cr cr cr
[ENDIF]

It seems that gforth in CR calls EMIT internally, so it took debugging so that it doesn't do 2 wraps.

I used the word PLUS_X to make it ...more FORTH

[_dw]

i am not a scientist too. i am an engineer, doing practical things. most of the software i am using now written by myself, and it works. i am rarely shutting down my PC (half-year uptime is not that much , and i want my software to work all this time without restarting and crashes. and it does.

i started programming more than 20 years ago, and never read any theory books back then. i only started to read them 5-7 years back, and i found that they really help. now i understand what Dijkstra was talking about, and why it matters.

but don't take me wrong, please. there is nothing wrong in ignoring those books. i'm just trying to say that they may (and prolly will) help you to write much better code, if you'll read them. it's not some "theoretical science never used in practice". Dijkstra wrote one of the first Algol compilers, for example, and he knew what he was talking about. actually, his book is about the expirience he gathered doing purely practical things.


sure. there is no rule which cannot be broken. the question is "why?". i mean, first, write the most straightforward code, and prove its correctness. then you can optimise it if you need to. if your code is proven, then you can transform it as you like, keeping all the invariants intact.


it's not about code efficiency, it's about programmer efficiency. if you can mathematically prove your code, you don't need 1000+ tests to make sure it works: it Just Works. after i learned the apparatus Dijkstra was taliking about, most of the time my code Just Work. in the end of the day, i am spending less time both writing and debugging. and it helps with everything, including Z80 asm.

and i found that most of the time manual microoptimisations don't matter at all, even for 8-bit CPUs. yes, you need fast sprite routine, and some other low-level gfx thingys. but game logic, for example, can be very straightforward, and without any tricks. manual microoptimisation can speed things up by 5-10%. good algorithm can speed everyting up by orders of magnitude.

There is a translation of the first parts by Dijkstra in Czech, I borrowed them from the university library, but it was too difficult for me.
I didn't want to learn a made-up language of symbolic instructions just for the sake of explanation in a book.
And above all, he was a mathematician and I just didn't understand some things. And I'm sure I haven't gotten better at math since then.
At that time I was more interested in intel processors and chess programming.

[_dw]

There is a translation of the first parts by Dijkstra in Czech, I borrowed them from the university library, but it was too difficult for me.
I didn't want to learn a made-up language of symbolic instructions just for the sake of explanation in a book.
And above all, he was a mathematician and I just didn't understand some things. And I'm sure I haven't gotten better at math since then.
At that time I was more interested in intel processors and chess programming.

Looks like I confused it with another book...




The Art of Computer Programming
https://en.wikipedia.org/wiki/The_Art_o ... rogramming
https://en.wikipedia.org/wiki/Donald_Knuth


If I look for the author Dijkstra in the book database, I can only find the Dutch author Aron Dijkstra with the book Nick the Knight, Dragon Slayer .)

But you mean another Dutchman, Edsger W. Dijkstra (none of his books have ever been translated).

[ketmar]

Looks like I confused it with another book...

yeah, two completely different things. ;-) Knuth is more like a cookbook: he covered many areas, and you can simply take what you need and use, ignoring everything else. if you need some algorithm — check Knuth books, chances are the algo you need is there.

but Dijkstra is about programming as an excesise in theorem proving. ;-) any program can be looked at as a theorem, and you can prove that it does what you think it does.

[ketmar]

at first glance it looks like your code won't work

Code: Select all

: type ( addr count ) for dup c@ emit 1+ next drop ;

because FOR ends at ZERO INCLUDING and that was the reason why I rejected the FOR loop at the very beginning, because it would still have to treat the empty string and reduce the index by 1...
but after editing it really works and I really don't know what I did wrong before, right?

that's why i wrote that standards sux. ;-) in UrForth, FOR is slightly different, and i didn't noticed extra printed char in gforth. ;-) it should be "0 DO … LOOP" instead. and i deliberately ignored the possibility of empty string. slightly better code looks like this:

Code: Select all

: type  ( addr count )  0 max 0 ?do dup c@ emit 1+ loop drop ;

or even like this:

Code: Select all

: type  ( addr count )  0 max over + swap ?do i c@ emit loop ;

just4fun, in UrForth i'd write it like this:

Code: Select all

: type  ( addr count )  swap << over +?^| c@1+ emit 1-under |? else| 2drop >> ;

Code: Select all

: ." postpone s" postpone type ; immediate

It didn't work for me and I tested it, of course, also in gforth, which is a regular package in my distribution.

it works in my gforth version (which is several years old, tho), in bigForth, in SP-Forth. but maybe they broke something in new gforth, because standards sux. ;-)

I used the word PLUS_X to make it ...more FORTH

it's better, yes. but your "plus_x" still does two completely different things. i splitted it to "advance" and "wrap" because each word should do only one thing: the code is easier to manage this way.

also, i used the weakest test possible ("=") instead of (">="), because i prefer the code to crash/do unusual thing in case i did something wrong. stronger tests tend to hide bugs. i know that the counter in monotonically incrementing, so it could never be greater than the limit. hence, no need to use ">=" there. and if i'll violate this rule somewhere, let the printing routine skip the swap, so the bug will be visible. ;-)

[ketmar]

But you mean another Dutchman, Edsger W. Dijkstra (none of his books have ever been translated).

that's a shame. and that's why you need English, so you won't need a translation ;-)

[_dw]

I also added support for colors in gForth. Quite fun to see. It's just completely reversed, it does EMIT and takes the ZX Spectrum 48Kb ROM standard and replaces it with color codes for bash.
Correctly, I should probably convert it in the fth2m4.sh script from bash color codes to ZX standard color codes.
Maybe other time...

Code: Select all

  : white   .\" \e[0;37m" ;
  : l_blue  .\" \e[1;34m" ;
  : red     .\" \e[0;31m" ;
  : purple  .\" \e[0;35m" ;
  : green   .\" \e[0;32m" ;
  : cyan    .\" \e[0;36m" ;
  : l_brown .\" \e[1;33m" ;
  : black   .\" \e[0;30m" ;
  : emit ( char -- )
    set_color @ if
      255 and
      case 
        0 of white   endof ( zx black --> white)
        1 of l_blue  endof ( zx blue)
        2 of red     endof ( zx red)
        3 of purple  endof ( zx magenta --> purple)
        4 of green   endof ( zx green)
        5 of cyan    endof ( zx cyan)
        6 of l_brown endof ( zx yellow --> light brown)
        7 of black   endof ( zx white --> black)
        ." \e[m" ( reset)
      endcase
      0 set_color !
      exit
    then
    dup emit 
    255 and 
    dup 32 >= if 
      plus_x
      0 set_color !
    else dup 10 = if
      0 x !    
      0 set_color !
    else dup 16 = if
      -1 set_color !
    then then then
    drop
  ;

[ketmar]

that's great!

but why using "case" there? this is just a regular switch over the monotonous range, and it's easier and faster to implement it with array of pointers to color printers. like this:

Code: Select all

create cctable
' white ,
' l_blue ,
' red ,
' purple ,
' green ,
' cyan ,
' l_brown ,
' black ,
' other ,

: color ( idx )  0 max 8 min cells cctable + @ execute ;

also, "set_color" can be replaced with pointer to print word too.

Code: Select all

variable emitter
variable default-emit
: emit-normal ( char )  dup 16 = if drop ['] color emitter ! else emit then ;
: reset-emit  default-emit @ emitter ! ;
' emit-normal default-emit ! reset-emit
: emit  ( char )  emitter @ reset-emit execute ;

and if you'll need to implement more special codes, you could use pointer array, like "cctable", but for "emitters".

less C, more Forth. ;-) using such "vectored words" and "execution tables" is a common Forth idiom. now each your Forth word is doing one, and only one thing again. easy to write, easy to read, easy to debug.

[_dw]

The Forth terminology for strings is so confusing...

When I came across the word COUNT, I found out that it should also be able to compute Pascal-style strings.
But I didn't find the word that made them.
Besides >,"<
So I created the word LSTRING in M4_FORTH, which should be similar to the word STRING.
I named the LSTRING as >ls"< in "forth" because the second word is >s"<.
Now the script failed me while translating https://rosettacode.org/wiki/Tic-tac-toe#Forth
He doesn't know the word >C"<
I wonder what it's supposed to do... ah, so this is the default name for LSTRING.
So I edited the script and renamed LSTRING to CSTRING in M4 FORTH. I don't really like the name because I read words starting with C as CHAR meaning "byte".

I added to the conversion script support for "local variables" which gforth does.
It seems to have the syntax >{ name1 name2 name3 ... -- just a comment }<

When I looked at what it does, it can be manually rewritten as

...
VALUE fn_name_name3
VALUE fn_name_name2
VALUE fn_name_name1

It is prefixed with the function name because these are global variables and gforth can have the same names in every function/word.

https://rosettacode.org/wiki/Cycle_detection#Forth

Now it can compile correctly.
This will lose the effectiveness of Forth when you have the necessary variables on the stack. But from a certain point it is probably better to at least partially keep it in the computer's memory. Unfortunately, the program (Cycle_detection) crams it all into memory, so it's almost C.

I came across some https://rosettacode.org/wiki/Man_or_boy_test by Donald Knuth. But even gforth broke his teeth on that code. (I apologize for the idiom, perhaps it is also in English). Recursion is so bad on the Z80 that Donald jerks his foot in the grave. I'll accept that I'll die old and still be a boy... .)

PS: Damn he's still alive!

that's great!

but why using "case" there? this is just a regular switch over the monotonous range, and it's easier and faster to implement it with array of pointers to color printers. like this:

Code: Select all

create cctable
' white ,
' l_blue ,
' red ,
' purple ,
' green ,
' cyan ,
' l_brown ,
' black ,
' other ,

: color ( idx )  0 max 8 min cells cctable + @ execute ;

also, "set_color" can be replaced with pointer to print word too.

Code: Select all

variable emitter
variable default-emit
: emit-normal ( char )  dup 16 = if drop ['] color emitter ! else emit then ;
: reset-emit  default-emit @ emitter ! ;
' emit-normal default-emit ! reset-emit
: emit  ( char )  emitter @ reset-emit execute ;

and if you'll need to implement more special codes, you could use pointer array, like "cctable", but for "emitters".

less C, more Forth. using such "vectored words" and "execution tables" is a common Forth idiom. now each your Forth word is doing one, and only one thing again. easy to write, easy to read, easy to debug.

Code: Select all

[DEFINED] M4_FORTH [IF]
  define({USE_FONT_5x8})
[ELSE]
  51 constant max_x
  variable x 0 x !
  : cr 0 x ! cr ;
  : plus_x ( -- )
    x @ 1+ dup x ! 
    max_x >= if
      cr
    then
  ;
  : white   .\" \e[0;37m" ;
  : l_blue  .\" \e[1;34m" ;
  : red     .\" \e[0;31m" ;
  : purple  .\" \e[0;35m" ;
  : green   .\" \e[0;32m" ;
  : cyan    .\" \e[0;36m" ;
  : l_brown .\" \e[1;33m" ;
  : black   .\" \e[0;30m" ;
  : other   .\" \e[m" ;
  create cctable
  ' white ,
  ' l_blue ,
  ' red ,
  ' purple ,
  ' green ,
  ' cyan ,
  ' l_brown ,
  ' black ,
  ' other ,
  variable case_emit
  variable default_emit
  : reset_emit default_emit @ case_emit ! ;
  : emit_color ( idx )  255 and 0 max 8 min cells cctable + @ execute reset_emit ;
  : emit_char ( char -- )
    dup emit 
    255 and 
    dup 32 >= if 
      plus_x
    else dup 10 = if
      0 x !    
    else dup 16 = if
      ['] emit_color case_emit !
    then then then
    drop
  ;
  ' emit_char default_emit !
  reset_emit
  : emit ( char -- )
    case_emit @ execute 
  ;
  : type
    dup 0= if 2drop exit then
    1-
    for 
      dup @ emit 1+
    next 
    drop 
  ;
  : .\" POSTPONE s\" POSTPONE TYPE ; IMMEDIATE
  require random.fs
  : -! dup @ rot - swap ! ;
  : u/ swap abs swap abs / ;
  : setseed 255 and for rnd drop next ;
  cr cr cr
[ENDIF]

I reworked it according to you (almost).
It's not so easy to do, without the trick of using >variable default_emit<, which you set back to the correct address of the function, you end up with dependency recursion.

[ketmar]

The Forth terminology for strings is so confusing...

that's mostly because everybody have their own code for string handling. ;-)

remember that Forth came from computers with very small amount of RAM, so handling dynamic strings in RAM wasn't the high priority. most of the time people used BLOCKs to store big amount of data. so the system had a word to compile a string literal, and that's mostly it.

I added to the conversion script support for "local variables" which gforth does.
It seems to have the syntax >{ name1 name2 name3 ... -- just a comment }<

When I looked at what it does, it can be manually rewritten as

...
VALUE fn_name_name3
VALUE fn_name_name2
VALUE fn_name_name1

It is prefixed with the function name because these are global variables and gforth can have the same names in every function/word.

yeah, most of the time globals will do the trick. but the best way to deal with locals is to throw away such code. ;-)

https://rosettacode.org/wiki/Cycle_detection#Forth

this is awful C code writen with Forth syntax for some reason. and it is using locals for no reason at all (besides being a disguised C code, of course).

I came across some https://rosettacode.org/wiki/Man_or_boy_test by Donald Knuth. But even gforth broke his teeth on that code.

this is mostly because that thing tries to test something Forth neither have, nor need at all. ;-) while it is possible to implement it (and slightly better than shown there), there is no reason to do it. actually, it is easier to directly add closures and lambdas to Forth compiler than trying to emulate them. the only problem you'll have after doing that is "why the hell i had spent my time on this useless thing?" ;-)

actually, most Forth code you may find in teh internets is BAD. it ranging from simply bad to hilariously ugly and bad.

: emit_color ( idx ) 255 and 0 max 8 min cells cctable + @ execute reset_emit ;

"255 and" already converted the number to positive, so "0 max" is not needed here, it is effectively a NOOP.

also, you can use another trick: "10 - dup if 6 - if exit then ['] emit_color case_emit else x then !" instead of emulating "CASE". it is ugly and barely readable, but fun. ;-)

don't be afraid of using "EXIT": it is bad for "normal languages", but Forth words are usually very small, and "EXIT" is not that bad there. tbh, i'd rewrite your "emit_char" like this:

Code: Select all

: emit-printable  ( char ) emit plus_x ;
: emit-control  ( char ) 10 - dup if 6 - if exit then ['] emit_color case_emit else 10 emit x then ! ;
: emit-char ( char )  dup 32 >= if emit-printable else emit-control then ;

"emit-char" is definitely a high-level definition, so it should look closer to pseudocode. "emit-control" is ugly, though, and prolly need to be factored to two words too. basically, there is a rule of thumb: one word should rarely have more than one "IF". several IFs are hard to read, and i am usually using them as preconditions checks with explicit "exit" (like your "dup 0= if 2drop exit then").

(p.s.: i'd move "emit-printable" and "emit-control" to the separate vocabulary, to get rid of "emit-" prefix. it is nothing more than a crude emulation of namespaces anyway, and vocabularies do namespaces much better.)

It's not so easy to do, without the trick of using >variable default_emit<, which you set back to the correct address of the function, you end up with dependency recursion.

yeah, mutual recursion is ugly. but it is tolerable here, i believe.

[_dw]

I incorporated your solution into the code and it stopped working.
So I gradually converted my code to yours in small steps and ended up with an unbalanced stack. I tried again.. and again.. and I thought why am I doing this when the original code worked? .)

I finally came up with a workable solution.

Code: Select all

  : emit_printable  ( char ) emit plus_x ;
  : emit_control  ( char ) dup emit 255 and 10 - dup if 6 - if exit then ['] emit_color case_emit else x then ! ;
  : emit_char ( char ) dup 255 and 32 >= if emit_printable else emit_control then ;

--------------

Do you have port or beeper/speaker support in Forth? It seems that the standard does not address this and even gForth does not support a beeper/speaker.

I reworked the program from https://rosettacode.org/wiki/Morse_code#Forth

and this really can't be compiled. Redefines basic letters and numbers to words/functions. It even does it somehow "locally" and then returns to the original definition. It saves written strokes from the string to PAD (another one I looked at and ignored when supporting the standard) and then pulls them out from PAD as words/functions and not as written words.

So I had to completely rewrite it (I added support for ZX_BEEPER calling the ZX ROM function) and normal forth would still start saying that >c"< must be inside the word definition. But otherwise it is a forth/c program that can be compiled in M4 FORTH.

https://codeberg.org/DW0RKiN/M4_FORTH/s ... e_code.fth

PS: https://morse.halb.it/?channel=ch1
I recommend training on the "training" channel first (https://morse.halb.it/?channel=training), because it is quite difficult. Also change wpm and try to understand the controls and how it tells you when it's "." and when "_" and when the timer is shown, and where you can delete it before sending if it's nonsense. But it's probably only for radio amateurs and not greenies.

[ketmar]

I incorporated your solution into the code and it stopped working.
So I gradually converted my code to yours in small steps and ended up with an unbalanced stack. I tried again.. and again.. and I thought why am I doing this when the original code worked? .)
I finally came up with a workable solution.

i must confess that i haven't read your code carefully enough, and wrote my own without testing. ;-) it was meant to show you the idea, not to be copypasted anyway. sorry.

don't take my words as The Ultimate Truth, please: i am trying to show you a better, more… "Forth-like" way, but not The Best One. the idea is to get you on track, so you could write your own version, preferably better than mine. ;-)

Do you have port or beeper/speaker support in Forth? It seems that the standard does not address this and even gForth does not support a beeper/speaker.

you usually cannot do it in GNU/Linux without root access, because access to speaker device is restricted. the easiest way is to write raw PCM data to a file, and then play it with something like `aplay`.

I reworked the program from https://rosettacode.org/wiki/Morse_code#Forth

and this really can't be compiled. Redefines basic letters and numbers to words/functions. It even does it somehow "locally" and then returns to the original definition. It saves written strokes from the string to PAD (another one I looked at and ignored when supporting the standard) and then pulls them out from PAD as words/functions and not as written words.

that code is awful too. the idea is nice, but the implementation is not. the standard is very limited, so the author had to resort to "EVALUATE" (which is ugly by itself). and i cannot say what is wrong there, because 92, ANS, 2012, and current standards are all slightly different, and to be honest, i can't care less about standards and their differences. ;-) UrForth/x86 tried to be "standard" (more or less), but UrForth/Beast doesn't even have "IF" in the base system. ;-)

So I had to completely rewrite it (I added support for ZX_BEEPER calling the ZX ROM function) and normal forth would still start saying that >c"< must be inside the word definition. But otherwise it is a forth/c program that can be compiled in M4 FORTH.

yeah, you cannot create string tables this way in ANS. actually, there is no standard way to create string tables at all.

but i like your code. ;-) it looks less C then your previous programs. much less. "MORSE_CSTRING_EMIT" is still way to big, tho. i'd do it slightly different. first, shorten the table by removing lower-case variants: you can upcase the letter instead. second, i'd create something like "translate ( char — idx )", which will take ASCII char, and translate it into index in your table (returning "0" for invalid chars, and have the first table entry as an empty one). then create "beep ( duration )", because dot and dash are different only in duration. so the final code could look like this (rough untested draft! ;-):

Code: Select all

: xencode ( dot-dash — duration )  46 = if dit_dur else dah_dur then ;
: morse-emit  ( morse-char )  xencode beep char-dur pause ;
: morse-type  ( addr count )  over + swap do i c@ morse-emit loop ;
: try-blank ( ascii ) 32 = if space-dur pause then ;
: try-other  ( ascii )  translate 2* mtable + @ count morse-type ;
: char-emit ( ascii ) dup try-blank try-other ;

now we have one word for one action (roughly), and each word is small enough to debug it in your head. this is more-or-less how good and readable Forth code should look like: small words, consisting of 5-8 calls to other words. the only ugly word (which i didn't wrote here) is "translate", where you have to have a "CASE". or create yet another translation table, with byte indicies.

I recommend training on the "training" channel first (https://morse.halb.it/?channel=training), because it is quite difficult. Also change wpm and try to understand the controls and how it tells you when it's "." and when "_" and when the timer is shown, and where you can delete it before sending if it's nonsense. But it's probably only for radio amateurs and not greenies.

i always wanted to learn Morse code, but sadly, i am still too lazy to do it. ;-)