Performance of Forth

[animaal]

I was curious as to how the performance of Forth (Abersoft) relates to Sinclair BASIC and raw Z80. So I tested it. It's a very simple and far from comprehensive test, but perhaps somewhat indicative.

The aim is to fill the screen with pixels, and set all attributes to 255. the code is simple, and enclosed below for each language.
Spoiler: Forth is pretty good! I wonder if it was ever used to write commercial Spectrum software?


Results (Time taken):
basic : 72.50secs
forth : 1.50secs
assembly: 0.04secs


Disclaimer:
For each language, the most simple/obvious approach was chosen. The general intent was to compare languages by implementing a loop with 6912 iterations each setting a single memory location (although it's debatable that a Z80 LDIR instruction is exactly that).

There are ways each language's implementation could be optimised. E.g. the Z80 implementation would be far quicker with an unrolled loop of PUSHes into display memory. If the aim is to set all pixels to 1 and all attributes to White, then a quicker BASIC implementation might be 24x instructions PRINTing a string of 32 UDG characters (or INVERTed spaces) with INK, PAPER and BRIGHT set. But that might be considered cheating



CODE:

Code: Select all

; Z80 Assembly
; LDIR tstates = (21*6911)-5 = 145126 : approx 0.04secs
ld hl,16384
ld de,16385
ld (hl),255
ld bc,6911
ldir
ret

Code: Select all

( forth)
: FILLIN 23296 16384 DO 255 I ! LOOP ;
FILLIN

Code: Select all

10 REM BASIC
20 FOR i=16384 to 23295: POKE i,255: NEXT i
run

[ketmar]

but... it's not fair! if we're going to use LDIR in asm, then with Forth, we could use FILL! ;-) but really, it is most simple/obvious solution for Forth system, that's why it has FILL/ERASE.

sorry, i just couldn't resist the temptation. i see that you wanted to benchmark basic language operators.

also, it's better to use "C!" instead of "!", it should be marginally faster (and won't overwrite extra byte). or use "-1 ! 2 +LOOP" to utilise 16-bit values.

anyway, Aberforth is not the fastest one around. it is FIG system with ITC. SoloForth (or my latest unreleased dsForth ;-) is slightly faster due to DTC (~1.2 times faster).

i don't know about commercial software, but i will definitely use dsForth for several of my future projects. there was White Lightning too, but it looks like it wasn't very succesfull.

basically, if you have a rich system, you may be out of RAM quite fast. and no native ZX Forth system could create "turnkey" apps by throwing away all unused words. Forth code itself is quite compact, but you still need RAM for gfx, and it may be alot. at least in my case i'm wroking on PC-based cross-compiler, which will automatically create minimal kernels, otherwise i simply won't have enough RAM for anything more-or-less serious. this may be one of the reasons of not using Forth for commercial ZX developing, i guess.

[+3code]

As Spectrum games in Forth I only remember now this:
http://programandala.net/en.program.tron_0xf.html

[animaal]

Very interesting!

I never developed in Forth, but was always a little curious. It was certainly a brave choice for the Jupiter Ace to come with Forth instead of BASIC.

I can't imagine writing an arcade game in Forth, but seeing Tron above, it's obviously possible. I don't see Forth as a filter when searching for games on SC, so it obviously wasn't popular with professional Speccy developers back in the day

It's a fair point about FILL vs LDIR. LDIR could be turned into a simple "real" loop, losing some performance. Either way, I'm impressed by the speed of Forth.

For me as a youngster, Forth would have been a nice balance between the slow but friendly BASIC, and the speedy but unforgiving Z80. I missed out.

[p13z]

Code: Select all

10 PRINT INK 7; BRIGHT 1; FLASH 1; " *page full of solid gfx* "

Misses the last two lines - but POKEing the display memory doesn't feel in the spirit of BASIC programming.
If LDIR can be used for this in MC - using PRINT in BASIC doesn't really seem like cheating.

[Morkin]

As a slight aside, I always wondered if anything was produced using White Lightning (the utility program, not the drink) back in the day. From what I remember, the commands were Forth based.

[animaal]

Code: Select all

10 PRINT INK 7; BRIGHT 1; FLASH 1; " *page full of solid gfx* "

Misses the last two lines - but POKEing the display memory doesn't feel in the spirit of BASIC programming.

Yeah, agreed. I did this for fun, using 24 separate PRINT statements. the phrase "#1" can be used to print to the last 2 lines; i.e. PRINT #1; INK 7; PAPER 7; BRIGHT 1; FLASH 1; at 0,0; "line full of solid gfx"

[ketmar]

As a slight aside, I always wondered if anything was produced using White Lightning (the utility program, not the drink) back in the day. From what I remember, the commands were Forth based.

yep, it is full-featured Forth system with alot of extensions for gfx (like Laser Basic, prolly even using the same base kernel), and even with interrupt-driven multitasking.

[+3code]

As a slight aside, I always wondered if anything was produced using White Lightning (the utility program, not the drink) back in the day. From what I remember, the commands were Forth based.

Yes, it's a standard Forth with a sprite library/utility or so.

[Lethargeek]

Code: Select all

( forth)
: FILLIN 23296 16384 DO 255 I ! LOOP ;
FILLIN

try this:

Code: Select all

: FILLIN 255 23296 16384 DO DUP I C! LOOP DROP ;

might be a bit faster than literal

or 16-bit version:

Code: Select all

: FILLIN 65535 23296 16384 DO DUP I ! 2 +LOOP DROP ;

[animaal]

try this:

Code: Select all

: FILLIN 255 23296 16384 DO DUP I C! LOOP DROP ;

might be a bit faster than literal

or 16-bit version:

Code: Select all

: FILLIN 65535 23296 16384 DO DUP I ! 2 +LOOP DROP ;

To be honest, my stopwatch skills aren't really accurate enough to do justice here... but the second version seems to be significantly faster. Maybe a little over 1 second on my Spectaculator. Makes sense when there are half the number of iterations.

[Joefish]

White Lightning included a copyright notice that anything written in it had to be offered up to Oasis Software to publish. I doubt anyone ever bothered. The graphics libraries were powerful enough (they resurfaced in Laser Basic) but the Forth editor they provided was a terrible thing to program in.

[Alone Coder]

It is possible to make a Forth system that generates call:call:call instead of dw:dw:dw. This will be a lot faster.

[catmeows]

It is possible to make a Forth system that generates call:call:call instead of dw:dw:dw. This will be a lot faster.

Beauty of Forth is that you can customize performance and code density and you can quickly build a domain language on top od it. I'm using tokenized (8-bit tokens) Forth-like langugage scripts in Black Flag. IT Is much easier to manage game logic in Forth than in asm.

[ketmar]

This will be a lot faster.

no, it won't.

[Sokurah]

The graphics libraries were powerful enough (they resurfaced in Laser Basic) but the Forth editor they provided was a terrible thing to program in.

Well, at least the manual was awesome

[Joefish]

Beauty of Forth is that you can customize performance and code density and you can quickly build a domain language on top od it. I'm using tokenized (8-bit tokens) Forth-like langugage scripts in Black Flag. IT Is much easier to manage game logic in Forth than in asm.

The real attraction of Forth is that it's very easy to write an interpreter for it on pretty much any machine-code architecture. It's then fairly easy to write simple scripted actions in it, so it's a good choice for something like this. I'm adding a script engine to Go-Go BunnyGun at the moment, although I'm not sure I'm past the tipping point of implementing a fully-featured language like Forth. Although if I needed any maths functions I probably would do it as a Forth stack calculator.

The problem comes when you subject some other poor sod to the thing you wrote!

[Lethargeek]

This will be a lot faster.

no, it won't.

it might, if short primitives are inlined and the code is not too heavy on hi-level words

[ketmar]

no, it won't.

it might, if short primitives are inlined and the code is not too heavy on hi-level words

which will be a mostly useless microbenchmark code. ;-) real benefits (on a real-word code) from STC (over DTC) is not even x2 most of the time. and to get close to x2 you have to have a quite sophisticated peephole optimiser, which will take most of the free RAM for itself. pretty straightforward x86 DTC UrForth is about 1.9 times slower than optimised STC BigForth (and about 1.2 times slower than unoptimised SP-Forth with branches/loops inlined). and this is on x86, with its rich choice of addressing modes. so for real apps it can be slightly faster (and the code is about 1.3 times bigger). there is simply no way to make it "alot faster" if you won't revert to pure asm, or won't stick with specially crafted microbenchmarks.

[Lethargeek]

which will be a mostly useless microbenchmark code

no, which may well be an actual game code, as it tends to be low-level most of the time

which will be a mostly useless microbenchmark code. real benefits (on a real-word code) from STC (over DTC) is not even x2 most of the time. and to get close to x2 you have to have a quite sophisticated peephole optimiser, which will take most of the free RAM for itself

nothing sophisticated and not much RAM for a simple optimizer doing a fusion of the last 2-3 primitives

pretty straightforward x86 DTC UrForth is about 1.9 times slower than optimised STC BigForth (and about 1.2 times slower than unoptimised SP-Forth with branches/loops inlined). and this is on x86, with its rich choice of addressing modes. so for real apps it can be slightly faster (and the code is about 1.3 times bigger). there is simply no way to make it "alot faster" if you won't revert to pure asm, or won't stick with specially crafted microbenchmarks.

as i said, it all depends on a hi-level vs low-level ops ratio in the code
with z80 you have a choice optimising either parameter stack ops OR return stack ops
but with things like 6809/6309 you can do both (and not just forth but any threaded code)

[ketmar]

which will be a mostly useless microbenchmark code

no, which may well be an actual game code, as it tends to be low-level most of the time

why? almost nobody's going to write gfx kernel (or something like that) in ZX Forth anyway, it makes little sense. and the more high-level code, where Forth really shines, calls alot of high-level words (or the authors are doing it wrong, and creating a write-only mess even they won't be able undestand in a week).

which will be a mostly useless microbenchmark code. ;-) real benefits (on a real-word code) from STC (over DTC) is not even x2 most of the time. and to get close to x2 you have to have a quite sophisticated peephole optimiser, which will take most of the free RAM for itself

nothing sophisticated and not much RAM for a simple optimizer doing a fusion of the last 2-3 primitives

this doesn't have much sense without removing intermediate stack operations. that's mostly what unoptimised SP-Forth does, and it has very little impact. and on x86 we have "xchg esp,ebp" to quickly swtich between our stacks. simply optimising away stack switches is not enough to get huge speedups, you have to use registers for intermediate values. so you need at least basic-block peephole optimiser and register allocator. simple optimiser will not give you even more-or-less stable x2. so to get more speed you need to hardcode alot of special cases, and that quickly gets out of control. just take a look at any decent optimising STC compiler: it is either a mess of normal/inlineable code all over the place, or a huge list of peephole rules. or both. and it still cannot beat simple DTC even to x4 (which is not that huge after all).

that's why i abandoned optimising STC after some R&D: it's complexity simply doesn't pay off.

p.s.: of course, comparing Z80 and x86 is kinda like comparing apples and oranges, but the basic numbers are very close, and x86 is what i had tested recently, so i can operate with real results i've seen, instead of trying to remember exact numbers for unfinished R&D compilers.

p.p.s.: that is, STC can be faster, of course, but it is not alot faster, and creating good STC code is much more complex task than simply using DTC. and it's complexity is not worth the final speed gain, i think. of course, i'd be glad to be wrong here, because there cannot be enough speed. ;-)

[Lethargeek]

why? almost nobody's going to write gfx kernel (or something like that) in ZX Forth anyway, it makes little sense. and the more high-level code, where Forth really shines, calls alot of high-level words (or the authors are doing it wrong, and creating a write-only mess even they won't be able undestand in a week).

who said gfx kernel? low-level forth is more than enough for game logic
just look at the Shaw brothers games that are compiled (integer and very low-level) BASIC

this doesn't have much sense without removing intermediate stack operations. that's mostly what unoptimised SP-Forth does, and it has very little impact. and on x86 we have "xchg esp,ebp" to quickly swtich between our stacks. simply optimising away stack switches is not enough to get huge speedups, you have to use registers for intermediate values. so you need at least basic-block peephole optimiser and register allocator.

but this IS simple - even possible just using macros in the assembler

simple optimiser will not give you even more-or-less stable x2. so to get more speed you need to hardcode alot of special cases, and that quickly gets out of control. just take a look at any decent optimising STC compiler: it is either a mess of normal/inlineable code all over the place, or a huge list of peephole rules. or both. and it still cannot beat simple DTC even to x4 (which is not that huge after all).

repeat - it all depends on the hi/lo level code ratio, in your use cases it won't, in my use cases it will

p.s.: of course, comparing Z80 and x86 is kinda like comparing apples and oranges, but the basic numbers are very close, and x86 is what i had tested recently, so i can operate with real results i've seen, instead of trying to remember exact numbers for unfinished R&D compilers.

NO, and don't even bring 16 bits here, these things are fundamentally different

p.p.s.: that is, STC can be faster, of course, but it is not alot faster, and creating good STC code is much more complex task than simply using DTC. and it's complexity is not worth the final speed gain, i think. of course, i'd be glad to be wrong here, because there cannot be enough speed.

using hardware stack as parameter stack gives a boost
also don't forget that inlined primitives lack NEXT
for example, C! becomes just

Code: Select all

pop de
ld (hl), e
pop hl

...and then the fusion might eliminate "pop de" and/or "pop hl" checking just a few possible register states
(OTOH return stack handling becomes slower and uglier, possibly demanding alternative registers use)

[catmeows]

who said gfx kernel? low-level forth is more than enough for game logic

I tend to agree here, my general vocabulary is quite simple. I may add or drop some definitions in future, but in general it is all I need.
Only MIN and MAX are compound words, rest is native implementation.

Code: Select all

uForthTable
	jr uForthDup	;00 DUP
	jr uForthDrop	;02 DROP
	jr uForthSwap	;04 SWAP
	jr uForthOver	;06 OVER
	jr uForthRot	;08 ROT
	jr uForthPlus	;0A PLUS
	jr uForthMinus	;0C MINUS
	jr uForthEqual	;0E EQUAL
	jr uForthNotEqu	;10 NEQUAL
	jr uForthGreat	;12 GREAT
	jr uForthLess	;14 LESS
	jr uForthEqLess	;16 EQLESS
	jr uForthEqGrt  ;18 EQGREAT
	jr uForthOne	;1A ONE
	jr uForthZero	;1C ZERO
	jr uForthInc	;1E INC
	jr uForthDec	;20 DEC
	jp uForthBranch ;22 BRANCH
	jp uForthBrTrue ;25 BRTRUE
	jp uForthBrFals	;28 BRFALSE
	jp uForthToR	;2B TOR
	jp uForthFromR	;2E FROMR
	jp uForthCpR	;31 COPYR
	jp uForthSubr	;34 SUBR
	jp uForthReturn ;37 RETURN
	jp uForthAbs	;3A ABS
	jp uForthNeg	;3D NEG
	jp uForthMin	;40 MIN
	jp uForthMax	;43 MAX
	jp uForthSgn	;46 SGN
	jp uForthAnd	;49 ANDOP
	jp uForthOr	;4C OROP
	jp uForthCpl	;4F CPLOP
	jp uForthMOne	;52 MONE
	jp uForthXor	;55 XOROP
	jp uForthCByte	;58 CBYTE
	jp uForthCInt	;5B CINT
	jp uForthBFetch ;5E BFETCH
	jp uForthIFetch ;61 IFETCH
	jp uForthBStore ;64 BSTORE
	jp uForthIStore ;67 ISTORE
	jp uForthMul	;7A MUL
	jp uForthDiv	;7D DIV 
	jp uForthShl	;80 SHLOP
	jp uForthShr	;83 SHROP
	jp uForthShra	;86 SHRAOP
	jp uForthCUByte ;89 CUBYTE
	jp uForthCBFetch ;8C UBFETCH
	jp uForthExit	;8F EXIT

[ketmar]

[mention]Lethargeek[/mention] it seems to me that we're talking about slightly different things here. seems that you're talking about small and simple Forth-like scripts, and i am talking about writing everything except the very lowest parts in Forth. that can explain why you have mostly words calling primitives, and i have mostly words calling other high-level words. those are two very different things indeed, and STC is much faster in your case (but still not enough to call it "alot" for me ;-). and that's prolly why i got mostly similar results for x86 and Z80 (as i said, the relative numbers were very close). (and that's why i started cross-compiler project in the first place -- word headers overhead became quite noticeable.)

so i agree with you that for "script-like" use cases there might be a good reason to switch to STC, and it may give some very noticeable speed boost.

also, i wonder if [mention]animaal[/mention] silently writing some words to DROP us with our offtopic arguments.

[programandala.net]

I was curious as to how the performance of Forth (Abersoft) relates to Sinclair BASIC and raw Z80. So I tested it. It's a very simple and far from comprehensive test, but perhaps somewhat indicative.

The aim is to fill the screen with pixels, and set all attributes to 255. the code is simple, and enclosed below for each language.
Spoiler: Forth is pretty good! I wonder if it was ever used to write commercial Spectrum software?


Results (Time taken):
basic : 72.50secs
forth : 1.50secs
assembly: 0.04secs

Code: Select all

( forth)
: FILLIN 23296 16384 DO 255 I ! LOOP ;
FILLIN

I couldn't resist the curiosity. I've added a new benchmark to Solo Forh (version 0.14.0-rc.124 was recently released) to try the 5 possible options to code that in Forth. I copy the code and the results below, in ticks (system frames) and milliseconds:

Code: Select all

( scr-fill-bench )

  \ 2020-11-26: Benchmark written after the test found in the
  \ following forum thread, titled "Perfomance of Forth":
  \ https://spectrumcomputing.co.uk/forums/viewtopic.php?f=6&t=3487

need dticks need reset-dticks need do need +loop need dticks>ms

: end ( d ca len -- ) cr type ." : " 2dup d. ."  ticks (" 
                      dticks>ms d. ." ms)" key drop ;
  \ Display the result of a benchmark. _d_ are the ticks and
  \ _ca len_ is the name of the bench.

: fill-16b ( -- ) reset-dticks 23296 16384
  do 255 i ! loop dticks s" 16b-loop" end ;
  \ The original method used in the forum.

: fill-8b ( -- ) reset-dticks 23296 16384
  do 255 i c! loop dticks s" 8b-loop" end ;
  \ Simpler and a bit faster 8-bit variant.

: fill-16b+ ( -- ) reset-dticks 23296 16384
  do 65535 i ! 2 +loop dticks s" 16b-+loop" end ;
  \ Much faster 16-bit variant with a 2-byte loop step.

: fill-16b+dup ( -- ) reset-dticks 65535 23296 16384
  do dup i ! 2 +loop drop dticks s" 16b-dup-+loop" end ;
  \ A bit faster variant that duplicates the value.

: pure-fill ( -- ) reset-dticks 16384 6912 255
  fill dticks s" pure" end ;
  \ The fastest option by far, a pure Forth loop-less `fill`.

: scr-fill-bench ( -- ) fill-16b fill-8b fill-16b+ fill-16b+dup
  pure-fill ;  scr-fill-bench

  \ 2020-11-26: Results:

  \ Test            ticks    ms
  \ -----           -----  ----
  \ fill-16b           59  1180
  \ fill-8b            57  1140
  \ fill-16b+          36   720
  \ fill-16b+dup       35   700
  \ pure-fill           2    40

The test has been run on an emulated ZX Spectrum 128 with a Plus D disk interface, using the Fuse emulator on Debian.

Note Solo Forth is DTC, while Abersoft Forth is an old ITC fig-Forth.