Forth - better than BASIC?

[ParadigmShifter]

Watched this yesterday, ketmar may want to post a response lol

Weird programming language, feels a bit like a cult, ha ha

[ketmar]

Weird programming language, feels a bit like a cult, ha ha

yes, it definitely IS a cult! join us, we have reversed polish cookies! ;-)

but the video is full of errors. starting from relatively small ones (Forth was NOT invented to control the telescope; at that time Forth system was already quite mature), and to more fundamental ones: Forth is not about stacks at all (it is almost a side effect), it is about interactivity and flexibility.

for example, UrForth/C debugger doesn't even have a command line processor. because i am too lazy to write something i already have: the very same interpreter that drives the main system is perfectly usable in this case. i mean, exactly the same: i'm just putting the address of the new input buffer into the system variable, and call INTERPRET. and the main loop that reads and compiles ordinary Forth system code looks like this:

Code: Select all

BEGIN
  RP0!
  INTERPRET
AGAIN

the same INTERPRET word.

special debugger commands? easy deal: they're simply sit in a separate vocabulary (namespace, if you want), and i instruct the interpreter to look there first. this way, my debugger has special debugger commands avaliable, and the whole Forth system too. yay, no need to write any kind of scripting for it, i have fully-featured Forth system there! ;-)

that's one of the examples of Forth flexibility. small one, of course, it could do more. like structures, for example: done with the same namespaces and a bit of compiler hacking. OOP? yes, we have it, done in the same spirit. and so on.

[ketmar]

just for fun, simple OOP implementation:

Spoiler

Code: Select all

;; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Mini-OOF 12apr98py (Bernd Paysan)
;; https://bernd-paysan.de/mini-oof.html
;; this is small, but fully usable OOP system, with early and late binding
;; see "samples/minioof-demo.f"
;; Ketmar Dark: added parent class pointer and some helper words
;; Ketmar Dark: ported verbatim from UrForth/x86
;; class layout:
;;   cell instvars_size  (in bytes)
;;   cell methods_size   (in bytes)
;;   cell parent_class   (pointer to the parent class, added by k8)
;;   method pointers follows (vmt)
;; first object instance var is always a pointer to the class
;;
: method  ( m v -- m' v )  create over , swap cell+ swap
  does>  ( ... o -- ... )  @ over @ + @execute ;
: var  ( m v size -- m v' )  create over , +
  does>  ( o -- addr )  @ + ;
: class  ( class -- class methods vars )  dup 2@ ;
: (end-class)  ( class methods vars -- newclass )
  here >r ,  ;; put instance size
  dup ,      ;; put class size
  over ,     ;; put pointer to the parent class
  ;; fill vmt with "not implemented"
  3 cells ?DO ['] forth:(NOTIMPL) , 1 cells +LOOP
  ;; copy original class vmt  ( class | newclass )
  cell+ dup 2 +cells r@ rot @ 3 cells string:/string move r>
; (hidden)
: end-class  ( class methods vars -- )  create (end-class) create; drop ;
: defines  ( xt class -- )  ' cfa->pfa @ + ! ;
: new  ( class -- o )  dup @ n-allot 2dup swap @ erase dup nrot ! ;
: ::  ( class "name" -- ... )  ' cfa->pfa @ + @ compile, ;  \ early binding
\ some extended methods
alias @ (@class)  ( instance -- classptr )  (hidden)  \ get object instance class ptr
: (@class-parent)  ( classptr -- classptr )  [ 2 cells ] imm-literal + @ ; (hidden)  \ get class parent ptr (result can be 0)
: @class  ( instance -- classptr )  dup if (@class) then ;  \ get object instance class ptr
: @class-parent  ( classptr -- classptr )  dup if (@class-parent) endif ;  \ get class parent ptr (result can be 0)
\ dynamic binding (k8)
: (m-exec)  ( ... classptr mtofs -- ... )  + @execute-tail ; (hidden)
: m-invoke  ( instance class "name" -- ... )  compiler:?comp
  ' cfa->pfa @ [compile] literal compile (m-exec) ; immediate
\ dynamic binding (k8)
: m-inherited  ( instance "name" -- ... )  compiler:?comp
  compile dup compile (@class) compile (@class-parent)
  ' cfa->pfa @ [compile] literal compile (m-exec) ; immediate
\ create root object
create Object
  1 cells ,  ;; instvars size (class pointer is always there)
  3 cells ,  ;; methods offset
  0 ,        ;; pointer to the parent class (object has none)
create;

yes, that's it. an ordinary Forth program. and there's how you use it:

Spoiler

Code: Select all

: .hex8  ( n -- )  BASE @ SWAP HEX <# # # # # # # # # #> TYPE BASE ! ;

." ROOT OBJECT=0x" object .hex8 cr

Object class
  cell var text
  cell var len
  cell var x
  cell var y
  method init
  method draw
end-class Button
." BUTTON=0x" Button .hex8 cr

;; Now, implement the two methods, draw and init:

:noname  ( o -- ) >r
  \ r@ x @ r@ y @ at-xy
  ." x=" r@ x @ 0 .r ." ; y=" r@ y @ 0 .r ." ; text=<"
  r@ text @ r> len @ type
  ." >"
;
Button defines draw

:noname  ( addr u o -- ) >r
." INIT: class=0x" r@ @class .hex8 ." ; parentclass=0x" r@ @class @class-parent .hex8 cr
  6 r@ x ! 12 r@ y ! dup r@ len !
  ;; allocate string at "here"
  dup n-allot dup r> text ! swap move
;
Button defines init

;; For inheritance, we define a class bold-button, with no new data and no new methods.
Button class
end-class Bold-Button
." BOLD-BUTTON=0x" Bold-Button .hex8 cr

: bold    ( .ansi-escape ." 1m" ) ." [BOLD]" ;
: normal  ( .ansi-escape ." 0m" ) ." [NORMAL]" ;

\ this does early (static) binding
\ :noname  ( o -- ) bold [ Button :: draw ] normal ;
\ this does dynamic binding
:noname  ( o -- ) bold  m-inherited draw  normal ;
Bold-Button defines draw

;; And finally, some code to demonstrate how to create objects and apply methods:
Button new constant foo
" thin foo" foo init
\ page
foo draw cr
Bold-Button new constant bar
" fat bar" bar init
bar y 1+!
2 bar x +!
bar draw cr

the output:

Code: Select all

ROOT OBJECT=0x000110FC
BUTTON=0x0001127C
BOLD-BUTTON=0x0001143C
INIT: class=0x0001127C; parentclass=0x000110FC
x=6; y=12; text=<thin foo>
INIT: class=0x0001143C; parentclass=0x0001127C
[BOLD]x=8; y=13; text=<fat bar>[NORMAL]

yes, it's that easy. 2KB of code for fully-featured OOP implementation, with inheritance and virtual methods.

[ParadigmShifter]

Well it's still 2K of code.

Once you know how to implement OOP in C it's easy to convert that to ASM anyway. Not that OOP is a holy grail kind of thing

Advantage of doing it in C/C++ or converting that to ASM, dunno how this compares to your implementation, but if it is simple it probably isn't as flexible as a simple C implementation translated to ASM would be.

You don't have to allocate objects on the free store/heap.
You can preconstruct objects so you don't have to do any initialisation at run time if they are global. If they are dynamically allocated you can also have a pooled allocator for certain objects which already have vtables etc. already set up for you when you create them. (Last PS1 game I worked on had entire packets of 16 gouraud shaded quads already set up with all the polygon type IDs that were required all set up and never overwritten by code. Those were allocated from a pool via a stack allocator - so however many there were existed in global memory and there was a stack which had addresses of available packets pushed onto it. To allocate one you just popped the address off the stack and to free pushed the address back onto the stack).

You can avoid making a virtual call if you know the object type in advance.

Only hassle with OOP in C is having to call init and destroy functions manually. But at least you can return an error from those unlike C++ constructors (which have to throw errors).

To do OOP stuff in C

Overloading: just decorate name with the argument types. You can overload on return type if you wish as well then
Inheritance: Just have struct Base as the first elements of struct Derived
Virtual Functions: just use a vtable (or you can choose to copy the vtable directly into the class if you don't want an extra dereference for vtbl->func()). Only for the most derived class of course.
Member functions: just pass the this pointer as first argument (or in a register)

In C it's easy to have private members as well (you just have a separate header with the real implementation and the public header hides the implementation - e.g. FILE pointers in C).


EDIT: Forth does have the nice feature that you can use the interactive read evaluate loop to create objects you need and then save your code once you have done that of course. LISP and even Sinclair Basic allow you to do that though (you can RUN a program to initialise a load of variables then delete the code if you want, as long as you don't then CLEAR (and does RUN also do that? Can't remember, but you can always GOTO 0)). Not that using BASIC just for that is a good plan

And all the above optimisations can be done by making the compiler more sophisticated of course. C++ compilers these days can run loads of code before they emit any object code (with template metaprogramming and stuff like that). Not that C++ is a nice language or anything lol

[ketmar]

basically, everything you described is implemented in that 2K code. and you don't have to do any manual work anymore: the compiler takes care of initialising internal fields for you. you're extending the compiler itself, not bolting some messy macro-magic on top of the existing rigid one.

the actual implementation i am using is slightly bigger (about 8KB), but it allows accessing object fields inside a method body without "r@", just by name. it automatically tracks "self" for you. it has O(1) "isa" checks, and check that in runtime, so accessing invalid object is properly caught. it supports dynamic dispatching with strings (you can call method by its name). it has easy calls to inherited methods (either simple "inherited", or "inherited: <mtname>"). and so on. and you don't have to manually track anything, or manually create any internal data structures.

and this is just a normal Forth code, that can be loaded only if you need it. run-time overhead is "isa" checks (can be turned off), and indirect call via vmt — nothing more than you'd have by doing `vmt->func()`.

of course, you can create an object system which better fits your needs for a given task instead of using this one. they also could coexist. just show me how can i extend C compiler using the same C language in compile time to implement object-oriented extensions i need, and i'd say "ok, C is good enough too". ;-)

the startup time of the whole UrForth/C system is about 6 msecs. this includes compiling more than 50KB of Forth code. (and this is A LOT for Forth!) it is that slow because the kernel performs safety checks on each memory access; if i remove that, it prolly be about 1-2 msecs.

UrForth/x86 (native system) rebuilds itself in ~300 msecs (about 350KB of source code). this is full 2-stage rebuild, including self-tests. and it needs only UrForth/x86 binary to do that, nothing more. it also has built-in assembler, and many low-level words are written in asm. it was written from the scratch as a several weekends project. UrForth/C basically took several days to write (actually, it is at version 4 now, and each version was rewritten almost from the scratch too).

(tbh, the part that took a lot of time in native system was x86 asm. i HAET x86!)

now, how long it will take to write a C compiler from scratch? ;-) even if we omit heavy optimisations, it will prolly take several weeks, not weekends. and if i'll add native code generator to my Forth system (a weekend project again), it will easily outperform, for example, TCC. i simply don't need it right now, because it is almost of TCC speed already. and i have a way more powerful system to develop programs than a bare C compiler, btw: it includes simple editor, REPL, and built-in debugger (both low-level, and source code level).

it was easier to create that Naughty Dog's LISP, it is already faster without a sophisticated compiler (because LISP cannot be fast without heavy optimisations ;-), it doesn't need GC, and you can have fully-featured interpreter/compiler along with the other code (invaluable for debugging) on the target system, with cooperative multitasking built-in.

i am not using it for my public projects only because it would be a PITA for other people to build them. but i am using various UrForth incarnations (this is a whole family of Forth systems, actually) for internal coding a lot.

p.s.: btw, UrForth/x86 has full support for exception throwing, with the ability to add custom unwinders, and it supports OOP extension too (acutally, OOP extends the system with its custom unwinders).

[ParadigmShifter]

Well I wouldn't write a C compiler myself

But I regularly write code in C (on paper or in comments mainly) and turn it into ASM since it expresses the concept in a more high-level way.

I'm not dissing Forth in any way of course I stated in another thread if I was going to do a complex expression evaluator for an adventure game I'd basically implement a stripped-down version of Forth just for that. I'd probably write a script language compiler on the PC which emitted the data in the correct format rather than write directly in a Forth like language though. I imagine the Quill and GAC (don't know much about PAW) do exactly that kind of thing, rather than emit machine code instructions for the routines they emit, but that's just a guess

Infocom ZIL and Level 9 both went for a virtual machine I think (Infocom one compiled from a LISP like language, and Level 9 scripting language is unknown last I looked). Obviously to make cross platform development and reusing the code base for subsequent projects easy peasy.

It's good to be familiar with lots of languages anyway and how they are implemented, then you can pick the best tools for the job. I'm just saying OOP is very easy in C and C is very easy to turn into ASM since they are both procedural.

[ketmar]

and i'm not fighting with you too, i'm trying to convert you to Forth zealot. ;-)

[ParadigmShifter]

Yup I know the talk about different languages is always interesting though. ASM is of course a terrible language to develop in really, but needs must

There's a good reason so many languages these days are based on C though - they pretty much nailed it early on (adding function prototypes was needed though and it should not have been optional of course - but backwards compatibility and all that). That's why many languages still have the wrong precedence for several of the operators

Spoiler for why function prototypes should not be optional in C (and more like C++ tbh)

Spoiler

What arguments are you allowed to pass to this function? Why is that a bad idea? Clue: What happens differently in C++ if you extern "C" this prototype?

void func();

[ketmar]

actually, C is a terrible language; one of the worst i ever had a displeasure to use. and it is sad that it was hyped so much, and that people are still see at as something to draw inspiration from. Modula-2 is much, much better, and Oberon basically closed the need to invent new imperative languages.

most of the software i writing for myself these days is done with BlackBox Component Builder (if it is not Forth ;-). with heavily reworked GNU/Linux native version, of course. again, i wouldn't touch C with a nine miles pole if i haven't a need for other people to easily build my public code.

(don't be misguided by "Component Pascal" name: it is just a marketing ploy, the language is actually slightly modified Oberon, and the system was called Oberon/F in its early days.)

C is worse than BASIC even: i prolly could turn BASIC programmer into a real one, but C programmers are mostly incurable. ;-)

[ParadigmShifter]

Well I didn't say C was great either, it does it's job though (and is easy to convert to assembly language).

When we were forced to code in C we didn't like it and wanted to code in C++. Then we saw terrible C++ code and thought maybe that was a bad idea

These days I use C#. For beginners (on PCs) I'd recommend Python or something (defo not Javascript lol).

C and Pascal are very similar anyway. EDIT: Except in C it's easy to do stuff most high level languages don't allow which is of course dangerous but not if you know what you are doing and use it wisely It's like abusing the stack in Z80 - if you know what you are doing it can be hugely beneficial but if you don't it's gonna come back and bite you in the arse.

C - gives you the power to shoot yourself in the foot.
C++ gives you the power to shoot yourself in the foot and REUSE THE BULLET

EDIT2: No language is perfect despite what Bjarne Stroustrup seems to say

EDIT3: Apt quote I can't remember from where though - "If the only tool in your toolbox is a hammer then every problem you encounter starts looking kinda like a nail"

[ketmar]

Well I didn't say C was great either, it does it's job though (and is easy to convert to assembly language).

a[i++] = b[++i] + c[i++]++;
asm, please. ;-)
(this is UB, of course, but it is still allowed C syntax. ;-)

When we were forced to code in C we didn't like it and wanted to code in C++. Then we saw terrible C++ code and thought maybe that was a bad idea ;)

terrible code in terrible language. while C is usable (barely), C++ is utter unusable garbage, made by somewhone who definitely had all his brain cells dead. ;-)

C and Pascal are very similar anyway. EDIT: Except in C it's easy to do stuff most high level languages don't allow which is of course dangerous but not if you know what you are doing and use it wisely ;)

that's a common misconception about C. the problem is that you HAVE to use every "dangerous" feature in the book to write more-or-less useful C code. it's not like you can stop using pointers, or have your array indices checked, for example. you simply CANNOT write safe code with C.

but in the same time, C tries to convince you that you can. "oh, you only need some self-control, and a lot of manual work, and you'll have a perfect illusion of safe code. not safe code itself, of course."

some stats: buffer overruns, use-after-free and use-pointer-to-somewhere bugs found in Oberon system since 1987: 0.
number of similar bugs found in any decent C project: about +inf.
(i mean "undetected bugs which can corrupt your data, or can be exploited.")
mind you, Oberon is fully-featured OS writen from scratch on bare metal, without other development tools available. by two people; in 6 monthes of their spare time. with TUI, GUI, dynamic Oberon code loading, and a lot of applications written over the years. still zero such bugs.

this alone is the reason to never ever use C if you're not forced to.

Forth, for example, never ever claimed that it will help you to write any safe code. there are zero safety nets, be them real or illusionary. but it gives you the way to extend the compiler, so some of my Forth software is way safer than C (almost the Oberon level of safety).

EDIT2: No language is perfect

except Oberon. ;-) which oscillates near local maximum. Oberon, Oberon-2, Component Pascal, Oberon-07 — it's still Oberon, adapted for the task at hand.

by the way. i think we should ask Peter to move this conversation to the separate topic. i feel somewhat… uncomfortable derailing my own thread, but i'll happily continue this flamefe… conversation in a new topic. ;-) are you agree?

[Wall_Axe]

I watched a video of a professional game dev saying that a C buffer overrun bug cost a lot of development time on his game. In the 90's.

[ketmar]

eh, just read recent CVEs for various software. nearly 90% (or even 99%) of them are either buffer overruns, or "use-after-free". that's what we got by using C to write real-world software.

p.s.: and by the way. my Doom sourceport has dynamic array class where it is impossible to turn off array bounds checking. yep, even in "release builds". also, there is no way to turn off asserts too. no performance problems so far, but i really feel much safer this way. (yep, C++. not my choice. but at least i am not using STL, or, god save me, Boost.)

[R-Tape]

On request of the thread owner - this is a placeholder for a split thread.

@ketmar are these the correct posts? PM me if any need to be moved back.

[ParadigmShifter]

Obviously Forth is better than basic

I wouldn't say the STL is bad either. iostreams are bad but they aren't part of the STL. <algorithm> is pretty good as are the containers.

What is bad is all the boilerplate code you have to write to make it so your objects can be placed in a container.

EDIT: Since it's all templates you can see what most implementations are doing as well which is a good way to look at how to implement algorithms efficiently.

[ketmar]

On request of the thread owner - this is a placeholder for a split thread.

@ketmar are these the correct posts? PM me if any need to be moved back.

everything seems to be ok. thank you a lot!

[ketmar]

templates are basically the total opposite of effectiveness. ;-) or you'll get a code which nobody (including the author) can read. in D i can use templates with static ifs to generate almost optimal code, and it will be readable. in C++ with its SFINAE it is simply not possible. besides, using STL with custom allocators is PITA. STL strings sux. STL vectors sux. STL iterators sux. STL <insert-anything-here> sux.

tbh, i'd better go with intrusive data structures and macro magic than with C++ templates. at least it will be effective and readable, despite the ugliness of C macro processor.. ;-)

templates are a very poor substitute for compile-time code execution and generation. LISP has real AST macros. Forth is built around the possibility to freely switch between interpretation and compilation. C++ simply sux. ;-)

that's how i create accessor words in Forth, for example:

Code: Select all

;; create "class->name"
: (create-ofs-accessor)  ( ofs-in-bytes addr count -- ofs-acc-cfa )
  " class->" string:>pad string:pad+cc
  string:pad-cc@ (create) latest-cfa swap
  , does> ( info-addr pfa -- info-addr-with-offset ) @ +
;

;; create "class->nameX"
: (create-accessor)  ( addr count ofs-acc-cfa cfa-r/w char -- )
  >r " class->" string:>pad 2swap string:pad+cc r> string:pad+char
  string:pad-cc@ compiler:(create-forth-header)
  swap compile, compile, compile forth:(exit) compiler:smudge
;

: (create-peek-accessor)  ( addr count ofs-acc-cfa -- )
  ['] @ [char] @ (create-accessor)
;

: (create-poke-accessor)  ( addr count ofs-acc-cfa -- )
  ['] ! [char] ! (create-accessor)
;

;; accessors
: new-class-info-accessor  ( ofs -- ofs+4 )  \ name
  parse-name 2>r  ( ofs | addr count )
  dup 2r@ (create-ofs-accessor)  ( ofs ofs-acc-cfa | addr count )
  dup 2r@ rot (create-peek-accessor)
      2r> rot (create-poke-accessor)
  cell+
;

0
new-class-info-accessor typeid
new-class-info-accessor inst-size
new-class-info-accessor level
new-class-info-accessor def-pfa
new-class-info-accessor vocid
constant (class-header-size)

and now i have: "class->typeid", "class->typeid@", "class->typeid!", and so on. of course, i could use <structs.f> module instead (because those words are actually struct field accessors), but meh…

i mean, that's how real adaptive code generation works. and there is simply no way to write something like this with C++ (but you can write thing like this with D, btw).

ah, it's hard to see what this line noise does if you don't know Forth good enough, mea culpa. ;-) it generates code like this:

Code: Select all

: class->typeid  0 + ;
: class->typeid@ class->typeid @ ;
: class->typeid! class->typeid ! ;

and such. i mean, exactly this, exactly the way "normal compiler" would compile it from the source. (almost. offset addition is slightly different, done with "does>".)

such code generation is the normal way to generate boilerplate code for Forthers. now, don't tell me about C++ templates, it's like comparing Lamborghini with a cardboard car model. ;-)

[Lethargeek]

Obviously Forth is better than basic

no big wonder as almost everything is better than basic

jokes aside, i think it's more correct to say that forth is much more useful for low-performance systems; even though it's inherently much more unsafe in its simplest form (you can easily screw the whole system with careless usage of many common ordinary ops - unlike most basics where you need a wrong poke address for the same effect); and even though sadly most 8-bit instruction sets are not well suited for low-level forth implementations (with a notable exception of 6809)

[ketmar]

to not spam XTE topic, i will spam here instead. ;-) as new UrAsm is one pass, i need some way to postpone evaluation of expressions with yet unknown labels. it can be done in several ways: by recording fixup info (labels with associated operators), by simply saving a string and re-evaluating it later… or by compiling expressions to executable Forth code. ;-)

expression parser simply generates native Forth code, in the same way "normal" Forth compiler do. it inserts calls to "(get-label)" for all identifiers; and "(get-label)" THROWs the exception if label is not yet defined. so, i simply run the compiled expression code, and CATCH errors. if there are none, we can use evaluation result, and throw compiled code away. otherwise, we put pointer to the code into the list of unresolved expressions.

the engine re-evaluates the whole list when each new label is defined. it may seem slow, but remember that we have compiled code there, and that code consists mostly of simple math and label access. so it is as fast as keeping the list of referenced labels with each expression, and check that list first (because this is basically what evaluation does ;-), but without any extra complexity. win-win! ;-)

and by the way: yes, this is fully working infix-to-Forth expression compiler, independent of other asm parts (except the lexer). i.e. with small modifications it can be used outside of asm. not that i need that, it just… happened. ;-)

the source code is about 7KB now. your bog-standard table-driven recursive descent expression parser. ah, btw, it also does short-circuit boolean evaluation, of course.

in C, i'd be forced to either parse all expressions to AST, or to use some kind of stack VM a-la Forth, and compile expressions to bytecode. i.e. either more complex data structures, or built-in Forth-like language anyway. ;-)