Implementing a Forth

Problem with Forth and Lisp is that you can freely start to improve language, so it sort of destroys the usefulness in a company of full of Salarymen and Office Droids.

Here is Grok's scheme to make Forth more readable:

  Below is a Forth implementation of a simple parser that transforms
  (. (+ 1 2)) into 1 2 + . and executes it. This assumes the input is
  provided as a string or entered interactively, and the parser outputs
  the transformed Forth code to the input stream for execution.
  
  
  \ Buffer to store transformed output
  20 CONSTANT BUF-SIZE
  CREATE BUF BUF-SIZE ALLOT
  VARIABLE BUF-PTR
  
  : INIT-BUF ( -- )
    BUF BUF-SIZE ERASE
    0 BUF-PTR ! ;
  
  : ADD-TO-BUF ( c -- )
    BUF-PTR @ BUF-SIZE < IF
      BUF BUF-PTR @ + C!
      1 BUF-PTR +!
    ELSE
      ." Buffer overflow" ABORT
    THEN ;
  
  : PARSE-EXPR ( addr u -- )
    INIT-BUF
    BEGIN
      DUP 0> WHILE
      OVER C@ CASE
        '(' OF DROP 1 /STRING ENDCASE  \ Skip (
        ')' OF DROP 1 /STRING ENDCASE  \ Skip )
        '.' OF BL ADD-TO-BUF '.' ADD-TO-BUF BL ADD-TO-BUF
               DROP 1 /STRING ENDCASE
        '+' OF BL ADD-TO-BUF '+' ADD-TO-BUF BL ADD-TO-BUF
               DROP 1 /STRING ENDCASE
        BL OF BL ADD-TO-BUF DROP 1 /STRING ENDCASE  \ Handle spaces
        ELSE
          DUP ADD-TO-BUF  \ Copy number characters
          DROP 1 /STRING
        ENDCASE
      REPEAT
    2DROP ;
  
  : EVAL-EXPR ( addr u -- )
    PARSE-EXPR
    BUF BUF-PTR @ EVALUATE ;  \ Execute the transformed string
  
  \ Example usage
  : TEST ( -- )
    S" (. (+ 1 2))" EVAL-EXPR ;
  
  TEST

I adore the `Threaded Interpretive Languages` book! It very clearly describes the internals of a real language, and variations on how to get it all done.

https://archive.org/details/R.G.LoeligerThreadedInterpretive...

I am under impression that more people implement Forth than use it for programming...

I went a completely different route when implementing zeptoforth (https://github.com/tabemann/zeptoforth) -- I went right for implementing a fully-featured system rather than focusing on minimalism, and came out with something that I am extremely comfortable writing non-trivial code with rather than a toy that shows how small of an implementation I can make without regard to being a practical tool. And yes, zeptoforth is very big as Forths (especially microcontroller Forths) go -- because the goal is to make something that can be used to program real systems out of the box with minimal effort on the user's part.

It's a bit of a shame that Forth has become almost synonymous with indirect threading implemented in assembly. This is true and somewhat interesting of most implementations, but obfuscates the deeper philosophy of Forth. For me, this is about self-reliance and building your own tools that are suited for the specific task at hand.

I don't care for a lot of the standard forth-isms, such as the way branching and recursion are handled. But I make extensive use of the threaded execution model. One quickly notices that anything that can be pipelined naturally fits into this model. Programming in forth style is often similar to programming with unix pipes except you aren't restricted to string input and output and you do not need to fork subprocesses (though you certainly can!).

Learning how to program in forth, which as so many have pointed out is best done by implementing your own, will fundamentally change the way you program in all languages. Chaining method calls in data pipelining is somewhat forth-like.

As far as implementing a forth, the main interpreter can be written in 14 lines of lua (using lua varargs as the parameter stack):

  local thread thread = function(first, next, ...)
    if first then
      if next then
        local cont = thread(next, ...)
        return function(...)
          return cont(first(...))
        end
      else
        return first
      end
    else
      error "require at least one task"
    end
  end

  local push = function(value) 
    return function(...)
      return value, ...
    end
  end
  local print = function(msg, ...)
    print(msg)
    return ...
  end
  local hello_world = thread(push "Hello, world!", print)
  hello_world()

  local dup = function(top, ...) return top, top, ... end

I have made a living with Forth since 1981. I have written very few new Forth kernels, but have ported them to many CPU architectures. Forth is no longer fashionable, but it works.

Writing a new Forth may be an interesting project, but you will not write a good one until you have a few applications under your belt. Forth is a very subtle language. The internet is full of abandoned Forth kernel projects. Many of these may well have succeeded if they had a purpose that was not yet satisfied.

It is perfectly possible to write large applications in Forth. One of our clients has an application of 1.4 million lines of Forth source code. It is hosted on VFX Forth, which compiles Forth to native code. The VFX version runs at least ten times faster than the previous threaded code version, built on MPE's ProForth compiler.

I went down the "make your own Forth" rabbit hole about 45 years ago.

In January 1979, Byte Magazine's Language Forum contained the article, "IPS, An Unorthodox High Level Language."[1] The article described IPS, a language based on Forth, but with the word names translated to German. Thus, Forth's SWAP became VERT, short for vertauschen. The intriguing article concluded with a reference to Charles Moore's 1974 paper, "FORTH, a New Way to Program a Minicomputer,"[2] which was discussed on HN in 2022.[3]

I had recently assembled a Quest Electronics Super Elf computer[4] with an expanded memory size of 4 KB. The IPS article mentioned implementations under 6 KB for 8080, 6502, and RCA COSMAC microprocessors, so I thought Forth might fit. The Super Elf included an 1861 video display controller chip, with a resolution of 64 x 128 pixels (big pixels!). I designed a font of 3 x 5 pixel characters to provide 21 lines of 16 characters. Good luck distinguishing M, N, H, U, and W without some context. I bought a (possibly surplus) keyboard from Radio Shack and screwed it onto a wedge of wood to achieve a usable typing angle.

Moore's paper described about 75 Forth words. I wrote them on index cards and jotted down Forth definitions or RCA 1802 assembly code. I wrote an 1802 assembler in Fortran to ease the conversion into 1802 machine code. I still have the printouts and punch cards in storage.

Development proceeded slowly. During a break at work, I would punch my assembly code for a few Forth words, run the assembler, and bring home the printout. That night I would load previous work into the Super Elf from cassette tape, key in the new words (and changes) via the Super Elf's hex keypad, and save back to another cassette tape. Then I would test the new Forth words and note any changes needed on the printout. Lather, rinse, and repeat until, at last, it all worked.

The finished Forth system consumed a little more than 3 KB of the 4 KB of memory. User programs, data, and Forth stacks occupied the remaining memory. The R key stuck. The @ symbol was a 3 x 5 pixel blob (this is the very important Forth memory fetch operator). But it worked!

I demonstrated the system at the local personal computer club--spun off from the local ham radio club. Based on their enthusiasm, I advanced to developing tinyForth[5] for the TRS-80, a story for another day.

[1] https://archive.org/details/BYTE-MAGAZINE-COMPLETE/197901_By...

[2] https://articles.adsabs.harvard.edu/pdf/1974A%2526AS...15..4...

[3] https://news.ycombinator.com/item?id=33134663

[4] https://www.oldcomputermuseum.com/super_elf.html

[5] https://archive.org/details/80-microcomputing-magazine-1980-...

Not sure if this counts as an implementation, but this is my Forth compiler, written in the high-level language Go [0]. This is also a challenge, as Forth usually requires machine language, call stack access, and is naturally written in assembler. Took several tries until I found working data structures.

The funniest thing is, you can easily write and invent your own control structures in Forth, as well as change the behavior existing ones such as the do/while loop.

    \ alter the behavior of do by printing the current index in each loop.
    : do immediate
        [COMPILE] do   \ first compile previous do impl.
        ' i ,          \ then print current index
        ' . ,
    ;

I am currently implementing a stack based language as an intermediate language for a C compiler that I am developing. I have developed a relatively straight forward compiler for the stack language to the M1 assembler from live-bootstrap. This stack language has local variables. It actually makes use of two stacks: One in the traditional sense and one for the local variables (and the return address for calls). It reads as a kind of very primitive post-fix C kind of language. See: https://github.com/FransFaase/MES-replacement and https://www.iwriteiam.nl/D2506.html#2b

I recently implemented a Forth, compatible with the Forth Haiku dialect used by https://forthsalon.appspot.com/ -- it uses a tail call/continuation-passing dispatching method, and performs some rudimentary optimizations. At some point it also spit out some C but I decided to give this feature the axe until it has a better infrastructure for optimizations and codegen.

The idea is to use it to drive an LED matrix and have a simple web UI to develop "fragment shaders" in Forth. It's developed as part of the Lwan project, although currently it generates GIF files on the fly rather than drive a LED matrix.

The source code is here for those that want to play with it: https://github.com/lpereira/lwan/tree/master/src/samples/for...

I've messed around with Forth before, and even created my own one. I could do cool things with it, and gotten a DSL-like language.

I used words like IMMEDIATE and POSTPONE to create words that create words. I quickly get lost doing that, though.

Forth is very very cool to play with. Pragmatically, virtually any other language is better.

My first interpreters were Forth-derivatives, it's a great way to start designing your own languages.

I've recently been working on a project to lower the barrier to entry for budding language designers. The language is strictly prefix, but could easily be turned into postfix with minor changes.

https://github.com/codr7/shi

Could you use the Crafting Interpreters book as a guide to build a Forth interpreter?

https://craftinginterpreters.com/

Is writing a Forth in a high-level language feasible/useful as a learning exercise? Almost all the Forth discussion I’ve seen has been about implementing in assembly.

One of the best articles on tiny Forth interpreters that I've seen on HN for a long time, with many excellent links for further study!

[dead]