Forth-eV Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== A library for Forth ======

*W.O.*



===== The idea =====

Add a mechanism to Forth that allows code sections/chapters to be loaded from a large 'file' with a simple command. How the file is stored and what it contains is implementation-dependent. What matters is the interface. The sections or chapters are accessible through one or more keywords.

The interface consists of the following word: ''%%NEED%%''

  * ''%%NEED%%''  ''%%( i*x "name" -- j*x )%%'' Make sure “name” is present in the dictionary. If not, load it from a library. If "name" already exists, do nothing.

An optional word as suggested by Ulrich Hoffmann
  * ''%%FROM%%'' ''%%( "name" -- i*x )%%'' Open the library "name" for use with ''%%NEED%%''

Optional words as suggested by Willem Ouwerkerk
  * ''%%NEEDED%%''  ''%%( i*x a b -- j*x )%%'' Make sure that name represented by the string a b is present in the dictionary. If not, load it from a library. If name already exists, do nothing.
  * ''%%RUN%%'' ''%%( i*x "name" -- j*x )%%'' Find "name" in the library and load & run the script from a library.
  * ''%%.LIB%%''  ''%%( -- )%%'' Show all section/chapter headers of the current library in one or more columns
  * ''%%VIEW%%'' ''%%( "name" -- )%%'' View the code from the libary, that belongs to "name".

===== Pseudo code =====

<code>
Function: NEEDED   ( a +n -- i*x )  \ a +n is keyword
    a) Check if the word represented by the string a +n exists in 
       the current search order.
    b) Do nothing the string was found, otherwise start searching 
       the active library for a keyword that matches this string.
    c) When the keyword is found load the source code chapter
       otherwise issue an error message
    
Function: NEED     ( "name" -- i*x )
    Perform the function of NEEDED using the parsed "name".
    
Function: .LIB     ( -- )
    Print all chapter headers in a human readable form.
    
Function: VIEW     ( "name" --)
    When the keyword "name" is found view the source code chapter,
    otherwise issue an error message
    
Function: FROM     ( "name" -- )
    Make the library "name" the current active library
    When "name" was not found issue an error message
</code>



==== Contributions ====

<html><h2 style="background-color:yellow">Implementations</h2></html>

===== A library for RP2040 Flash =====

Here, I will elaborate on an example that stores the source in Flash memory. For this implementation, there are the following design requirements:\\

    - Simple structure ( idea A.N. )
    - Quick search function
    - Extensible: OPEN-LIB CHAPTER CLOSE-LIB WIPE-LIB
    - Easy to use: NEED etc.

===== Structure =====

This library mechanism is based on two control characters ''%%09%%'' and ''%%0D%%''.
They act as separators and are not part of the text itself.

The ''%%09%%'' marks the end of a chapter, like this:\\

  * chapter-1 ''%%09%%'' chapter-2 ''%%09%%'' last-chapter ''%%09%%''

The ''%%0D%%'' marks the end of a line:\\
  * line-1 ''%%0D%%'' line-2 ''%%0D%%'' last-line ''%%0D%%''

The first line of each library chapter starts with a case insensitive keyword. The word ''%%CHAPTER%%'' builds this the first line, and provides this line with a
backslash to prevent the keywords for being executed while loading a chapter. Then it adds all source code until the ''<nowiki>%%</nowiki>'' marker is found. The end of each chapter is marked by ''%%09%%''
The end of the library is kept in a pointer that is updated when we extend the library.\\

<code forth>
chapter 2TUCK
\ Copy top double below second
: 2TUCK   2swap 2over ;     ( x1 x2 x3 x4 -- x3 x4 x1 x2 x3 x4 )
%%

chapter 2ROT
\ Rotate third double to top of stack
: 2ROT    2>r 2swap  2r> 2swap ;
%%

chapter -2ROT
\ Rotate top double to third double position on the stack
: -2ROT   2swap 2>r  2swap 2r> ;
%%
</code>

The above library chapters leave the following ASCII structure in Flash memory.\\
When after the <09> there is no more text, it's the end of the library.\\
<code>
\ 2TUCK<0D>
\ Copy top double below second<0D>
: 2TUCK   2swap 2over ;     ( x1 x2 x3 x4 -- x3 x4 x1 x2 x3 x4 )<0D>
<09>
\ 2ROT<0D>
\ Rotate third double to top of stack<0D>
: 2ROT    2>r 2swap  2r> 2swap ;<0D>
<09>
\ -2ROT<0D>
\ Rotate top double to third double position on the stack<0D>
: -2ROT   2swap 2>r  2swap 2r> ;<0D>
<09>
</code>

===== Pseudo code for the Flash library =====

<code>
Function: NEEDED   ( i*x a +n -- j*x )
    a) Save the string a +n and make the string uppercase
    b) If this string is already present in the dictionary, do nothing
    c) Start searching the library for this string
    d) When a keyword is found load the source code section until 09 is found
    e) When a keyword is not found issue an error message
    
Function: NEED     ( i*x "name" -- j*x )
    a) Parse the next word form the input stream and perform the function of NEEDED

Function: .LIB     ( -- )
    a) Scan the library for a 09
    b) When it's not the last, print the found library chapter/section header
</code>

===== Pseudo code for expanding the Flash library =====

<code>
Function: OPEN-LIB    ( -- )
    Open the library by reading the last incomplete file sector to the 
    sector buffer. Adapt the library write administration in such a way
    that new source code will be added behind whats read from Flash.
    When the library is empty, lay down the start of a section header 
    with a 0D & 09 character.

Function: CLOSE-LIB   ( -- )
    Close the library, writing the last added code (incomplete sector)
    Update the end of library pointer

Function: WIPE-LIB    ( -- )
    Erase a stored library completely, restore the end of library pointer

Function: CHAPTER     ( -- )
    Add a new library section, first add the section header.
    Note the given keywords in uppercase, sepatated by spaces and ended 
    with a 0D. Read and store the Forth source code behind it until the 
    end of source marker %% was found
</code>

===== noForth t library use =====

This sample is taken from the noForth t library mechanism. Here the library is stored in the remaining flash that is not needed by the noForth binaries. For readability some noForth specific words are replaced.

<code forth>
  create KEYWORD  20 allot  \ Hold keyword
  1008,1000 constant LIB    \ Library start address
  LIB value LIBHERE         \ Current end address of library

: LIB-FIND    ( a +n –- sa ) \ a,+n=name, sa=begin-source-section
    keyword place           \ Save keyword
    keyword count upper     \ Convert to uppercase
    libhere  lib            \ Library address range
    begin
        2dup > 0= throw 2dup \ Nothing found?
        0D scan nip over 1+ \ Get line with keywords, skip \
        begin
        2dup > while        \ Line not done?
            bl skip dup >r  \ Skip leading spaces
            bl scan         \ After keyword
            r@  over r> -   \ Keyword length
            keyword count s<> 0= \ Keyword found, leave source address
            if  2drop nip exit
            then
        repeat
        2drop  09 scan  1+  \ Find library section, skip 09
    again ;

: LIB-REFILL  ( source-id –- f ) 
    to ib  false            \ Start of new line
    ib c@ 09 = ?exit        \ End of source section, ready
    ib FF + ib  0D scan     \ Find 0D (end of current line)
    dup ib - to #ib         \ Calc. & save line length
    1+ to source-id  drop   \ Save start of next line (behind 0D)
    >in !  true  ;          \ Refill succeeded

: LIB-LOAD    ( a –- i*x ) 
    ?dup 0= ?exit           \ Do nothing when zero
    @input >r 2>r           \ Save current input source
    to source-id            \ Set new input source
    #ib >in !  interpret    \ Force a REFILL when loading source section
    2r> r> !input ;         \ ib #ib,>in@ source-id restore input source

: NEEDED      ( a +n –- i*x ) 
    ['] lib-refill  to 'refill   \ Add library REFILL
    LIB-FIND keyword find nip 0= \ in the current forth search order
    and lib-load ;

: NEED        ( a +n –- i*x )   bl word count  needed ;

: .LIB        ( –- )      \ Show library contents by printing keyword lines
    libhere  lib                 \ Lib. range
    begin  cr
        20 for
            1+ 1+  2dup > 0= if  \ Skip backslash, not past end of file?
                2drop rdrop exit \ Ready
            then
            dup >r  0D scan      \ Get end of keyword line
            r@ over r> - type    \ Show keyword line
            28 hor - 0 max spaces \ In fixed column size
            hor 28 > if cr then
            09 scan  1+          \ Find section marker
        next
    key bl <> until  2drop ;

</code>

===== noForth t extending the library =====

Note that: This sample code uses calls to the built-in ROM API functions.\\
It uses them for reading, writing and erasing flash memory.
<code forth>
10000000 constant XIP        \ Start of XIP memory
  
create BUFFER  180 allot     \ Sector buffer with overflow (noForth t)
0        value PTR           \ Buffer index
: LC,    buffer ptr + c!  incr ptr ;  ( b -- )    \ Compile a byte in the library buffer
: LM,    bounds ?do  i c@ lc,  loop ; ( a +n -- ) \ Compile the string a +n in the library buffer

: LIBWRITE      ( +n -- )   \ Write library sector
    libhere xip - buffer 100 write-flash \ Write lib. sector to flash
    dup +to libhere   negate +to ptr    \ To next lib. block & correct pointer
    buffer 100 FF fill                  \ Erase first buffer
    buffer 100 + buffer ptr move ;      \ Move overflow to sector buffer

: BUFFER-FULL   ( -- )      \ Buffer overflow, write & restore
    ptr FF > if  100 libwrite  then ;

: ADD-LIB       ( text -- ) \ Add library section, save it when a buffer is full
    begin   buffer-full                 \ Buffer overflow, write & restore
            refill drop                 \ Read new line
    ib 2 s" %%" s<> while               \ No (%%) delimiter?
         ib #ib lm, 0D lc,              \ Store line
    repeat  refill drop ;               \ Read next line

: (-BL)         ( -- )
    parea  bl skip nip  ib - >in ! ;    \ Skip leading spaces in IB

: CHAPTER       ( text -- )  \ Contruct new library section
    s" \ " lm,                      \ New lib. header
    begin
        (-bl) bl parse              \ Find keywords
    ?dup while
        2dup upper lm,  bl lc,      \ Store uppercase in buffer
    repeat  drop  0D lc,            \ Add formfeed when done
    add-lib  09 lc, ;

: OPEN-LIB      ( -- )  \ Open a lib. for writing, read incomplete lib. sector too!
    libhere lib <> if               \ Lib. not empty?
        libhere 100 /mod  100 *     \ Get previous sector & ptr length
        dup to libhere              \ Correct lib. pointer & calc. XIP address
        buffer 100 move  to ptr  {w \ Read uncomplete sector to buffer, open flash
    then  0 to ptr  {W ;            \ First lib. entry, open flash

: CLOSE-LIB     ( -- )      \ Close library, save unfinshed buffer too
    buffer-full  ptr libwrite  W} ; \ Write last sector if any & close flash

: WIPE-LIB      ( -- )          \ Remove previous stored library from flash
    lib xip -                   \ Convert to start sector address
    libhere lib -  FF000 and    \ Convert to erase sector length, max. 1 Mbyte
    1000 +  {W wipe-flash W}    \ Erase one sector extra
    lib to libhere ;            \ And reset library pointer
</code>

===== A viewer for the library =====

This viewer shows 16 or less lines at a time.
It stops at a <09> character, prints a divider line and waits for user input.
When the spacebar was hit, it goes on displaying the next chapter.
Any other key leaves the viewer.

<code forth>
( A library consists of 09 {tab}, 0D {cr} and ASCII chars )
( ranging from blank to ~ others chars are not allowed )
\ Show the library source code of a chapter
: .LINE     ( a1 -- a2 ch ) begin c@+ dup BL < 0= while emit repeat ;
: .DIVIDER  ( -- )          cr  10 0 do ." -- " loop  cr ;

: LIBTYPE   ( a -- )
    begin
        10 0 do                         \ Max. 16 lines at a time
            cr .line 09 =               \ End of source chapter?
            if  .divider leave  then    \ Show divider & ready
        loop
    dup libhere < while                 \ More library source to be done?
        key BL <> if drop exit then     \ Yes, ask key, stop on non space
    repeat  drop ;

: VIEW      ( "name" -- )       bl word count  lib-find libtype ;

</code>
===== A sample library source =====
This examples shows how many different code parts can be used inside the library.\\
  - Just a version message printing some strings in a script
  - Interpret some noForth commands as script adapting a pointer
  - The addition of some high level routines, the VIEW function
  - Again a script, but with numeric input from the stack, changing the noForth configuration. It includes a test of the changed part.

<code forth>
open-lib

chapter version versie vsn
cr .(    NEED version 0.32     )
cr .(   Library version 0.22   )
cr .( Size is about 174 kBytes )
%%

chapter RESTORE-LIB
v: inside   \ Restore LIBHERE in case it got lost
    lib hx 50000 +  lib   hx FF scan  nip to libhere
    cr .( Libhere ) libhere u.
    cr .( Size ) libhere lib - dm . .( bytes )
v: fresh
%%

chapter VIEW
( A library consists of 09 {tab}, 0D {cr} and ASCII chars )
( ranging from blank to ~ others chars are not allowed )
v: inside also  definitions \ Show the library source code of a chapter
: .LINE     ( a1 -- a2 ch ) begin c@+ dup bl < 0= while emit repeat ;
: .DIVIDER  ( -- )          cr  10 for ." {} " next  cr ;

: LIBTYPE   ( a -- )
    begin
        10 0 do                         \ Max. 16 lines at a time
            cr .line 09 =               \ End of source chapter?
            if  .divider leave  then    \ Show divider & ready
        loop
    dup libhere < while                 \ More library source to be done?
        key bl <> if drop exit then     \ Yes, ask key, stop on non space
    repeat  drop ;

v: extra definitions
: VIEW      ( ccc -- )          bl-word count  lib-find libtype ;
v: fresh
%%

chapter PIN  SQUERY
( GPIO -- ) \ Change GPIO pin for S?
need [IF]
dup dm 30 2 within [if]  drop  dm 24  [then] \ Invalid switch pin?
1 cfg 1+  c!        \ GPIO-xx for S?
4 cfg @ abs 4 cfg ! \ Make sure to (re)start the second image
config
cr .( Test S? ) s? .
%%

chapter 12MHZ
\ change clock frequency
dm 12       0 cfg ! \ Set frequency in MHz
4 cfg @ abs 4 cfg ! \ Make sure to (re)start the second image
config              \ Test new configuration
%%

close-lib
</code>

===== The library is just bare ASCII =====

When displayed the library chapters look like this:\\

<code forth>
\ VERSION VERSIE VSN
cr .(    NEED version 0.33     )
cr .(   Library version 0.22   )
cr .( Size is about 174 kBytes )

    \ RESTORE-LIB
v: inside   \ Restore LIBHERE in case it got lost
    lib hx 50000 +  lib   hx FF scan  nip to libhere
    cr .( Libhere ) libhere u.
    cr .( Size ) libhere lib - dm . .( bytes )
v: fresh

    \ VIEW
( A library consists of 09 {tab}, 0D {cr} and ASCII chars )
( ranging from blank to ~ others chars are not allowed )
v: inside also  definitions \ Show the library source code of a section
: .LINE     ( a1 -- a2 ch ) begin c@+ dup bl < 0= while emit repeat ;
: .DIVIDER  ( -- )          cr  10 for ." {} " next  cr ;

: LIBTYPE   ( a -- )
    begin
        10 0 do                         \ Max. 16 lines at a time
            cr .line 09 =               \ End of source section?
            if  .divider leave  then    \ Show divider & ready
        loop
    dup libhere < while                 \ More library source to be done?
        key bl <> if drop exit then     \ Yes, ask key, stop on non space
    repeat  drop ;

v: extra definitions
: VIEW      ( ccc -- )          bl-word count  lib-find libtype ;
v: fresh

    \ PIN SQUERY
( GPIO -- ) \ Change GPIO pin for S?
need [IF]
dup dm 30 2 within [if]  drop  dm 24  [then] \ Invalid switch pin?
1 cfg 1+  c!        \ GPIO-xx for S?
4 cfg @ abs 4 cfg ! \ Make sure to (re)start the second image
config
cr .( Test S? ) s? .

    \ 12MHZ
\ change clock frequency
dm 12       0 cfg ! \ Set frequency in MHz
4 cfg @ abs 4 cfg ! \ Make sure to (re)start the second image
config              \ Test new configuration
</code>

==== A little extra ====

Loading a lot of library code in one go. It must be used like this: ''%%need( -rot -2rot d- asm\ )%%''\\
Note that ''%%BL-WORD%%'' is a typical noForth word that uses ''%%REFILL%%'' inside, so it can load multiple lines of library words when you need too.

<code forth>
v: inside
: NEED(     ( keyw-0 .. keyw-n -- )
    begin  bl-word count    \ Read next keyword
           2dup s" )" s<>   \ Not the closing paren?
    while  needed  repeat   \ Ok, perform NEEDED on the keyword
    2drop ;                 \ Ready
v: fresh
</code>

==== More ====
<html><h2 style="background-color:yellow">Other implementations</h2></html>