#!../src/tops -i -s ../sys  -u ../usr

/*----------------------------------------------------------------------
 
A list of I/O tasks posed by Al Danial:

1. Write an ASCII integer, floating point number, and a string to
   the file 'x.dat', one number per line.  Use the
   format "% 22.16e" for the float.

2. Same as #1, except in binary, and no newline between entries.

3. Write an R x C matrix to the text file 'x.dat'.  There should
   be no limit to the size of the matrix (as you get with .m when
   the matrix is too big).  The numbers should be written with
   the format "% 22.16e".  All terms in a row must appear on the
   same line.  For example, saving '2 3 random "% 22.16" mformatset'
   would result in a file with these two lines:
 3.3284172896893777e-01  2.6809120563235656e-01  6.5710120865008853e-02
 7.0938780936849660e-02  8.0889306301665165e-01  3.9000137820374287e-01

4. Same as #3, except in binary, and no newlines.

5. Read the matrix stored with #3 from the file 'x.dat'.

6. Read the matrix stored with #4 from the file 'x.dat'.

7. Save the top three stack items--regardless of type--to the binary
   file 'x.dat'

8. Load the file of #7 and put the three items back on the stack.

------------------------------------------------------------------------

Examples for doing these I/O tasks are shown below by Dale Williamson.

Here are some notes about these examples:

   Word save is a simple word to save a text VOL to a file, no file
   handle required.

   File handles give control needed when doing byte-by-byte operations
   on files.

   Handy words to use just before running word "file" or word "open" to
   make a file handle are:

      closeif to close file handle in case it is still open from a 
      prior use

      deleteif to delete file by name (files cannot be deleted using 
      their handles)

   Given a file handle, word fprops shows its properties and its current
   pointer offset (as does word fpos); if the handle is not valid, it
   will tell you that too.

   Word xray is a great help for seeing exactly what is on a file or in
   an item on the stack; it was one of the first capabilities in tops 
   and has been essential in debugging its code.

   Word xray can be used to show hex bytes of files (given the handle) 
   as well as items on the stack.

   Xrays of files start at the current file pointer, so you can xray
   small pieces of very large files.

   INF is handy with words xray and fget, when exact length is not 
   important and you just want it all; fget, for example, never gets 
   more than the number of bytes remaining from file pointer to end 
   of file; xray works similarly.

   VOLs do not have newlines at the end of each line; in fact lines are
   padded with blanks to the width of the widest line.  Word textput
   is used to pack a VOL down into a STR with newlines between lines
   and no blank padding.  Word textput is used in Examples 3 and 7 for
   VOLs written to file, and its inverse, word textget, is used in 
   Example 8 (inside new word convert) to get a VOL back.

   VOLs are really general arrays of bytes, unlike MATs that are arrays
   of IEEE floating point numbers of a very specific form.  When raw
   bytes are read from files with word fget, they go to the stack in a 
   one-row VOL, as in Examples 5 and 6.  

   In Example 5, the VOL from fget contains text that represents num-
   bers.  The text must be converted to IEEE number patterns before 
   any math can be done.  Word numerate is used for this, converting
   the one-row VOL from fget into a one-column MAT.  Word matrix then
   redefines the one-column array to have the correct number of rows
   and columns. 

   Word matrix also rearranges the order of the terms so they are stored
   in the MAT by columns since the original text definition had the num-
   bers strung together by rows (as the xrays demonstrate).  The terms 
   in MATs must be stored by columns for proper matrix operations.

   In Example 6, the bytes from fget are known to already be IEEE num-
   bers.  So word vol2mat is used to simply convert the one-row VOL from
   fget into a one-column MAT without making any changes to the bytes.
   Then, knowing the number of rows the MAT is supposed to have, word 
   fold defines the rows and columns of it.  Word fold is used here, 
   instead of word matrix used in Example 5, because the values are al-
   ready stored by columns and do not need to be rearranged.

   The vol2mat and fold operations in Example 6 do nothing to the raw 
   bytes read from the file.  They simply dress up the stack item that 
   contains the bytes, making it into a MAT so the program can perform 
   math with it.  

   On the other hand, words numerate and matrix needed in Example 5 do 
   a lot to the original bytes.  First they are converted from text to 
   IEEE numbers, and then they are moved around so the resulting MAT 
   contains numbers stored by columns.

   To do Example 7 just with raw bytes on a file, a mapping must be 
   saved to reconstruct things (the mapping could be written to the 
   file too, which is sort of what words put and get do).  I made some 
   arrays called TYPES, ROWS, START, and BYTES to do this.  Then to do 
   Example 8, I wrote a word called "convert" that automates the con-
   version of file bytes back to stack items using the mapping arrays.

   Example 7 writes items already on the stack, so top of stack (last
   on stack) is first on file and bottom of stack is last on file.  The
   loop in Example 8 is run backwards so last on file is first on stack
   and things come back on the stack in their original order (another 
   way would be to use word revn, which reverses the order of the n top-
   most stack items).

   Examples 7 and 8 are redone in 7A and 8A using higher level words
   put and get.  Words put and get only deal with MATs and VOLs, and
   NUMs need to be converted to 1-by-1 MATs and STRs into 1-row VOLs.
   Word hand (convert to handle) is used to do this.

   But getting the NUM and STR items back in 8A from their MATs and VOLs
   involved some trickery.  I used the put name to bury the stack item 
   type between < > like html, and extracted the type in the get loop.

   Also in 8A, a toc was written to the stack instead of to the display
   using >stk, and its number of rows gave count, ROWS, for the loop to
   extract the file items.  And a loop to roll all the stack items to
   the order of 7A is needed (word revn could have been used instead).

   There are other ways to do these examples, including vectoring output
   to a file (word set_sysout) or vectoring output to a VOL on the stack
   (word >stk, as used in 8A).  They would be good for future examples.

   [Note: recently get and put have been revised to handle NUM and STR,
   so examples 7 and 8 and 7A and 8A could be greatly simplified.  But
   in their present form they are useful in demonstrating other aspects
   of I/O.]

----------------------------------------------------------------------*/

   nl();
   dot("Example 1");
   nl();

   nl(dot(
   "Write ASCII integer, floating point number, and a string to a file."
   ));

   T = [intstr(123456)            ; // IEEE number into int string
        "123456"                  ; // string
        format(10*pi, "% 22.16e") ; // IEEE number into formatted string
        "This is a string"];        // string

   save(T,"/tmp/x.dat"); // VOL T saved to text file /tmp/x.dat
   
   nl(dot("Load text file and display it:"));
   nl(dot(asciiload("/tmp/x.dat")));

//----------------------------------------------------------------------

   nl(dot(nl("Example 2")));

   nl(dot("Same as #1, except in binary."));

// Making a file handle:
   deleteif((FILE="/tmp/x.dat")); // delete old file if there is one

   file( FILE,       // this is name of file on system
      new,binary,    // properties
      "BIN",         // this will be the file handle name here
      closeif("BIN") // close handle BIN if I forgot to previously
   );
   fput(123456,BIN);             // IEEE number to file
   fput("123456",BIN);           // string to file 
   fput(10*pi,BIN);              // IEEE number to file
   fput("This is a string",BIN); // string to file

   nl(fprops(BIN)); // info about BIN; note current pointer
   rewind(BIN); 
   nl(fprops(BIN)); // rewind has put current pointer to 0

   nl(dot("Xray of bytes on binary file:"));

   xray(BIN,INF) // no semicolon keeps xray text on stack for dot().
   nl(dot());

   close(BIN);

//----------------------------------------------------------------------

   nl(dot(nl,"Example 3"));

   nl(dot("Write an R x C matrix to the text file /tmp/x.dat."));

// This eliminates "matrix too big" messages from .m, which uses 
// word mtext for formatting (man .m, and man mtext mention this):
   mtext.ColsMax = INF; // ColsMax in library of mtext set to INF
   mtext.RowsMax = INF;

// Make a file handle as in #2, reusing names FILE and BIN:
   deleteif(FILE); 
   closeif("BIN");
   file(FILE,new,binary,"BIN"); // making handle to FILE called BIN

   A = random(4,3)-0.5;
   rows3 = rows(A); // rows for #5

   nl(
      dot(
         [  "Here are the matrix bytes converted to text form:" ; 
   
         // All cols of A formatted into text VOL T:
            (T = format(A, cats(" % 22.16e",cols(A)))) 
         ]
      )
   );

/* This puts nl at the end of each line of T with no trailing blanks 
   (save() in Example 1 did this automatically when it wrote to file), 
   and converts T to a string: */
   S = textput(T);

// Here fwrite is used instead of fput, to get back pos and len info
// when file is written:
   (pos, len3) = fwrite(S, BIN);

   nl(dot(nl,"Xray of text file; shows 0A newlines at end of rows:"));
// You can see 0A due to textput() after every 3rd number:
   nl(dot(xray(rewind(BIN), BIN, INF))); 

   deleteif("/tmp/x.dat");
/*
WORK IN PROGRESS
\-----------------------------------------------------------------------

   "Example 4" nl . nl
   "Same as #3, except in binary." . nl

 \ After xray at the end of Example 3, the file ptr is at end of file. 
 \ Without rewinding, write a binary matrix after the text of #3:

   2 3 random (hA) 
   "Here is the matrix:" . nl 
   (hA) dup " % 22.16e" those cols cats format (hT) . nl

   "Xray of matrix on stack about to be appended to binary file:" . nl
   (hA) dup INF xray . nl \ show xray of bytes on stack

   (hA) its rows "rows4" book \ need to know rows for Example 6
   (hA) BIN fput \ append bytes to BIN

\-----------------------------------------------------------------------

   "Example 5" nl . nl
   "Read the text stored in #3 from the file /tmp/x.dat." . nl

   BIN rewind BIN len3 fget (hT) \ fetching len3 bytes of Example 3

   "This xray of fetched stack item matches xray of Example 3 file:" 
   . nl
   (hT) dup INF xray . nl \ xray matches Example 3 text

   "And here is the text converted to a matrix for doing math:" . nl
   (hT) numerate rows3 matrix (hA) \ rows3 saved from Example 3
   (hA) " % 22.16e" those cols cats format (hT) . nl

\-----------------------------------------------------------------------

   "Example 6" nl . nl
   "Read the binary matrix stored in #4 from the file /tmp/x.dat." . nl

   BIN len3 fseek    \ seek past Example 3 bytes
   BIN INF fget (hT) \ fetch bytes of Example 4

   "This xray of fetched stack item matches Example 4 xray:" . nl
   (hT) dup INF xray . nl \ xray matches Example 4

   "And here are the chained bytes folded back into the matrix:" . nl 
   (hT) vol2mat rows4 fold (hA) \ have MAT A, ready to do math
   " % 22.16e" those cols cats format (hT) . nl

\-----------------------------------------------------------------------

   "Example 7" nl . nl
   "Save the top three [four] stack items--"
   "regardless of type--to binary file." cat
   72 .out nl 

 \ Putting four items on the stack, one of each stack item type:
   depth push

      "a string"                    (qS) \ STR
      98765.4321                    (n)  \ NUM
      "Top line" "Bottom line" pile (hT) \ VOL
      3 4 random                    (hA) \ MAT

      yes nl .sf                         \ display stack

   depth pull less "count" book

 \ Not using fancier words put and get, we need to keep track of things 
 \ ourselves to get their bytes back from a file:
   count 1 null "TYPES" book
   count 1 null "ROWS" book
   count 1 null "START" book
   count 1 null "BYTES" book

   BIN rewind

 \ Writing to BIN and keeping track of data needed for fetching:
   count 1st
   DO (hA) dup type TYPES I poke
      (hA) dup rows ROWS  I poke
      (hA) dup type VOL = IF textput THEN \ put nl at end of VOL bytes
      (hA) BIN fwrite (pos len) \ same as fput, but receive pos, len
      (len) BYTES I poke
      (pos) START I poke
   LOOP

\-----------------------------------------------------------------------

   "Example 8" nl . nl
   "Load the file of #7 and put the items back on the stack." . nl

 \ This new word will convert raw bytes from binary file, and is used
 \ in the loop below:
   inline: convert (hT rows type --- hA) \ convert VOL T to type
      "type" book "rows" book
      type NUM = IF (hT) vol2mat 1st pry return THEN
      type MAT = IF (hT) vol2mat rows fold return THEN
      type STR = IF (hT) 1st quote return THEN
      type VOL = IF (hT) textget notrailing return THEN
      " convert: " type int$ cat " is not a valid type" cat ersys
   end

   BIN rewind

 \ Note that loop runs backwards, fetching from last, so items are in
 \ the original stack order (another way would be to reverse the rows
 \ in START, BYTES, and ROWS (word reversed), and use a forward loop):

   1st count \ these are reversed to do backward loop
   DO BIN START I pry fseek
      BIN BYTES I pry fget (hT)
      ROWS I pry TYPES I pry 
      (hT rows type) convert (hA)
      -1 \ negative loop increment
   +LOOP \ word +LOOP instead of LOOP

   "These items now on the stack agree with Example 7:" . nl yes .sf

\-----------------------------------------------------------------------

   "Example 7A" nl . nl
   "Save the top four stack items--regardless of type--to binary file "
   "using words put and get." cat
   72 .out nl

   "/tmp/x.dat" dup deleteif "BIN" dup closeif new binary file
   BIN rewind

   xx \ clear the stack

 \ Putting four items on the stack:
   depth push

      "a string"                    (qS) \ STR, first on stack
      98765.4321                    (n)  \ NUM
      "Top line" "Bottom line" pile (hT) \ VOL
      3 4 random                    (hA) \ MAT, top of stack

      yes .sf nl                         \ display stack

 \ Words put and get only work with MAT and VOL.  As each stack item
 \ is written, word hand is used to convert NUM to 1-by-1 MAT and STR 
 \ to 1-row VOL:
   depth pull less 1st (N 1st)
   DO (hA) this type push \ capture type on local stack
      (hA) hand           \ convert NUM to MAT and STR to VOL
      (hA) this named "<" pull int$ ">" cat cat cat naming 
      (hA) BIN put 
   LOOP

   "Contents of file /tmp/x.dat (file handle BIN) written by put:" . nl
   BIN "toc" >stk neat . nl

\-----------------------------------------------------------------------

   "Example 8A" nl . nl
   "Load the file of #7A and get the four items back on the stack." 
   . nl

   BIN "toc" >stk (hT) rows "ROWS" book \ number of items from toc rows

   ROWS 1st
   DO BIN "" get (hA)
      (hA) this named "<" ">" between number (n f) not
      IF no THEN "TYPE" book

      TYPE STR = IF (hA) 1st quote THEN
      TYPE NUM = IF (hA) 1st pry   THEN
      (hA)

   LOOP 

   ROWS nit 1st DO I roll LOOP \ reversing ROWS items on stack (could
                               \ have used 'ROWS revn' instead)

   "Items are back on stack as in #7A:" . nl

   yes .sf nl

\-----------------------------------------------------------------------

   "/tmp/x.dat" deleteif
   halt
*/
