
-  \:/ messes up the line count in error reporting.
   (need to attach extra structure to files).

!  envs usually want arguments as key-value pairs, so
   make some default syntax for that (rather than the apply trick).
-  set __env__ key with each environment ?
   \push{dollar}{foobar} syntax (sets __env__ to foobar).

-  consider
   \set{zut}{<>}
   \<foo \zut>{<><>}

   might one e.g. want foo="zut &lt" type syntax?
   so that really only < and > must be output in device scope?

!  redo the special design
 #    I  suppose I want to temporarily map ' ' to \~.
         (in conjunction with " to "" mapping ..)
 #   II  URL encoding.
   the special level stuff isn't all that great.
   some local \@ syntax to push and pop or even just select definitions might
   be a lot better:
   what would it take? indexed special arrays, enable switching ?
-  specials and \N etc don't mix for special==\n
   what's the design/policy/contract?

-  replace tabs by spaces ??? make \"tab" or \@{\t} or \'0xy' or whatever?

#/ syntax for <br>; now
   \<br!> results in <br>

!/ clean up zmm packages, esp doc.zmm -> repl \@{<>}FOO{\@</>} by \<>{FOO}
   syntax. div.flushright ???

-  further tweak system inspection, e.g. looking at rm cp ln mv dd ?
   but also need to look at sh tcsh bash esh ....... and on and on ???

-  how about --compat flag: allow system primitives to be overwritten.
   (but primitives are looked up first ..)

#  zinsert *does* strip comments.

-  how about a preprocessing directive that removes the preceding newline?
   (would make verbatim content look prettier).
\verbatim{\:
   foo
   bar
   zut
\:{<}}

-  why not make inline file syntax: \:{foo} ?
   (there are several reasons to think of)

-  make dictionaries named: (e.g. itemize).
   enable a way of setting a macro in a named dictionary.
   this would solve the \set{$marginleft}{foo} problem.
   so .. that would be
   \pop#2 (name as well)
   \push#2 (name as well)

   and begin/end/env could set the session key
   \__env__
   \set{env=$itemize:foo#1}{zut \1}

-  what happens with e.g. \system{ date } ?
   (it fails)

-  -safe date [allow date command]

-  rewrite macro packages such that only and all non-exposed primitives 
   share the same prefix. exposed primitives should then be simply
   greppable.

-  Unicode support. Perhaps this will require a zoem space warp;
   if so zoem will bring forth zoef.
   byte-orientation is one aspect.

-  better names for --unsafe-silent, --system-honor, --unsafe.
-  should system-honor be the default, and system-relax be the option?
-  perhaps make key from which the system settings can be derived. 
-  system prompting: stdin is not shown.

-  yamExpandKey present in apply#2 and inspect#4

-  key definitions can be preparsed -> more bloated data structure,
   but faster interpolation -> needed length can also be computed
   in advance.

-  in the end I may want to switch to full paragraph enclosure,
   and more direct control on the margin. It is unsatisfactory that
   every \item implicitly has paragraph/text mode following it.

-  can I do 3 below ?
   perhaps introduce 'BOL' special

    The copy mode behavior can be quite tricky, but there  are  some  rules
    that ensure a safe usage.

    1.     Printable  backslashes  must  be denoted as \e.  To be more pre-
           cise, \e represents the current  escape  character.   To  get  a
           backslash glyph, use \(rs or \[rs].

    2.     Double all backslashes.

    3.     Begin  all text lines with the special non-spacing character \&.

    This does not produce the most efficient code, but it should work as  a
    first  measure.   For  better  strategies,  see the groff info file and
    groff_tmac(5).

-  -i foo (!-o) results in foo.ozm
   -i -  (!-o) results in stdout
[gershwin hobo zoem/src > devzoem
=== Interactive session, I should recover from errors.
=== If I exit unexpectedly, consider sending a bug report.
=== A single dot on a line of its own triggers interpretation.
\__fnin__ \__fnout__ \__fnentry__ \__fnbase__
.
----------------------------------------
stdia - - -

!  let decimals handling is crude.

-  what would it take to enable 'break' in while ? a *lot* ...

*  interactive mode uses device filter; it does not map \|, \~, \- .
   if one uses txt filter, whitespace is not munged.
   so that's a bit annoying; it seems not to be a nice solution to
   initialize mapping of zoem glyphs for device filter; it would
   currently need explicit bookkeeping whether user is doing first time
   call or later time call (in order to wipe them out).

!  when mcxIO supports buffering, readfile should use mcxIOstep and
   buffered input.

-  inline files may cause premature exit (no recovery).

?  default turn off white space munging ??

!  precision_g:
   should be taken relative to absolute value!
   (now 0.00000009 == 0.00000001)

*  The advantage of \&{ .. } and \@{ .. } and \%{ .. }{ .. } syntax over
   \`snt` like syntax is that the former is better
   extendible ...  i.e. you can later give meaning to \&&{ .. } or \&foo{ .. }
   etc etc.
#  (fixed \=fname= to \={fname}).

#  \inspect{posix}{(([a-z])*)}{_#2{\!ucase{\1}}}{abcde2347#)$*&xyz}
   does not work; must be
   \inspect{posix}{([a-z])*}{_#1{\!ucase{\1}}}{abcde2347#)$*&xyz}

#  for statement is ugly in zoem:
   \for{\set{i}{1}}{
      \cmp{lt}{\i}{10}}{
      \setx{i}{\let{\i+1}}}{
      do stuff 
   }

#  margins not ok with latest groff.
-  is there a way to get pagewidth in troff (so I can make the
   solution to the above problem less crude than .ll -2m)?

#  current zoem troff output not ok with apropos 

?  how about  a primitive to make a subtree of the global tree
   the new rooted tree ??

-  make grape dictionary ...

!  more primitives could act like '%': being able to take variable
   number of arguments.
   it would be an additional property of a primitive.

   BUT the parser has to know this; for '%' it knows by virtue of
   the first character.

   So, such primitives would e.g. have to share some prefix. like '%'.
   \%system{a}{b}{c}
   it's not an interesting avenue methinks

*  -  zoem does not have 'real' array syntax.
   -  "data" in zoem must obey zoem syntax.

-  NOTE parsing stuffs everything in arg1_g;
   %#1 then *again* extract arguments.
   this is so that the macro location mechanism need not be adapted.
   (we can simply search for %#1)

-  zoem GOTCHAS
   try to remove helper files (e.g. fqf-html, fqf-roff),
   if errors keep occurring;

   it is possible that a helper file contains erronenous or obsolete input; if
   it is newly created the error will go away. As long as the helper file
   contains erroneous input, the macro package in charge will never get to
   remove/renew it, since zoem will exit when encountering an error.

   look at log messages "wrote to stream".

   enable appending?
   explicit closing?
   sth like
   \fdo{my.out}{open}
   \fdo{my.out}{append} open for appending
   \fdo{my.out}{close}
   \fdo{my.out}{flush}

-  xml sugar:
   optify white space rules

-  xml sugar:
   make '<' and '>' yamSpecial tokens (259 and 260), so that the
   functionality can be supported for other formats too.
   this requires some additional rules which are not yet clear.

/  removed \; feature. how about making protect smarter?
   the feature was meant for creating the effect of \protect{ ...  \: comment }
   (which is in that form impossible as the comment is stripped at read time).


-  make mode where inline documentation is parsed. doxygen-like, e.g.
   /* zoem::doc
    foo bar *\,/ blah
    zut.
   */
   should be separate package .. code.zmm or sth.

-  interactive:
   \| and \~ etc are ignored ??

-  mailing list.

-  clarify inspect interface, make test suite, retry to refactor code.
   reinvestigate GNU posix regex long string/match bug.

-  document doc.zmm, or point out why it is not.

-  further inter and intra link *_zmm.azm files

-  can I make zoem crash by writing to __ type keys,
   or undeffing them? e.g. via unchecked key-pick-ups,
   or via assumed pointer identity, or ..
   the file.c fntxt_g variable does not feel right.

!  how does nested \zinsert{stdia} work?
   it actually seems to work, but what is happening with fp's meanwhile?

!  how grave is this:
   pushing a $-dict means the increment of $itemcount is screwed,
   (because set will create a new one).
   so changing styles by pushing is very much not to be recommended.

/, clean up the in/out/stdia code,
   clean up entry.c
   some stuff done; entry.c remains long listing of things
   that need to be done in order.
-  should separate yamEntry into yamInit and yamEntry.
-  make clean functions in certain modules. For example, the entry function may
   want to let the key module complain when not all opened scopes have been
   closed.

?  should exit really be premature?

-  explain tracing encoding.

-  equip yamfilterat with extra argument, so that tracing can
   show more information.
   note that \@{asldf fkdalk} fragments are currently not shown by tracing.

!  try to make handle for yes/no tracing file read.
   it can be interesting to see comments stripped etc.

-  get rid of the yam prefix.

?  valgrind says:
   ***** failing system call:
   ***** \system{ls}{{-#}}
   ls: invalid option -- #
   Try `ls --help' for more information.
   ==22922== Warning: invalid file descriptor 1 in syscall write()

   but it is difficult to trace this to zoem code.

!  document optkvp, reqkvp etc in manual page. other additions?
!  how about a 'dump' primitive;
      dump keys according to conditions.

#! let sprintf liability.

?? customize escape character, for both scopes.

?  split \W, \w for both both scopes.
   hum ho, best to think of device target as a single scope, whitespacewise.

-  make html.zmm file, with html specific stuff only.
   (specials).

!! much duplication between kvp and opt macros: '\%s%' vs '='
!  ellipsis in html, troff.

!  should separate zoem macro package from zoem source package.

#/ find out what the original line length is (for terminal).
   simply store .ll in register ?
   not simple -> look at andoc.mac or one of its siblings,
   pbb done via command line arguments and terminal info.
   uh, it's done via prepending to file stream I believe.

?  'skip empty newlines' (zoem.azm)

-  fork, close, fileno in yamSystem; understand the details and get them right.
   opportunity to debug fileno/dup/close/fork knowledge.

-  check c gotchas wrt isspace, signed/unsigned char. etc.

-  make \'010' ascii byte codes.
   should be converted during file input. do not allow on the fly
   creation of these ?

   \'010' \'0a' \'011100011'
   \'ff'\'ab'\'23'
   \'ff'\'ab'\'23'

   length determines base.
   hum ho but why not allow encoding of 16-bit and 32-bit and 64-bit
   values as well?
   (refer to s-expression syntax proposed by Rivest.
   http://theory.lcs.mit.edu/~rivest/sexp.txt)

-  audit print, err, tell for print format stuff. beware of %c.

-  is buzzz.1 installed in regular path?

-  \| can be given multiple definitions. that's either useful
   or funny.  are glyph constants only seen at filter time?

-  formatted output, C printf like?

!  verbatim automatic indent in html, not so in roff.

-  what kinds of chars are acceptable as anchors in html?

x  inspect#2 bug which is hard to fix:
   \inspect{posix{\\b}{()}}{ab cd ef gh}
   \b matches; pointer is set to next char; \b matches again.

==========================================================================

chapter support:

-  write chapter mode in zmm macros:

in order to offer more info-like behaviour (e.g. chapters in different
-utput files):

-nly additional zoem primitive needed is ability to change default output file.
\defaultfile{..}

refs need to include an extra infix.

\sec does either
   \setx{refinfix}{chap\ctrput{sec}}
-r \setx{refinfix}{}

\anch{} {... \refinfix ...}
\setx{%{refinfix}{intro}}{\refinfix}

\iref{intro} then uses \%{refinfix}{intro}
and e.g. \base\%{refinfix}{intro}.html#\1

then there might (or might not) be some issues with .zmr and .zmt files.
dunnow about them yet.

==========================================================================

regular todos:

?  remove 9 args limitation?
   WELL, the vararg stuff seems to lend enough power.
?  maintain stack of ting pointers to keys currently being parsed.
   this should make error analysis easier.  For ease I might even include the
   ting pointer in the segment struct.  it can be printed while unwinding.
   HOWEVER, it simply prints the reverse now found with --trace-keys.

-  how about the totally hackish callback prefix trick \.foo idea? (s/dotfoo)
-  more error checking in let wrt overflow, nan.
-  portability of isnan, isfinite. on CWI, <ieeefp.h>
-  fix the YAM_ARG_MAX vs 10 misery.
-  do I have separator that vanishes with eval? just \""? #ans: \!{}
-  let interface is not so clean, int/i32/mcxenum/flags - wise speaking.
-  perhaps different naming for look,grep in inspect2.
-  clean up parsing code (the char slurping guys).
-  clean up filtering code.
-  find out which units are supported by both html and troff.
?  add troff top+bottom margin support to spacing env.
-  may need print macro for numbers; e.g. to limit nr of decimals printed.
-  give let a decimals argument.
-  reconsider debugging and verbosity design


/  ownership output is divided between file and filter modules.
   well, no: it's owned by file, but filter keeps track of the name
   of the current default output stream.

/  closing file handles: need to set handle to NULL (otherwise if I inspect
   it a second time it'll contain gargbage).
   also, mcxIO filehandles need to be closed by method.
   (what did I mean here? -- there's mcxIOclose)

?  yamParseScopes does not choke on {aaa}: ok? (rather than {} or {{aaa}}).

/  tagoffset skips \!{ .. } enclosed parts.
   is that really what I need?

/  ownership + mcxTings as function argument.
   yamReadFile
   yamInlineFile

/  unify yamOutput and yamDigest (nah) ?

#  [inspect2 grep possible bug. grep seems not implemented?]
   mnot correct needs look as well ..
   grep can be used with substitute pattern and without look;
   look really needs grep to be useful I believe.

#  if yamElemsParse errs, it does not know  how many tings it wrote
   should add sth in struct to count that.
#  fased out usage of yamElemsParse; it indicated poor interfaces
   (e.g. inspect#2).

#! buzzz inspect2 inclusion bug (size limitation?????)
   seems known regexp bug (google regex exec long string fails)
   temporary fix: make separate special purpose stripping program.  grrr.

?  at directive: don't write newlines (shorthand for cumbersome
   translate expression)?

-  perhaps better support for counters attached to stacked scopes
   (current practice is somewhat cumbersome)

-  do finsert and zinsert share code with dofile? should not be the case
   I believe (e.g. wrt inline files).

-  what when i want to apply different keys to the same data?
   currently difficult wrt backslash i believe.

   \def{foo#1}{[\1]}
   \def{bar#1}{{\1}}
   \def{zut#1}{<\1>}
   \apply{_#1{\!\1{a}\|}}{{foo}{bar}{zut}}
   does not work because after expansion apply uses
   anon key containing '\\1'

 ! \apply{_#1{\1{a}\|}}{{\!foo}}
 ! *does* work

   \def{$foo#1}{[\1]}
   \def{$bar#1}{{\1}}
   \def{$zut#1}{<\1>}
   \apply{_#1{\!$\1{a}\|}}{{foo}{bar}{zut}}
   works; apply uses
   \$\1

   \def{foo#1}{[\1]}
   \def{far#1}{{\1}}
   \def{fut#1}{<\1>}
   \apply{_#1{\!f\1{a}\|}}{{oo}{ar}{ut}}
   works also.

   dotfoo
   A conceivable feature 
   is that during parsing
   \.foo is simply interpreted as \foo
   this would enable callbacks, e.g. for shipping typeface information:
   bf, it, tt etc.
   \apply{_#1{\!.\1{a}\|}}{{foo}{bar}{zut}}
   but it is seems too special purpose to be good.
   \def{bf#1}{bf[\1]}
   \def{it#1}{it[\1]}
   \def{st#2}{\apply{_#1{\!{}\1{\2}\|}}{{foo}{bar}{zut}}}

   nuther solution is to make cb_bf, cb_it, cb_tt keys that map
   to bf, it, tt.

-  can it be useful to keep track in yamfilterdata, the state in
   which the last character was output (at or plain)?
   This pertains also to the squashing problems.

-  interface for precision

?  \inspect{posix,grep,lines{^From:}{}}{\finsert{/var/mail/stijn}}
   does not grep.
   \inspect{posix,grep,lines{(^From:.*)}{_#1{\1}}}{\finsert{/var/mail/stijn}}

?  enable \itembar in itemize. The generic_zmm.azm man page looks ugly
   (without it).

-  Is it interesting to see how much of m4's functionality can
   be supported? e.g. formatted output.
   No general formatted output: where is the end?
   par functionality, columns ..
   C printf family: maybe.

-  suppose paths can contain slashes (as under MS), then it has
   to be unprotected before use.

-  perhaps \car or \par should be required everywhere, so no implicit
   \par after e.g. \itemdef.

-  ZOEM_TRACE_REGEX make it a macro (and thus remove
   the ugly '& tracing_g' part)
   The current trace interface is rotten. difficult to find sth
   satisfactory though.

?  for lref etc, make the troff printed argument italic?

-  how about a '\register#2{name}{descr}' command, so that it's easy
   to find all macro's supported by a package.
   .zmm files are not parsed by -i flag. perhaps

   zoem -r man.zmm
   should list all registered macro's in man.zmm

-  after \itemdef, it is pbb best custom to require \car (or \par).
   for that matter, every piece of flow writing should be  preceded
   by one of those.

?  perhaps generalization of the stacking mechanism: multiple stacks.

-  do we need more protect/unprotect subtleties, finer control by user,
   e.g. wrt to sending data and retrieving output to/from system commands? 

-  it would be nice if you could sort composite structures on substructures ..
   this is pbb plain stupid to strive for though.

-  Different file names can refer to the same file (i.e ./foo, foo).
   Right now zoem hashes by file name. Is there a thing that is
   constant as long as the generic name is constant, i.e. the inode
   or file descriptor (I badly need education)?

-  make \dots commando.

-  repeated parsing stuff in read.c (viewed alone),
   repeated parsing stuff in parse.c (viewed alone),
   repeated parsing stuff in {read,parse}.c

   repeated stuff in filter.c, messy look of filter.c

#  difference between sibref and aref wrt to html extension.

-  check all uses of href and httpref (local use of httpref is conceivable).
-  should sibref take full name?
   href is now used for both internal and external references.
   should perhaps rename lref to iref, href to lref.

?  can car work in conjunction with itemize?
   should  implement it for troff, tis now unconditional par skip.

-  find out the best values for $w1 and $w2 in itemize env, and
   find out whether width and height should have dimension in <td>.

-  \protect, so that I need not escape backslashes with vim all the time.
   that could make \finsert obsolete, except that it is more efficient.

-  should use \<> syntax more often: does not require the repeated
   \@{..} \$xx \@{..} \$xx \@{..} alternation.
   this may require though that the indentation rules are changed.

-  zoem in interactive mode opens stdia.ozm.

-  do sth with ftinc in html.

-  checks by xml sugar should pbb not be fatal -- easier for debugging.

-  perhaps make substr macro, also allowing -1 syntax.

#  in html, after verbatim, no <p> makes ns forget about text-align:justify:
   <p> present makes ns insert extra vertical space.
   made \car and \car#1 to combat this, but it is slightly ugly,
   as it is html-specific whether car or par should be used.

?  summary attribute for itemize?
?  anchor attribute too?

-  can I remove table#5? mm not entirely replaceable by apply.

man.zmm:
   o  should make indent smaller.
      perhaps make it adjustable, because faq and man need different amount?

-  automatic spacing with xml sugar is different from what I do when
   coding in at scope.
   (e.g. <table><tr>). remove automatic tabbing from sugar, or remove
   everything?

x  how about TOC's where all numbers are aligned to the left :)
   with those neatly dotted lines.  That would not be an itemize env, but
   probably a toc env. interesting whether it'd be doable in html without gifs.

-  unprotect could take list of chars to unprotect, or take mode arg.
   1=bs, 2=lc, 4=rc.
   should inpspect unprotect bs? myes,
   \B must be specified as \\B.
   \\ must be specified as \\\\.
-  wat would  unprotect do with @ scope? 

#  the folllowing should be fixed:
\inspect{posix}{$}{()}{ab
cd
ef}
   $ matches;  pointer is set to after 'ab'; $ matches again.

?  integrate 'margin itemize' from faq with plain itemize?
   then w1+w2 align right up to the current margin border.
   perhaps difficult, as the faq aims for a particular visual effect.
-  would also need to provide 'dotted' functionality.
-  would also need to provide right-align in margin.

?  make exported symbols known to the world.
   \export{key}{signature}{descr},
   \export{key}{signature}.

?  interactive mode input buffering does not work ok:
   try pasting
\system{sort}{}{Security Updates for Woody. Some of you may have noticed already that
the Security Team is already supporting Woody in their [5]advisories.
Apparently the new [6]security build structure is working properly.
In order to use the updated packages automatically, you should add deb
http://security.debian.org/ woody/updates main contrib non-free to
a}
   a few times and you'll see a yamClosingCurly freaking out.
(this seems fixed)

#  '/' close indicator at start (as in </foo>)
   and in unary (as in <foo/>) are assumed to be at begin/end resp,
   presence of intervening ws is not checked. Is it allowed in XML anyway?

   so perhaps add checking for ws, complain and warn if found.
   shd now be easy given the new mcxStr[R]Chr{Is,Aint} functions.

-  ownership of the mcxTing objects, is a potential fountain for bugs.
   so far i seem to do ok. sth like a design, or rules, or documentation
   would be cool though. I have hinted at ownership at some places, for a start.

-  does count work without vararg or lines?

-  in the parsing stuff, try to define and use character classes,
   e.g. issyntax (for \, {, and }).
   possibly, the mcxStr[R]Chr{Is,Aint} are handy too.

#  the xml stackidx_g hack works with multiple files
   only if writing begin and end tags are nested as should.
   (i.e. multiple files are treated as one big file).

-  inspect2: moet duidelijke specs en restricties maken.
-  inspect2: implement limit

-  make test cases:
      patterns
         ([^o]*)
         ()
         (.*)
      strings (as scopes)
         {}

-  yamKeyGet does not return a const Ting. Why not?
   it should be clear which routines return the *real thing* and
   which return a *copy*.

-  finsert and zinsert should be able to use inline files as well.
   finsert should then escape the data afterwards (??).
   finsert: choice between escaping only backslashes or curlies as well.
   must escape curlies (for scope parsing).

-  \% could return a vararg if the access sequence maps to sth
   without a value, or should it become optional argument?

#? apparently, readFile searches for inline files, but these are
   only made by yamReadFile.
   made quick fix for this.

-  inspect: acts on zoem data. perhaps it should be able to act
   on 'raw' data -- that would require a primitive I guess.
   mm, pbb not a good idea. Right now raw data can be obtained
   by escaping every backslash, and that's fine.

-  would there be a use in zoem for user-defined named dictionaries,
   other than the d-tree? Would it have a persistency logic different
   from or extending pop/push?

-  check use of mcxTingEnsure where mcxTingEmpty should be used instead.

-  maybe more context should be avaible to the match.
   like, what the whole line was.
   can I now implement grepping in zoem?

-  can yam error functionality be replaced by general util/err stuff?

-  why do openscope and endscope differ wrt to ting and char* argument?

-  in \begin and \end, should I check for forbidden characters?

?  add warning at EOD(ocument) if a \begin scope is not closed.

?  make it possible to unformat a package

?  make sure that '_' is captured as a token, not as some
   undefined user key.

-  maybe I need a way to enforce match between block #k and
   nr of args of apply. nother primitive?

-  the many empty [>] filters are a bit annoying (all squashing ws).
   think of a way to show a one-line initial prefix of what's filtered,
   especially wrt ws.

-  traceput c argument is funny. better (symbolic) interface needed.

-  adding (various?) 'strict' behaviour; e.g. in relationship to
   the data tree (warn if access fails).

-  zoem data manipulation: need a way to turn of space munging ..
   or is \@{\w} sufficient?

-  maybe trace output should be more legible.

-  replace the oft-seen couple yamSegFree/mcxTingFree with yamSegInit.


primitive candidates:
length
substr
