                                hsh User Manual

Copyright (C) 2010 Alexander Taler (dissent@0--0.org)

This file is part of hsh.

hsh is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

hsh is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with hsh.  If not, see 
http://www.gnu.org/licenses/

Contents
--------

  Introduction
      Installation
      General Usage
  Display Elements
      Input Area
      Job Output View
      Job List Views
          Session List
          Running Jobs
          Command History
          Job History List
      Key Value Views
          Environment
          Aliases
  Command Line Syntax
       Prog
          Aliases
          Builtins
       Glob Expansion (*,?)
       Environment Variables ($)
       Named Directories (~)
       Job Reference (%)
       \ Escapes
       Quoted Tokens
       Job Directives (#)
  Jobs
       Piped Mode
       Pseudo Terminal
       Exclusive Mode
       Full-Screen Mode
  Commands
       Standard Builtins
       Default Aliases
  Configuration
  Trouble Shooting
       Facilities
       Common Issues

Introduction
------------

hsh is the {Happy|Human|History} shell, a command-line driven user interface to
your system.  It is inspired by the traditional shells (Bourne shell, C shell
etc.) but replaces the scrolling terminal style output with a full-screen
curses interface.

The hsh interface is split into two areas, the bottom few lines are for input,
and the rest of the screen is the main area, which shows one of a number of
different views, including job output, job lists, and environmental
information.  The cursor indicates which view has focus, and can be moved to a
new view using Alt-o or Alt-Tab.  The interface is dynamic, and will update
automatically according to events such as changes to running jobs or user
keystrokes.

    Installation
    ------------

hsh has a very simple manual installation.  The src and doc directories may be
placed anywhere on the filesystem.  The src/hsh.bin script is then used to
start hsh, and locates the other files relative to its location.  A .hsh
directory will be created in the user's home directory.

    General Usage
    -------------

Throughout this document keystrokes are specified using a keyname, which can be
a case-sensitive letter, a single special character, like $, or a
multicharacter special key name like F1 or PgUp.  Additionally a C- may be
prepended to indicate that the control key is held while typing the character,
or a M- to indicate that the key is 'escaped' by holding the Meta or Alt key
simultaneously, or typing an Escape immediately before.  Input in curses is a
funny thing, so some keystrokes are not available, eg. C-Tab.  The basic common
keystrokes are:

  C-f or Right
    Move forward one character along the line.
  C-b or Left
    Move backward one character along the line.
  C-a
    Move to the beginning of the line
  C-e
    Move to the end of the line
  F1 or C-h
    Show the welcome/help page.
  F2
    Show the job output view.
  F3
    Show the list of session jobs.
  F4
    Show the list of running session jobs.
  F5
    Show the command history.
  F6
    Show the list of all jobs.
  F10
    Show the last exception view.

Display Elements
----------------

    Input Area
    ----------

The input area is at the bottom of the screen, and is where you type your
commands for launching jobs.  By default the input area has five lines of text,
with the following uses:

  Line 1 shows the execution context: user @ hostname : current directory
  Line 2 shows the fully expanded command as it will be executed.
  Line 3 shows possible expansions for text currently being entered.
  Line 4 is where the command is entered by the user.
  Line 5 is a message line to alert the user of problems.

The cursor position is restricted to the single line of input text.

In addition to character entry and standard editing keystrokes, the following
keystrokes are accepted:

  Up
    Move back one row in history
  Down
    Move forward one row in history
  C-c
    Clear the currently typed command.  Don't run it, but save it in history.
  Tab or C-i
    Complete what is currently being typed as much as possible.
  Enter or C-m or C-j
    Execute the currently typed command, creating a new job.
  PgUp
    Move the main view up one page.
  PgDn
    Move the main view down one page.

    Job Output View
    ---------------

This is a main view which contains the output of a single job.  The top lines
contain information about the job, and the rest of the view contains the job's
output.  When the output is larger than the available area the cursor keys can
be used to move the visible area.

By default the header line colours are white on blue, and they contain the
following information:

  Line 1: context (user@host:directory) and unexpanded command
  Line 2: expanded command
  Line 3: job id, process id and state - output index - input mode - tail mode

The output area shows text in three different colours, white text is for
stdout, red text is for stderr, and yellow text is for manually entered stdin
(described just below).  By default the top of the output is shown when the job
is loaded, and the user needs to manually move the output to see the bottom.
However, tail mode may be enabled to cause the output view to constantly update
so that the end of the output is visible.

When a job is started, stdin is left open, and the job view may be used to
manually send input to the process's stdin.  Two input modes are supported,
'character' sends input to the process a character at a time, and 'line' allows
editing of each line and sends input only when enter is pressed.  Additionally,
stdin may be closed explicitly.  This is useful for interactive processes, such
as shells, or those which prompt for information.

Full-screen curses applications are handled differently.  For processes not run
in #exclusive mode (see Jobs and JobDirective sections below) hsh recognizes
that the process wants control of the terminal by identifying escape sequences
in the output and marks it as being in 'full-screen' mode.  When a job
requiring full-screen mode receives the focus, hsh relinquishes the terminal
and allows the user to interact with the process directly.  You can then return
to hsh by terminating the process, or typing C-z followed by Enter.

In addition to the standard movement keystrokes, the following keystrokes are
recognized in the job output view:

  M-i
    Cycle input mode.  Switches between 'silent', 'line' and 'character'.
  M-c
    Close stdin.
  M-t
    Toggle tail mode
  Enter
    Submit a line in 'line' input mode.

    Job List Views
    --------------

The job list views show a list of jobs, excluding their output.  There are
several such views, and more may be configured if desired.  The default ones
are: the session list which contains all jobs executed in this session; the
running jobs list which shows all jobs created in this session whose processes
are still alive; the command history which shows just the commands and includes
jobs which were never executed; and the job history list which shows all jobs
executed in this and other sessions.

A more restricted set of movement keys are supported, so that the cursor
highlights a single job.  (The highlighted job is not necessarily the same as
the job currently in the job output view.)

The session list and running jobs list show the execution context, unexpanded
command job id and status for each job, while the command history shows just
the job id and unexpanded command.

In addition to movement keystrokes, the following keystrokes are supported:

  Enter or C-m or C-j
    Within session list and running jobs list, this shows the highlighted job
    in the job output view.  Within command history it copies the highlighted 
    command to the input area.
  Delete or Backspace or C-d
    Only in session list this deletes the highlighted job.

    Key Value Views
    ---------------

There are several data dictionaries which the user can reference and modify.
Each of these shares the same interface to view and edit.  Additionally there
are builtins to display the view and provide an alternative for editing.

The view consists of two columns, the left contains the keys, the right the
values, with a row of text for each pair.  The cursor is restricted to the key
and value text fields, and the Tab key is used to switch between a key and its
corresponding value.  To edit a field, just start typing or using other editing
commands.  To save the changes hit Enter, otherwise hit Tab or the up and down
arrow keys to leave the field and abandon them.

Adding a new key value pair is not done using the interface directly, instead
the builtin command is used.  Pairs may be deleted by clearing the key field
and submitting the change with Enter.

Changes that have been made, but not yet put into effect, are indicated by
displaying the text in a different colour.

Key Value views currently provided are for the environment variables, accessed
by the 'env' builtin, and command aliases accessed by the 'alias' builtin.

Command Line Syntax
-------------------

hsh's command line syntax is still very young and not fully developed.  I
expect it to depart from that of other shells, since it will not be a
programming language.  Following the norms for process launching, the command
line generates a list of parameters, the first being the program to execute,
and the remaining ones passed to the program to use as it pleases.  There are
additional shell syntax tokens to complicate this.

Parameters are the coarsest level of separation on the command-line, and are
delimited by whitespace.

    Prog - the first element
    ------------------------

The first element of the command line specifies the program to run.  If this
starts with a . then the program will be searched for relative to the cwd
directory.  If it starts with a / then it will be searched for absolutely
within the filesystem.  If it starts with a + then the + will be ignored and
any builtin with that name will be run.  Otherwise, if it is the name of alias,
the alias will be expanded, and searching will begin again, except that aliases
will not be expanded again.  If it is the name of a builtin, the builtin will
be run.  Otherwise an executable will be searched for relative to all entries
of the PATH environment variable.

This differs from other shells in the handling of prog names containing a /,
but not starting with a / or ., eg imagemagick/mogrify.  hsh will search for
such things relative to the path, while other shells will search for it
relative to the cwd.

    Glob Expansion (*,?,[])
    -----------------------

Within a parameter '*', '?' and '[]' cause the text to be matched against
filenames, searching from the current directory.  '*' matches 0 or more
characters, '?' matches any one character, '[...]' matches any one character
contained between the brackets.  If multiple filenames match the pattern, then
it will expand to multiple parameters, and if no filenames match, it will
expand to nothing.

    Environment Variables ($)
    -------------------------

A '$' specifies an environment variable expansion.  The name of the variable
may optionally be enclosed in braces ('{' and '}'), if not it terminates on the
first non-alphanumeric, non-'_' character.  In the executed command this will
be replaced by the value of the environment variable.  This syntax is intended
to duplicate that of other shells.

    Named Directories (~)
    ---------------------

A '~' specifies a named directory expansion, and is followed by the name of the
directory.  The name may be optionally enclosed in braces ('{' and '}'), and if
not terminates on the first non-alphanumeric, non-'_' character.  In the
executed command this token is replaced by the user's home directory.  If no
name is given, the current user's home directory is used.  This syntax is
intended to duplicate that of other shells.

    Job Reference (%)
    -----------------

A '%' specifies a job reference expansion, and differs greatly from other
shells.  It can be used to access the output, input and pid of the job.  The
reference has the following syntax:
  '%' ['{'] [ job-id ] job-field [ ',' job-field ]* ['}']

  job-id: optional numerical job id, defaulting to the currently visible job.
  job-field: specifies what to expand.  Multiple job-fields may be specified
  separated by commas, which will result in an expansion to multiple
  parameters.  Possible values of job-field are:

  'p': Expands to the process id of a running job.
  'fo': Expands to the name of a file containing the job's standard output.
  'fe': Expands to the name of a file containing the job's standard error.
  'fi': Expands to the name of a file containing the job's standard input.
  'to': Expands to the text of the job's standard output.
  'te': Expands to the text of the job's standard error.
  'ti': Expands to the text of the job's standard input.

Some typical examples:

  kill %p

    Send the term signal to the currently visible job, if it's running.

  ls src/hsh
  egrep -v '\.pyc$' %fo

    Look at the directory listing, then remove all files ending with '.pyc'.

  sed 's/^\([^:]*\):.*/\1/' /etc/passwd
  finger %to

    Get login information for all system users.

    \ Escapes
    ---------

The \ character can be used to 'escape' the next character, indicating that it
should be taken literally and not subject to any special interpretation by hsh,
including any expansions, or parameter splitting in the case of whitespace.

    Quoted Tokens
    -------------

Two types of quoted tokens are supported, which are useful to simplify entry
when multiple characters must be entered without special interpretation.  Text
surrounded by double quotes ('"') will not be split on whitespace into multiple
parameters.  Text surrounded by single quotes (''') will not be split on
whitespace, and additionally will not be subject to any expansion.

    Job Directives (#)
    ------------------

A '#' starts a special directive for launching jobs.  These are interpreted by
hsh itself, and do not result in any expansion on the command line.  A job
directive can only start at the beginning of a parameter, #'s in the middle of
tokens or within quotes have no special meaning to hsh.  The token consists of
all characters following the # until the next whitespace.  Some of these
directives are explained in more detail in the Jobs section.  Some directives
come in groups from which at most one option may be selected, in which case the
last specified directive takes precedence.

Some groups of directives are mutually incompatible, and only the last one of
the group will be used: the first are the job I/O directives: '#piped',
'#exclusive', '#pty'; the second are the visibility directives: '#hide',
'#hideonsuccess', '#show'.

The following Job Directives are defined:

  "#exclusive": Launch job in exclusive mode.
  "#piped": Launch job in piped mode, which is the default.
  "#pty": Launch job in a pseudo terminal.
  "#hide": Do not make the jobs output visible.
  "#hideonsuccess": Do not make the jobs output visible if it succeeds in the
                    first quarter second.
  "#show": Always show the jobs output - the default action.

Jobs
----

When you type a command and hit Enter, hsh creates a 'Job'.  Each Job consists
of details about the command line used to create it, as well as some optional
stuff like an external program, process, output and input.  External programs
can be quite finicky about the environment they run in, so hsh must deal with
them appropriately.  There are still some rough edges to work out in this area.

    Piped Mode
    --------------

By default jobs use piped mode, where programs are run with pipes attached to
standard input, output and error.  This mode is appropriate for most
non-interactive programs, and some interactive ones.  It allows hsh the
greatest freedom for handling the external processes, without consumption of
system resources.  Interactive programs may wish to use pseudo terminals or
exclusive mode described below.

Piped mode may be selected explicitly using the '#piped' job directive.

    Pseudo Terminal
    --------------

Programs which require a terminal may be run within a pseudo terminal.  In this
mode the program gets access to a terminal interface on its standard input and
output, but hsh controls the other end of the pseudo terminal, and acts as an
intermediary to the actual terminal.  This allows the output and input of the
program to be stored by hsh.

Pseudo terminals are not available on all platforms, and are not the default
because they consume potentially scarce system resources.

To run a job in exclusive mode, use the '#pty' job directive.

    Exclusive Mode
    --------------

Exclusive mode is available for the most demanding programs, granting them full
control over the terminal.  This is essentially the same way that a traditional
shell executes all of its processes.  It is appropriate for jobs that require
terminal access, and which do not work within a pseudo terminal.

Output of jobs run in exclusive mode is not stored by hsh and cannot be
processed like output of other types of jobs.

To run a job in exclusive mode, use the '#exclusive' job directive.

Exclusive mode is useful for programs that write directly to the terminal,
notably security-minded programs like 'ssh', 'sudo' and 'gpg'.  Additionally,
some curses programs may be more manageable in exclusive mode than in a
psuedo-terminal.

As in job control for regular shells, a Ctrl-Z character will typically send a
STOP signal to the foreground process.  Hsh will recognize that the job has
stopped, resume itself and reclaim control of the terminal.

    Full-Screen Mode
    ----------------

Programs which are running in piped or pseudo terminal mode and send terminal
control codes are automatically put into full-screen mode, allowing the use of
curses interfaces.  When focus is given to a job in full-screen mode, hsh
relinquishes control over the terminal and allows the program to freely control
the terminal.  However, hsh is still acting as intermediary, which introduces
some limitations, and some benefits.

Jobs in exclusive mode are not being mediated by hsh and so will not use
full-screen mode.  Unlike exclusive mode, full-screen mode does not require the
user (or an alias) to explicitly enable it, instead it is automatically enabled
for any program which uses terminal codes.

Programs in full-screen mode are not able to determine the actual terminal
size, and communication of extended keystrokes may not work fully as expected.
Output is not recorded in full-screen mode, and consequently can't be processed
as for other jobs.

The Ctrl-Z character is interpreted specially in full-screen mode and is
trapped by hsh instead of being sent through to the job.  If the next character
is another Ctrl-Z then a single Ctrl-Z will be sent to the job, an action which
will often result in suspending the job's execution.  If the next character is
Enter (newline/Ctrl-M/Ctrl-J), then hsh will move focus away from the job, but
without suspending it.

Commands
--------

    Standard Builtins
    -----------------

  alias
    With no arguments, show the alias management view; with one argument,
    delete the named alias; with two arguments, set the alias named by the
    first argument to the value of the second argument, creating it if it
    doesn't exist.

  cause_internal_error
     Raise an exception, useful if you want to see the exception view.

  cd
     Change the working directory to the named directory, or home directory if
    no arguments are given.

  copying
     Display the terms for copying hsh, and the license under which it is
    distributed.

  debug_dump
     Evaluate the argument as Python code and write the results to the
    logfile.  Intended for development and tricky debugging.  Definitely
    could do with some improvements.

  env
     With no arguments, show the environment management view; with one
    argument, delete the named environment variable; with two arguments, set
    the environment variable named by the first argument to the value of the
    second argument, creating it if it doesn't exist.

  exit
     Exit the hsh process.

  hsh_view
     With no arguments, show a list of all the available views; with one
    argument, show the named view in the main area. 

  reexec
     Restart the hsh process, reloading any Python code.  This is mostly
    useful for development, and needs improvement to preserve more state.

  version
     Display the version of hsh this is.

    Default Aliases
    --------------

  gpg
  gpg #exclusive
     For security, and because gpg can trash the terminal.

  sudo
  sudo #exclusive
     For security, and because sudo can trash the terminal.

  ssh
  ssh #exclusive
     For security, and because ssh can trash the terminal.

  aptitude
  aptitude #exclusive
     Curses app requiring full screen access.


  python
  python #pty
     Python won't go interactive without a tty.

  sh
  sh #pty
     sh won't go interactive without a tty.

  top
  top #pty
     top insists on having access to a tty.

  quit
  exit
     quit is a familiar way of terminating a shell.

  logout
  exit
     logout is a familiar way of terminating a shell.

  bye
  exit
     bye is a cute way of terminating a shell.


Configuration
-------------

The user level configuration mechanism has not yet been defined.  Keybind lists
and visual configuration files exist in the source code, and the user
configuration will follow this mechanism.  It is acceptable to modify these
files currently in the installation, although this will create difficulty in
managing the source.  This area is high priority to address.

The source code files of interest are hsh/curses_display/keybinds.py and
hsh/config.py.

It is expected that many configurable options will be added.

Trouble Shooting
----------------

    Facilities
    ----------

hsh provides several facilities for tracking down problems:

Unusual events or user errors can trigger an alert message on the bottom line
of the display.

Exceptions in the main thread are caught and displayed in the Exception View.
hsh continues to run after the exception, although its runtime integrity may be
compromised.  The exception view is available on F10 by default.  Try running
+cause_internal_error for a harmless look at the exception view.

A log file is created as ~/.hsh/debug.log, which includes various debugging
information, most importantly the stack traces of any exceptions.

The +debug_dump builtin can be used to examine internal data structures using
python code directly.  (This facility could do with some improvement.)

    Common Issues
    -------------

Sometimes (particularly during development) a job's output will stop appearing.
This is typically due to an error in the job's output processing thread.  Take
a look in the debug.log file for any exception traces from the processing
thread.  (This issue really needs an alert message.)

