Hi,

Let me introduce this clean-up branch:


- includes/pre.php vs. includes/init.php

includes/pre.php is renamed to includes/init.php for clarity.

init.php used to include a lot of PHP files that weren't always
necessary; now we use a C-style include model, where each file
declares which files it needs to include.

This should avoid including half of the Savane code each time a page
is served, but the main pro is to control where the code comes from,
without functions "magically" defined.


- include/require/include_once/require_once

Let's only use 'require_once' to load code:

 * require: if using 'include', then not finding the code is not a
 fatal error, while it is.

 * once: if the code is required or included twice, PHP will issue an
 error about redefined functions.

This matches C's "#include" + "#ifndef _MYFILE_H_".


- Security:

Previously all GET/POST variables were passed to the sane_get() /
sane_post() / sane_all() functions, where they were backslashed if not
already (i.e. depending on magic_quotes). That way they could be
passed as-is to MySQL.

This is actually not a good idea, because you then work with modified
data in the code (eg: "a'b" is now "a\'b" is among others 1 character
longer). Besides, not all values passed to MySQL come from sane_*
functions, so there's still a risk of MySQL injection.

In order to improve both correctness and security, I use another
method:

- We set register_globals=Off, then all data are manually imported
  into the namespace using extract(sane_import(...)). This is a kind
  of secure replacement for register_globals=On - t1he rest of the code
  can be left unchanged.

- All MySQL queries are done via ADODB-style db_execute() and
  db_autoexecute(), plus possibly db_query_escape() -
  cf. includes/database.php. Do _not_ using db_query anymore, which
  assume the query is already escaped. Those new functions also use
  mysql_real_escape_string(), which is meant for MySQL, instead of
  sane_*'s add_slashes().

Once a page is clean-up wrt input (GET/POST) or output (MySQL), we
"tag" it with:

#input_is_safe();
#mysql_is_safe();
  
You can use devel/sv_check_security.pl will be able to check which
files remains to be cleaned-up based on those tags.

After the clean-up I removed these tags. The following files were not
cleaned-up because they are old unused code:
  ../frontend/php/include/trackers_run/mod_filters.php
  ../frontend/php/include/trackers_run/postmod_filters.php
  ../frontend/php/forum/old/save.php
  ../frontend/php/forum/old/expand.php
  ../frontend/php/forum/old/index.php
  ../frontend/php/forum/old/monitor.php
  ../frontend/php/forum/old/thread.php
  ../frontend/php/forum/old/who_monitors.php
  ../frontend/php/forum/old/admin/index.php


- Autoconfiscation:

In order to use proper paths (localedir,
/usr/lib/where-should-I-put-my-perl-mod, ...) and installation methods
(options vs. interactive prompts), I'm trying to switch the main build
system to autotools, complemented by a classic Makefile.PL
(ExtUtils::MakeMaker) for the Perl module.

Autoconf updates frontend/php/include/ac_config.php with variables
like localedir and VERSION. Similarly Autoconf updates
lib/Savane/Version.pm.

'make dist' should work to make the tarballs and release-related tasks
(check NEWS file, etc.). I don't think we can support the case where
the build directory is different than the source directory though.

This also replaces a chunk of Savane-specific Makefiles with upstream
code (Perl and .PO installation, configure prompts).

Some people suggest using Module::Build instead. I think, though, that
the fact ExtUtils::MakeMaker relies on 'make' helps with autotools
integration (plus this is a really simple Perl package anyway).

See also "configuration" below.


- Find the gettext locales:

    # Without a call to bindtextdomain(), gettext() will search in
    # libintl's default LOCALEDIR, which is set at compile time. The
    # most common case is:
    # $datadir/locale = $prefix/share/locale = /usr/share/locale

    # However we could run on a PHP+gettext compiled with, say,
    # prefix=/usr/local; that's why the gettext documentation
    # recommends to always use bindtextdomain() at application
    # initialization, using the app's LOCALEDIR (and not
    # libintl's). This also avoid messing with /usr when installing
    # in /usr/local.

    # Check <gettext-source>/gettext-tools/examples/hello-php/ for an
    # autoconf example. One solution is autoconfiscating the PHP app
    # to set @localedir@; another is to use the configuration file,
    # and default to gettext's default path (ie. don't call
    # bindtextdomain is $sys_localedir is empty).

    # We still need this variable for the .po Makefile. For that part
    # (PO), let's use autoconf.


- Adjust Perl path at configure time:

backend/*.pl -> backend/*.in
#!/usr/bin/perl -> #! @PERL@
(à la cvs/contrib)


- Tests:

Instanciate a private HTTP and MySQL daemons (eg. ports 50080 + MySQL
socket, and perform an installation on them). Cf. tests/MECHS


- Savane configuration:

In order to simplify the installation, init.php defines safe defaults
or try to detect pertinent values (e.g. the Savane www root directory
and URL). The notion of 'required' option falls apart.

On the backend side, a few essential questions are now specified via
autoconf (e.g. database name). I suggest dropping the tedious
interactive process and ask the user to edit the configuration file in
case the default values are not satisfying (similar to OpenSSH or
Apache2's configuration files).


To ease the installation, a default 'admin' user and a default
'siteadmin' project are created. They can be renamed / removed later
if needed, but at least this doesn't force the user to fakely register
a first project which a special handling. Cf. db/mysql/bootstrap.sql.


Also, old "backward-compatible" configuration variables aliases that
seemed to date from the SourceForge days were removed and replaced in
the code.


- Images:

I used symlinks as much as possible (to replace duplicate images and
factor out themes that use the same set of images), and tried to group
icons when they form a 'family' or consistent set. I also removed old
unused icons (ftp->download, error->wrong), the 'transparent'
pseudo-theme (replaced by the groups of icons). The images size is
appreciably reduced from 6.9M/12M to 1.6M/2.3M (without/with generated
smaller icons) which makes it much smoother to checkout the repository
over SVN.

To spot the identical images, I first wrote a script that sorted icons
by checksum and listed the ones that were used the most. I then
visually checked each theme to spot the files that were binary
different yet actually identical (usually only the meta-data
differed).

It should be easier now to create a new theme by picking up and mixing
the different sets of icon families: 'arrows', 'bool', 'contexts',
'misc' and 'roles'.


---------- TODO ----------


- Database:

I suggest manually writing the SQL files: using MySQL's DESCRIBE loses
a lot of specification information (such as BOOLEAN -> INTEGER, FK ->
KEY transformation, etc) and adds a lot of default information that
clutters the description and should more generally be ommitted, in
case the MySQL maintainers decide to change that default for a better
one. There was also an issue with using different versions of MySQL
(bug7926@gna).


------ Misc Changes -----

- Disable cutting URLs in trackers and project descriptions because it
  annoys users who can't safely copy/paste a complete paragraph with
  the links intact (check 105807@sv)

- Stop removing the web cvs links when the 'homepage' option does not
  match the group type default URL - there're too many false positives
  including aliases and exceptions, this brings lots of support
  requests about missing webcvs links and instructions.
