# $Id: FAQ,v 1.30 2004/09/29 08:10:09 andreas_o Exp $ #

Oinkmaster FAQ
--------------

Q1:  Oinkmaster says the file in URL does not exist (or it exists but
     doesn't work with my Snort).
Q2:  I want to download the rules archive using the program XXXXX, which 
     I can't do with Oinkmaster.
Q3:  Can Oinkmaster be useful for distributing rules locally to my 
     sensors?
Q4:  Can Oinkmaster be useful for distributing my homemade rules to my 
     sensors?
Q5:  In which order will the localsid/modifysid/disablesid/enablesid 
     configuration directives be processed?
Q6:  How can the 'include' statement in oinkmaster.conf be useful?
Q7:  Can Oinkmaster restart Snort after an update?
Q8:  Can I tell Oinkmaster to not touch certain SIDs in sid-msg.map so 
     the entries I add for my local rules don't get deleted?
Q9:  I want to copy the rules archive from a remote host to the sensor 
     via scp without having to supply a password or passphrase, but I also 
     don't want to give the sensors full access to that host. Can this be 
     done?
Q10: How do I setup proxy configuration?
Q11: Can Oinkmaster be used to update rules from www.whitehats.com?
Q12: Will Oinkmaster fail when new keywords are introduced in the rules?
Q13: Under what license are the official Snort rules released, and
     can I modify them without violating it?
Q14: Is there a GUI for Oinkmaster?
Q15: When starting Oinkmaster it says my Perl version is too old, or 
     gives me errors about "too many arguments for open".
Q16: If I make a modification directly in a rules file that is updated by
     Oinkmaster, will that change be permanent?
Q17: Should I run Oinkmaster as root?
Q18: I get "/usr/bin/perl: bad interpreter" when starting.
Q19: I've run Oinkmaster several times and it still says there are no
     changes in the rules, how often are the rules tarballs on 
     www.snort.org updated?
Q20: Can Oinkmaster send email or execute a command when the rules have
     been modified?
Q21: What do I do if I need to locally customize (not disable) some 
     rules (and so that those modifications become permanent)?
Q22: I want to disable rules by editing the files directly and putting 
     a "#" in front of the rules. How do these changes become permanent?

--------------


Q1:  Oinkmaster says the file in URL does not exist (or it exists but
     doesn't work with my Snort).

A1:  Check the "url = ..." line in oinkmaster.conf (or argument to "-u" 
     if you specify it on command line). The URL you should use depends 
     on the version of Snort you run. This means that the default URL does
     not work for everyone! So, you must make sure you really use the one 
     that is right for you. Visit http://www.snort.org/dl/rules/, read 
     the instructions there, and pick the right URL. As the official Snort 
     rules archives now have version information in the filenames, you 
     must remember to update the URL info for Oinkmaster when upgrading to 
     a new major version of Snort. You can also make a tiny wrapper to 
     create the right URL string on the fly by using the output from 
     "snort -V". Keep in mind though that the Snort people can change the 
     download layout again at any time, so if you experience problems, go 
     to the Snort web site and see if anything is mentioned there.

     If you are absolutely sure you're using the right rules for your
     version of Snort but still have problems, you may need to read the
     Snort manual because it is likely that your local configuration is
     broken. For example, the new rules maybe contains new variables that
     you forgot to add. But of course there can be times when the rules
     archive actually is broken somehow (remember that auto-update of the
     rules with automatic Snort restart is bad unless you are careful).
     These issues are often mentioned on the Snort mailing lists.



Q2:  I want to download the rules archive using the program XXXXX, which 
     I can't do with Oinkmaster.

A2:  Use whatever program you want to download the rules archive to the 
     local filesystem first, and then run Oinkmaster with
     -u file://<filename>.



Q3:  Can Oinkmaster be useful for distributing rules locally to my 
     sensors?

A3:  Yes, there are several situations where Oinkmaster can be handy when
     dealing with local rules management. If you have only a few sensors,
     you can simply run Oinkmaster in a traditional way to update the 
     rules from the Internet on all of them. However, if the sensors have 
     no way to talk to the outside, and/or you have many sensors you want 
     to control in a convenient way, you'll want to consider another 
     solution.

     A common way to solve this is to have a "master" host which downloads 
     the rules using Oinkmaster and does some global modifications to 
     them. With some simple scripting, it can then push the rules archive 
     to each sensor which runs Oinkmaster on it. You can also have each 
     sensor run Oinkmaster and pull the rules archive from the master, for 
     example using -u scp://... and point to the master. Each sensor can 
     have its own oinkmaster.conf to fine-tune the rules (even override 
     global settings done by the master).

     An example step-by-step list how this could be done:

     1) foohost, the master, uses Oinkmaster in a traditional way with 
        its own oinkmaster.conf to download the rules into some local 
        directory. Rules that you modify here will become modified on ALL 
        sensors by default. This is a convenient way to disable rules that 
        you don't want to use on any sensors instead of having to disable 
        them on each sensor, for example. If you don't want any global 
        preprocessing of the rules, you could just use a simple wget of 
        the archive on this host (or run Oinkmaster with a NOOP config).

     2) Create a script that re-creates the rules tarball from the rules
        just downloaded. You could then either push this new archive to 
        all the sensors or have each sensor pull the tarball from foohost 
        using '-u scp://user@foohost:/...'. If your sensor does not have 
        a way to talk to other hosts at all (like when only using 
        receive-only cables), the rules archive will obviously have to be 
        distributed manually using a removable media like a USB memory 
        stick or something. If you pushed the archive to the sensors, they 
        should run Oinkmaster with '-u file:///...' and point to this 
        file. In each local oinkmaster.conf, you can fine-tune the 
        rules to fit this particular sensor.



Q4:  Can Oinkmaster be useful for distributing my homemade rules to my 
     sensors?

A4:  Yes! This can be done almost exactly like in the example above.
     The main difference is that in step 1, you don't download from the
     Internet but rather point to a rules tarball created from a local
     directory (perhaps checked out from a cvs repository) containing 
     your homemade rules. 

     There are a few scripts in Oinkmaster's contrib/ directory (that all 
     handle multi-line rules) that may be useful in this case. For 
     example, addsid.pl will parse all your rules and add the next 
     available SID to rules that don't have any. This may be useful to run 
     before creating the rules tarball. So when adding new homemade rules 
     on the master (and you only have to add them there since they will be 
     distributed to all your sensors automatically), just leave out the 
     SID and it will be filled in for you automatically. You can also use 
     contrib/create-sidmap.pl to create a new sid-msg.map from your rules.



Q5:  In which order will the localsid/modifysid/disablesid/enablesid 
     configuration directives be processed?

A5:  localsid is processed first and if there is such a statement for a 
     rule, no further processing of that rule is performed.
     The order of processing for the others is:

     modifysid -> disablesid -> enablesid

     The order they appear in oinkmaster.conf does not matter. So if you 
     use both "enablesid 1" and "disablesid 1" in oinkmaster.conf (in 
     any order), SID 1 will first be disabled and then enabled. So if 
     SID 1 was enabled by default in the downloaded rules archive, this 
     is a NOOP. Disabling a disabled rule is also a NOOP.

     Note: If you specify "-e" on the command line (to enable all rules
           that are disabled by default), they will be enabled before
           the modifysid/disablesid/enablesid happens. It still has no
           effect on localsid rules; they are always preserved.



Q6:  How can the 'include' statement in oinkmaster.conf be useful?

A6:  There are many situations where this is useful, but perhaps the most
     common usage is to be able to use one global and one sensor-specific
     configuration file. You would usually want to make sure to include 
     the sensor-specific file last since possible values that are 
     redefined overrides the previous ones. Remember that you can also 
     use multiple "-C" arguments when starting Oinkmaster to make it load 
     multiple configuration files. They will be loaded in the order of
     appearance on the command line.



Q7:  Can Oinkmaster restart Snort after an update?

A7:  No. This functionality does not belong in Oinkmaster since almost 
     everyone has their own way of restarting Snort. Automatic restart of 
     Snort after an update is usually not a good idea either. See the 
     README for more information. If you really want to do an automatic 
     restart, just write a little wrapper that does what you want, and 
     remember to run "snort -T" first and refuse to restart if the test 
     is not successful.



Q8:  Can I tell Oinkmaster to not touch certain SIDs in sid-msg.map so 
     the entries I add for my local rules don't get deleted?

A8:  No. What you may want to consider instead is to use a separate
     sid-msg.map for your local rules (and not touch the map for the
     official rules), and then join the two before feeding it to 
     Barnyard/whatever. Or, what is probably even better, is to generate 
     the sid-msg.map automatically yourself after each update. There is 
     one script in Snort's contrib directory that does this, and there is 
     also one that comes with Oinkmaster (contrib/create-sidmap.pl). The 
     latter handles multi-line rules and can also take an arbitrary number 
     of rules directories as argument.



Q9:  I want to copy the rules archive from a remote host to the sensor 
     via scp without having to supply a password or passphrase, but I also 
     don't want to give the sensors full access to that host. Can this be 
     done?

A9:  Yes. You should really rely on the OpenSSH documentation on how to 
     do this as it is well documented there, but an attempt at 
     explaining how it could be done follows. The instructions assumes 
     OpenSSH on both client and server. 

     If you want to automate the process of just copying a file with scp, 
     you should never have to give one host full access to the other. One 
     solution is to generate a key pair where the private key has no 
     passphrase and the public key has restrictions to only allow the 
     copy. (If you don't plan on running Oinkmaster unattended, it is of 
     course recommended to use a passphrase anyway, even though the public 
     key should still contain as many restrictions as possible.)

     First, generate the key pair on the sensor (i.e. the host that is
     going to run Oinkmaster, not the master host that keeps the 
     archive):

     ssh-keygen -t rsa -C 'oinkmaster copy' -N '' -f oinkmaster_scp

     This will give you two files in the current directory, 
     oinkmaster_scp (private key) and oinkmaster_scp.pub (public key). The 
     public key must now be edited to add some restrictions to it.
     Details about how to add restrictions to a host is described in the
     OpenSSH manual. Perhaps the most important restriction in this case 
     is the command="command". You should also use as many other 
     restrictions as possible, such as from="pattern-list" to only allow 
     the sensor(s), no-pty to prevent pty allocation and also turn off all 
     types of forwarding. You will see that the public key begins with 
     something like "ssh-rsa AAAA...". To only add the command 
     restriction, modify the key to say something like:

     command="scp -f /etc/snort/snortrules.tar.gz" ssh-rsa AAAA...

     This assumes that the tarball will be found as 
     /etc/snort/snortrules.tar.gz on the master host, but change it to the 
     location where you store the tarball there. When you are done 
     editing oinkmaster_scp.pub, append the contents of it to the master 
     host's ~/.ssh/authorized_keys for the user that will be used to 
     perform the copy. This should not be a privileged user, the only 
     requirement is that this user has read access to the rules archive.

     Now make sure that you put the private key (oinkmaster_scp) in a 
     suitable directory on the sensor. Then tell Oinkmaster to use this 
     key by specifying "scp_key = ..." in oinkmaster.conf,
     You could use something like 
     "scp_key = /home/oinkmaster/.ssh/oinkmaster_scp".
     Or you can use ~/.ssh/config to specify the key as described in the 
     OpenSSH manual. Then you should be able to start Oinkmaster with 
     something like:

     oinkmaster.pl -u scp://user@foohost:/etc/snort/snortrules.tar.gz -o rules

     It will copy the archive without asking for a password/passphrase.
     You should see something like:

     ...
     Copying rules archive from oink@somehost:/etc/snort/snortrules.tar.gz using scp:
     snortrules.tar.gz                                 100%  110KB   0.0KB/s   00:00
     Archive successfully downloaded, unpacking... done.
     ...

     If you start Oinkmaster in verbose mode (-v), scp will be run in
     verbose mode as well.



Q10: How do I setup proxy configuration?

A10: If you download the snort rules archive from a web site (e.g. 
     www.snort.org) and you are a behind an http proxy, you need to tell 
     Oinkmaster to use that proxy or the download will fail. How you do 
     that depends if you use wget (i.e. use use_external_bins=1 in 
     oinkmaster.conf) or libwww-perl (use_external_bins=0). You can read 
     their man pages for complete configuration instructions, "man wget" 
     or "man LWP::UserAgent".

     Quick instructions for the impatient:

     For wget, you can either edit the wget configuration file (usually
     /etc/wgetrc for global settings and ~/wgetrc for user settings) and 
     set the http_proxy option. For example: 
     "http_proxy = http://your_proxy_host:3128:/"
     (or http_proxy = http://user:password@your_proxy_host:3128/ if you 
     are required to authenticate to the proxy). You can also set the 
     http_proxy environment variable. If you want your proxy server not to 
     return the file in its cache but rather get it from the web server 
     again, add "header = Pragma: no-cache" to your wget configuration 
     file.

     For libwww-perl, set the http_proxy environment variable as 
     described above (i.e. to "http://your_proxy_host:3128:/" or 
     "http://user:password@your_proxy_host:3128/").



Q11: Can Oinkmaster be used to update rules from www.whitehats.com?

A11: No, the Snort rules on www.whitehats.com have not been maintained for
     a long long time and should not be used, although the arachNIDS
     database is still a good source of information for older rules.
     Use the rules on www.snort.org instead.

     (And for Oinkmaster to work in a technical sense, the rules package 
     must be a gzipped tar file that contains a directory called 
     "rules", which holds all the rules files.)



Q12: Will Oinkmaster fail when new keywords are introduced in the rules?

A12: No. (Not unless there is a bug in Oinkmaster anyway.) Oinkmaster was 
     designed to be stupid and not try to understand every single part of 
     the rules. Basically, for a rule to be successfully identified by 
     Oinkmaster, it must start with one of the keywords in 'rule_actions' 
     specified in oinkmaster.conf. It must also contain 
     'msg: "some message";' and 'sid: <sid>;'. Some basic syntax checks 
     are also performed, but the rest is ignored.

     If a rule cannot be successfully parsed for some reason, it will 
     simply be regarded as a non-rule line (or multiple non-rule lines if 
     it is a multi-line rule). Since non-rule lines are also compared and 
     updated by Oinkmaster, things will work as usual anyway except that
     actions such as disablesid on that rule will have no effect.

     Remember that even if Oinkmaster may not have problems with the new
     rules, they can still be invalid when being read by Snort. This has
     happened a couple of times in the past when the official rules
     tarballs contained broken rules (containing keywords that weren't
     valid for that version of Snort) put there by accident. Until
     those broken rules are fixed, the easiest thing to do is usually
     to simply add temporary "disablesid" statements for the them.



Q13: Under what license are the official Snort rules released, and
     can I modify them without violating it?

A13: The Snort people have answered this a few times on the Snort
     lists, but here is a small summary. The official Snort
     signatures are part of Snort and are therefore also released
     under GPL. You can modify them locally as much as you want,
     but if distributing them, they need to follow GPL and be 
     GPL'ed as well.



Q14: Is there a GUI for Oinkmaster?

A14: Yes, check out README.gui and contrib/oinkgui.pl.
     This is just a primitive front-end for Oinkmaster though.
     Oinkmaster was designed to be simple and also easy to use in scripts,
     so if you want a more advanced graphical interface for maintaining
     Snort rules, you should probably be looking at another tool instead.



Q15: When starting Oinkmaster it says my Perl version is too old, or 
     gives me errors about "too many arguments for open ...".

A15: Your Perl version needs to be upgraded to at least 5.6.1.
     The issue is common on FreeBSD boxes, which usually use Perl 
     5.005. There is a more recent Perl version in the ports tree though
     (lang/perl5.8 at the time of writing), so just install that one and 
     execute "/usr/local/bin/use.perl port" to globally enable it.



Q16: If I make a modification directly in a rules file that is updated by
     Oinkmaster, will that change be permanent?

A16: No! Permanent modifications in files that are updated by Oinkmaster
     must be done by editing oinkmaster.conf. If you edit a rules file
     directly, the next time you run Oinkmaster it will see that the file 
     is different than the downloaded one, and so your file becomes 
     overwritten. There is no such thing as "if you edit a rule, it 
     will automatically be marked as locally customized and therefore 
     not automatically updated". This may be different than some other 
     rules updating tools, but it is the expected behavior as Oinkmaster 
     was designed to work this way. You can however use the "localsid"
     keyword to achieve this behavior. See Q21 for recommendations how 
     to handle local customization of the rules.



Q17: Should I run Oinkmaster as root?

A17: No! You must run Oinkmaster as a user that has read/write access to 
     your rules directory and all rules files in it, but it should *NOT* 
     be a privileged user such as root! Never run Oinkmaster as root.
     Adding a new user to be used for running Oinkmaster is a good idea, 
     just make sure it has read/write access to your rules.
     (The user you run Snort as, if different than the Oinkmaster user, 
     usually only needs read access to the rules.)



Q18: I get "/usr/bin/perl: bad interpreter" when starting.

A18: By default, Oinkmaster assumes that Perl is found as /usr/bin/perl.
     If you get this message, you need to edit oinkmaster.pl and modify 
     the first line so it points to the right location for your system.



Q19: I've run Oinkmaster several times and it still says there are no
     changes in the rules, how often are the rules tarballs on 
     www.snort.org updated?

A19: The tarballs are currently generated from the CVS repository many 
     times a day, but that does not mean there are always any changes in 
     it. My experience is that the rules are sometimes updated daily, 
     and sometimes in bursts with several days/weeks apart (also 
     depending on which rules tarball you use). If you run Oinkmaster 
     for several days and it still says that nothing has changed, this 
     does not automatically mean anything is wrong. You can verify this 
     by doing some arbitrary modifications in some rules file in your 
     rules directory and then run Oinkmaster. If those 'arbitrary 
     changes' are noticed by Oinkmaster, everything probably works as 
     expected. If you are behind a web proxy, make sure it is not trying 
     to be nice by caching the rules archive and always give you the old 
     version instead of actually polling the Snort site for an updated 
     version (see Q10). Check out http://www.snort.org/ for the latest
     information about the rules and the rules release process 
     (Oinkmaster and its author has nothing to do with this).



Q20: Can Oinkmaster send email or execute a command when the rules have
     been modified?

A20: There is no built-in function in Oinkmaster to do that, because that
     functionality does not belong there. However, it is very easy to do
     it with a small wrapper. You can for example run Oinkmaster with the
     -q flag so it doesn't give any output unless the rules had changed,
     and redirect the output to a file. Then check the size of this file.
     If it is empty, nothing had changed. If it has some content, 
     something had changed. So then you can either send the content of 
     the file in an email and/or execute some command. Here is a basic
     wrapper script to show how you could do this. There is also an
     example in the README file.

     #!/bin/sh

     tmpfile=`mktemp /tmp/oinkmaster.XXXXXXX` || exit

     oinkmaster.pl -o /etc/snort/rules -q > $tmpfile

     if [ -s "$tmpfile" ]; then
         echo "The rules have changed."
         # Do stuff here, like send content of $tmpfile in
         # an email or execute some command.
     else
         echo "No changes in the rules."
     fi

     rm $tmpfile



Q21: What do I do if I need to locally customize (not disable) some 
     rules (and so that those modifications become permanent)?

A21: As seen in Q16 you cannot just edit a local rules file and 
     customize the rules, because they would be overwritten during 
     the next update. There is however often a need of being able to do 
     such modifications to the rules, and there are a few different 
     ways you could do this with Oinkmaster. One way is often the best 
     one and the other two are often less good, depending on the 
     situation of course.

     The often best way - modifysid (and templates)
     ----------------------------------------------
     If you need to modify rules slightly, this is definitely the way to 
     go. For documentation about modifysid, see oinkmaster.conf (don't 
     forget README.templates and template-examples.conf if you use 
     modifysid a lot and want to simplify the process).

     A modifysid statement is a way to apply a substitution expression 
     on specified rules after each update. Here comes the whole point of 
     why using this approach is often the best one: If the default 
     version of this rule is later improved somehow in the new rules 
     archive, you would automatically get the updated version while your 
     customization would still apply (assuming your substitution 
     expression still matches the new version of the rule, of course). 
     This means that even rules that you need to customize can be 
     continued to be automatically maintained!

     Some typical examples where using modifysid is the best way to go 
     is where you pretty much like the default rule and always want to 
     use the latest version of it, but you want to change some detail
     (such as switching $HOME_NET/$EXTERNAL_NET) or add something to it 
     (such as a tag statement). Another point is that all these local 
     customizations by using modifysid become kind of self-documenting,
     which you will appreciate when looking at your rules and wonder 
     what those local modifications you've done over the years actually 
     are.

     The sometimes less good way #1
     ------------------------------
     If you need to modify a rule heavily in such way that it becomes 
     too complex and too meaningless to try to use and maintain a 
     modifysid statement for it, you need to solve the problem in 
     another way. I would recommend using a "disablesid" statement for 
     that rule and then rewrite and copy it to some local rules file 
     (one that is not updated by Oinkmaster) and change its SID to a
     local one.

     Note however that by doing this, the rule will no longer be 
     automatically maintained! Be very careful so you don't end up with
     lots of rules that become forgotten and not maintained, as this is 
     the exact opposite of what Oinkmaster is created for.

     A positive side-effect of using "disablesid" for these rules is 
     that Oinkmaster keeps track of changes even in disabled rules in 
     the official rules files. So you may want to pay attention to 
     these - you might find an interesting change some day that you want 
     to apply to your version of the rule as well.

     The sometimes less good way #2
     ------------------------------
     As of Oinkmaster 1.1, you can use variant of the above as it 
     introduced the "localsid" keyword.
     
     By using "localsid <sid>" in oinkmaster.conf, you mark this rule as 
     being "locally modified" which means it will never get overwritten.
     Instead, your local version of the rule is kept. Any updates in 
     the official version of the rule is silently discarded, but if 
     you run in verbose mode (-v) Oinkmaster will print the current 
     difference between your version and the official version.

     Again, be warned that all "localsid" rules will no longer be
     automatically maintained! If you are using this feature more than 
     in a very few cases, chances are that you are doing something wrong.



Q22: I want to disable rules by editing the files directly and putting a 
     "#" in front of the rules. How to these changes become permanent?

A22: As can be read elsewhere in this FAQ, you shouldn't ever modify 
     rules in anyway by editing the rules files directly. Normally, the 
     right way to disable rules is to use the "disablesid" keyword. 
     However, if you for some reason really want to disable rules by 
     editing the rules files directly and putting a "#" in front of 
     unwanted rules, you can use a little trick.

     Use contrib/makesidex.pl to generate "disablesid" lines for those 
     disabled rules and feed them to Oinkmaster. You can for example use 
     something like this to update this information automatically and 
     redirect to a file included by Oinkmaster (we don't touch the main 
     config file):

     makesidex.pl /etc/snort/rules >autodisable.conf 2>/dev/null && \
     oinkmaster.pl -C /etc/oinkmaster.conf -C autodisable.conf \
     -o /etc/snort/rules
