pmacct (Promiscuous mode IP Accounting package)
pmacct is Copyright (C) 2003-2008 by Paolo Lucente

A brief preamble: this FAQ document is pretty young and fresh. Because of this you
should not expect to find all possible answers here, that is, please don't take it
the oracular way: it will gradually get filled. The question codes (Q1, Q2, ..) do
not endorse any specific order and are solely meant for cross-documentation
references.


Q1: Hey boy, ok, funny preamble. What is pmacct project homepage ?
A: It's http://www.pmacct.net/ . Actually there isn't any other official mirror site. 


Q2: 'pmacct', 'pmacctd', 'nfacctd', 'sfacctd' -- but what do they mean ?
A: 'pmacct' is intended to be the name of the project; 'pmacctd' is the name of the
   libpcap-based IP accounting and aggregation daemon; 'nfacctd' is the name of the
   NetFlow v1/v5/v7/v8/v9 accounting and aggregation daemon which entered the project
   starting from version 0.7.0; 'sfacctd' is the name of the sFlow v2/v4/v5 accounting
   and aggregation daemon which entered the project starting from version 0.9.0.  


Q3: What does pmacct offer, what can it do for me, what are its main features ? 
A: pmacct can collect and export network data. Collect in memory tables, SQL databases
   (MySQL, PostgreSQL, SQLite 3.x). Export data speaking sFlow v5 and NetFlow v1/v5/v9.
   pmacct is suitable to aggregate data. It can perform tagging, to mark data subsets
   (up to singular flow granularity), classification (ie. this is Kazaa, that http and
   that eDonkey), filtering, random sampling and renormalization. It's actually not
   suitable to log raw data (ie. packet-by-packet as tcpdump does or flow-by-flow as
   flow-tools or sflowtool do). And finally it has not been thought to write into flat
   files.


Q4: I use flow-tools since many years and, because of the actual working environment,
   i cannot simply replace the flow-capture with nfacctd as NetFlow collector. So, do
   i have any change to let flow-tools and pmacct work together ?
A: Yes. Some reasonable approaches follow:
   a) the flow-send tool is able re-send flow-tools savefiles via UDP. In such scheme,
   flow-tools receives NetFlow datagrams from the NetFlow agent and saves them onto the
   disk. At some fixed time intervals it sends them to a running nfacctd instance. 
   b) pmacctd - since 0.8.2 release - is able to read libpcap savefiles, which is also
   one of the output formats supported by the flow-export tool. The following example
   is quite simple (no aggregation, tagging and selection features available in pmacct
   are involved):

   shell> cat /[...]/ft-v05.[...] | flow-export -f 1 | pmacctd -P mysql -I - -c src_host,dst_host,src_port,dst_port,proto

   The equivalent configuration directive for '-I -' switch (which roughly means: read
   libpcap savefile from stdin) is: 'pcap_savefile: -'. This scheme is similar to the
   previous one. 
   c) use the UDP samplicator tool (http://www.switch.ch/tf-tant/floma/sw/samplicator) 
   to replicate the received NetFlow datagrams to a set of collectors (e.g., nfacctd,
   flow-capture and flow-receive). Compared to flow-fanout, it is of generic use (that
   is, not limited to just pmacct and flow-tools) and should be lighter (because of no
   PDU verificaiton, no time handling, etc.). 


Q5: When using the libpcap-based daemon, 'pmacctd', i feel the sensation of an high
   CPU usage: i see the 'pmacctd' process lurking a great CPU share. Any chance to
   reduce it ?  
A: Yes, there are good chances to reduce the CPU usage, posed that the CPU you are
   using for accounting/aggregation purposes is someway 'compatible' with the amount
   of traffic it has to process. To avoid unnecessary copies of data, also optimizing
   and buffering the necessary ones, is the key strategy to lower CPU usage.
   Kernel-to-userspace copies are critical, thus the first to be optimized; for
   this purpose you may look at the following solutions: 

   libpcap-mmap, http://public.lanl.gov/cpw/ : a libpcap version which supports mmap()
   on the linux kernel 2.[46].x . Applications, like pmacctd, need just to be linked
   against the mmap()ed version of libpcap to work correctly. 

   PF_RING, http://www.ntop.org/PF_RING.html : it's a new type of network socket that
   improves the packet capture speed; it's available for Linux kernels 2.[46].x; it's
   kernel based; has libpcap support for seamless integration with existing applications.

   Device polling: it's available since FreeBSD 4.5REL kernel and needs just kernel
   recompilation (with "options DEVICE_POLLING"), and a polling-aware NIC. Linux kernel
   2.6.x also supports device polling. 

   Then look at the following solutions on pmacctd/nfacctd/sfacctd side (and for further
   details see also 'Communications between core process and plugins' chapter, INTERNALS
   document):

   'plugin_buffer_size': turns on bufferization. '1024', '2048' or '4096' are sufficient
   values for common environments. If the circular queue size (also referred as pipe size)
   is not defined, it is calculated the following way: ('plugin_buffer_size' / as) * dss .
   Where 'dss' is the default OS socket size and 'as' is the address size (2 bytes for a
   16 bit architecture, 4 bytes for 32 bit architectures, etc.).

   'plugin_pipe_size': sets the circular queue size. If bufferization is also enabled, this
   value has to be >= the buffer size. A warning message will advice you if the supplied
   parameters is exceeding the maximum allowed socket size (each Operating System imposes
   a maximum limit on the socket size, for example Linux implement such limits through
   the use of '/proc/sys/net/core/[rw]wmem_max'). Values like '1024000', '2048000' or
   '4096000' are sufficient for most common environments. 


Q6: I wish to account all traffic of my network, with an host breakdown; but i'm not
   interested in having the DB polluted from statistics about 'internet' hosts. Do
   i'm really forced to waste system resources and space ? Do i necessarily need to
   run more daemon instances ? 
A: No, you will be able to run a single daemon, attaching multiple plugins to it,
   each with its 'aggregate'/'aggregate_filter' directive pairs; you will need to
   'name' each plugin in order to bind a filter to it. A sample configuration fragment
   follows:

   ...
   aggregate[inbound]: dst_host
   aggregate[outbound]: src_host
   aggregate_filter[inbound]: dst net 192.168.0.0/16
   aggregate_filter[outbound]: src net 192.168.0.0/16
   plugins: mysql[inbound], mysql[outbound]
   sql_table[inbound]: acct_in 
   sql_table[outbound]: acct_out 
   ... 

   It will account all traffic directed to your network into the 'acct_in' table and
   all traffic it generates into 'acct_out' table. Furthermore, if you actually need
   totals, you will just need to play around with basic SQL queries.

   If you are just interested in having 'totals' instead, you may also rewrite the
   above piece of configuration the following way: 

   ...
   aggregate: sum_host
   plugins: mysql
   networks_file: /usr/local/pmacct/etc/networks.lst
   ...

   Where 'networks.lst' is a (local) networks definition file.  


Q7: I'm intimately fashioned by the idea of seeing all traffic flows in my network; i wish
   to aggregate my data enabling 'src_host,dst_host' primitives and run without any filter.
   I wish to see *EVERYTHING* !
A: Many technical consideration may be spent on this topic but they all have a common root: 
   while you can easily enumerate the number of hosts on your network (so, you can even
   approximately estimate the amount of resources you will need when running the application),
   you cannot estimate how many hosts are on the internet, that is, the number of peers your
   hosts will talk to. So, be careful and remember that if, say, 60.000 contemporary flows
   could be easily handled in a memory structure, they just would be an overkill if translated
   in SQL queries each few minutes. 


Q8: I use 'pmacctd', the libpcap-based daemon. What portion of the packets being captured from
   the network is included into the bytes counter ?
A: The portion of the packet accounted starts from the IPv4/IPv6 header (inclusive) and ends
   with the last bit of the packet payload. This means that are excluded from the accounting:
   packet preamble (if any), link layer headers (e.g. ethernet, llc, etc.), MPLS stack length,
   VLAN tags size and trailing FCS (if any). This is the main reason of minor skews reported
   comparing pmacct counters with SNMP ones, for example. 'pmacctd' counters should be what
   you are interested in; moreover, they are in line with those offered by NetFlow.
   However, including such additional lengths is - in most cases - as simple as adding to the
   bytes counter a fixed size (which is dependent on the underlying network architecture) per
   each packet accounted.
   Let's give an example; Ethernet header = 14 bytes, Preamble+SFD (Start Frame Delimiter) =
   8 bytes, FCS (Framke Check Sequence) = 4 bytes. It results in an addition of a maximum of
   26 bytes (14+8+4) for each packet. The use of VLAN tags will result in adding 4 more bytes
   to the foremensioned 26. 
   If using any SQL plugin - and starting from release 0.9.2 - such bytes adjust can be done
   directly by pmacct via the 'adjb' action, inside a 'sql_preprocess' directive.


Q9: How to get the historical accounting enabled ? I see the SQL table having 'stamp_inserted'
   and 'stamp_updated' fields but they do not get any value. 
A: Historical accounting gets enabled by adding to the configuration a 'sql_history' directive.
   It's also highly adviceable to associate a 'sql_history_roundoff' to it. For details about
   the syntax of the two directives and some examples, take a look to the CONFIG-KEYS document. 


Q10: While giving a look to the ugly 'numbers' returned by either a SQL query or pmacct
   client, i feel a deep sense of 'pretty useless'. Do i have any chance to graph them ?
A: RRDtool, MRTG and GNUplot are just some tools which could be easily integrated with pmacct
   operations. 'Memory plugin' is suitable as temporary storage and allows to easily retrieve
   counters:
 
   shell> ./pmacctd -D -c src_host,dst_host -P memory -i eth0 
   shell> ./pmacct -c src_host,dst_host -N 192.168.4.133,192.168.0.101 -r
   2339
   shell>

   Et voila'. We get on our screen the bytes counter for our flow. Because of the '-r', counter 
   reset directive, each time we will get an 'ABSOLUTE' counter. Let's now encapsulate our query
   into, say, RRDtool commandline:

   shell> rrdtool update 192_168_4_133.rrd N:`./pmacct -c src_host -N 192.168.4.133 -r`

   Starting from 0.7.6, you will also be able to spawn as much as 4096 requests in a single query;
   you may write your requests commandline (';' separated) but also read them from a file (one per
   line):

   shell> ./pmacct -c src_host,dst_host -N 192.168.4.133,192.168.0.101;192.168.4.5,192.168.4.1;... -r 
   50905
   1152
   ...

   OR 

   shell> ./pmacct -c src_host,dst_host -N "file:queries.list" -r
   ...

   shell> cat queries.list
   192.168.4.133,192.168.0.101
   192.168.4.5,192.168.4.1
   ...

   Furthermore, SNMP is a widespreaded protocol used (and widely supported) in the IP accounting
   field to gather IP traffic information by network devices. 'pmacct' may also be easily connected
   to Net-SNMP extensible MIB. The following line is an example for your 'snmpd.conf':

   exec .1.3.6.1.4.1.2021.50 Description /usr/local/bin/pmacct -c src_host -N 192.168.4.133 -r 

   Then, an 'snmpwalk' does the reminder of the work:
   shell> snmpwalk -v 1 localhost -c public .1.3.6.1.4.1.2021.50 
   .1.3.6.1.4.1.2021.50.1.1 = 1
   .1.3.6.1.4.1.2021.50.2.1 = "Description"
   .1.3.6.1.4.1.2021.50.3.1 = "/usr/local/bin/pmacct -c src_host -N 192.168.4.133 -r"
   .1.3.6.1.4.1.2021.50.100.1 = 0 
   .1.3.6.1.4.1.2021.50.101.1 = "92984384"
   .1.3.6.1.4.1.2021.50.102.1 = 0

   Taking a look into examples tree of the pmacct tarball you will also be able to find a few
   bare shell scripts that could be taken as reference to accomplish this kind of tasks.


Q11: I wish to use NetFlow accounting; but my router generates times in seconds rather than
   in msecs. What to do ?
A: You have to use nfacctd together with a configuration file; then you have to insert the
   'nfacctd_time_secs: true' line in it. Remember that 'nfacctd' is also able to generate
   brand new timestamps avoiding to rely on times generated by your network equipment.
   To let it work this way you have to insert the 'nfacctd_time_new: true' line in your
   configuration file. 


Q12: The network equipment i'm using supports sFlow but i don't know how to enable it. I'm 
   unable to find any sflow-related command. What to do ?
A: If you are unable to enable sFlow commandline, you have to resort to the SNMP way. The
   sFlow MIB is documented into the RFC 3176; all you will need is to enable a SNMP community
   with both read and write access. Then, continue using the sflowenable tool available at
   the following URL: http://www.inmon.com/technology/sflowenable 


Q13: I've configured the pmacct package in order to support IPv6 via the '--enable-ipv6' switch.
   Now, when i launch either nfacctd or sfacctd i receive the following error message:
   ERROR ( default/core ): socket() failed. What to do ? 
A: When IPv6 code is enabled, both sfacctd and nfacctd will try to fire up an IPv6 socket. The
   error message is very likely to be caused by a missing (not loaded) ipv6 kernel module. So,
   try either to load it or specify an IPv4 address to bind to. If using a configuration file,
   add a line like 'nfacctd_ip: 192.168.0.14' or 'sfacctd_ip: 192.168.0.14'; otherwise go with
   a commandline like the following: 'nfacctd [ ... options ... ] -L 192.168.0.14'.  


Q14: I'm using the memory plugin to store data temporarily. Each bunch of seconds i use the
   pmacct client to gather statistics and then resetting them. The problem is the volume of
   data exchanged often makes not suitable the use of 32bit counters. What to do ?
A: pmacct >= 0.9.2 sports a new configure switch '--enable-64bit'; by supplying it, the source
   code will be compiled with 64bit (byte/packet/flow) counters. Otherwise usual 32bit ones are
   used. It's adviceable to choose carefully whether the use of 64bit counters is required
   mainly because of the extra memory required to store them. Usually, overflowing counters are
   recognizable because of unexpected fluctuations in the counters value - in turn, caused by
   one or more rollovers. 
 

Q15: SQL table versions, what they are -- why and when do i need them ?  
A: You need to get involved with SQL tables when you use a SQL plugin (*astonishment and
   surprise*); pmacct gets shipped with so called 'default' tables; they are built by SQL
   scripts in 'sql/' section of the distritubtion tarball. Default tables enable you to
   start quickly with pmacct. Default tables have multiple versions because new features 
   have been introduced over the time and often backward compatibility when upgrading
   pmacct is a need. 
   
   Briefly, v1, v2, v3, v4 or v5 tables ? Few rules of thumb follow:

   - Do you need packet classification ? Then you have to use v5.
   - Do you need flows (other than packets) accounting ? Then you have to use v4.
   - Do you need ToS/DSCP field (QoS) accounting ? Then you have to use v3.
   - Do you need agent ID for distributed accounting and packet tagging ? Then you have to use v2.
   - Do you need VLAN traffic accounting ? Then you have to use v2.
   - If all of the above point sound useless for you, then use v1.

   People sometimes need to customize default SQL schema for various reasons; pmacct supports
   such customizations via 'sql_optimize_clauses' configuration key. It instructs the running
   SQL plugin on how to build queries.
   So, definitely, you will need versioning only when running default tables; in such case don't
   forget to specify which SQL table version you are currently using:

   commandline:    '-v [1|2|3|4|5]'
   configuration:  'sql_table_version: [1|2|3|4|5]'


Q16: What is the best way to kill a running instance of pmacct avoiding data loss ?
A: You have two chances. a) just kill a specific plugin that you need no more: you will have
   to identify it and fire the 'kill -INT <process number> command; b) kill the whole pmacct
   instance: you can either fire the 'killall -INT <daemon name>' command or identify the Core
   Process and fire the 'kill -INT <process number> command; it will do the rest of the job
   for you: will stop receiving new data from the network, clear the memory buffers, notify
   the exit to the running plugins (which in turn will clear cached data as needed - eg. the 
   SQL plugins case) and finally will nicely take the exit lane itself. To identify the Core
   Process you can either take a look to the process list (where setproctitle() is supported
   by pmacct) or use the 'pidfile' (-F) directive. Note also that shutting down nicely the
   daemon improves restart turn-around times: the existing daemon will, first thing, close
   its listening socket while the newly launched one will take advantage of the SO_REUSEADDR
   socket option. 


Q17: I find interesting store network data in a SQL database. But i'm actually hitting poor 
   performances. Do you have any tips to improve/optimize things ?
A: Few simple hints are summed here in order to improve SQL database performances. They are
   not really tailored to a specific SQL engine (ie. just MySQL, PostgreSQL) but of generic
   use. Many thanks to Wim Kerkhoff for the many suggestions he gathered on this topic over
   the time: 

   * Keep only necessary fields in the schema, remove all unused. Use sql_optimize_clauses 
     in order to reflect this scenario.
   * Try avoid using UPDATEs, thus enabling an INSERT-only scenario. Give a closer look to
     sql_dont_try_update. Holding sql_history == sql_refresh_time will be a pre-condition. 
   * If the previous point holds, then look for and enable database-specific directives
     aimed to optimize performances ie. sql_multi_values for MySQL and sql_use_copy for
     PostgreSQL.
   * Don't rely automagically on standard indexes but enable optimal indexes based on queries
     spawned by a) pmacct daemon and b) SQL reports (ie. external tools, web pages, scripts,
     etc.) to SELECT data. Then remove each unused index as they would negatively affect
     performances.  
   * Run all SELECT and UPDATE queries under the "EXPLAIN ANALYZE ..." method to see if they
     are actually hitting the indexes. If not, you need to build indexes that better fit the
     actual scenario. See the point above.
   * Sometimes run "SET enable_seqscan=no;" before a SELECT query can make a big difference.
     Don't underestimate the importance of daily VACUUM queries: 3-5 VACUUMs + 1 VACUUM FULL
     is generally a good idea. These tips hold for PostgreSQL.
   * Disabling fsync() does improve performance. But remember this will have almost horrible
     consequences in the case of not-proper shutdowns (remember power/UPS failures ..). 


/* EOF */
