Nfsen Installation Process

Nfsen

Before I begin, let me remind you of the excellent documentation of nfdump and nfsen on the following pages:

• Http://nfdump.sourceforge.net/
• Http://nfsen.sourceforge.net/

The only reason I 've compiled this step-by-step guide is because I tendency to always document the stuff I do. and I haven'tyet found any online guide for nfsen which goes into such details. so if you're looking for a braindead do-this-do-that type of guide,
Then you 've come to the right place.

Contents [Hide] 1 Installation guide 2 Standalone configuration 3 PortTracker 4Add a profile from CLI

Installation Guide

This guide currently describes the procedure to get nfsen up and running on a Ubuntu server (or most likely On Debian-based systems in general ). red Hat/Fedora/centos users shoshould have no problem following the steps and make small changes to fit their system.

First off, lets get the necessary basic tools to be able to compile software on this system:

apt-get install build-essential

Install Apache (or your favorite web-server, whatever that may be. But this guide is based around Apache 2)

apt-get install apache2

A PHP interpreter is needed. It will integrate it self with Apache. I also install the command line version of PHP for practical reasons.

apt-get install php5 php5-cli

Check the version of Perl already installed on your system (it shocould be> 5)

perl --version

Nfsen requires the mail-header and mail-Internet modules for Perl. I'm going to use CPAN to get them installed. First I initialize the CPAN shell.

perl -MCPAN -eshell

Then, install the needed modules.

cpan> install Mail::Headercpan> install Mail::Internet

Some other needed packages gets installed

apt-get install libpng12-dev libfreetype6-dev libart-2.0-dev bison flex

Download rrdtool from its homepage, and compile it with the optionEnable-perl-site-install. Currently, per July 2008, rrdtool versjon 1.3 is not supported by nfsen, so you'll have to get the latest stable 1.2.x. in this Guide, we use version 1.2.27,
You may have to adjust some of the commands below to suit your system.

wget http://oss.oetiker.ch/rrdtool/pub/rrdtool.tar.gztar zxvf rrdtool.tar.gzcd rrdtool-1.2.27./configure --enable-perl-site-installmake && make install

Then we download nfdump (as nfsen, does not include it) and compile it with supportNfprofile(Which nfsen uses). Again the path to rrdtool may have to be changed.

wget http://downloads.sourceforge.net/nfdump/nfdump-1.5.7.tar.gztar zxvf nfdump-1.5.7.tar.gzcd nfdump-1.5.7./configure --enable-nfprofile --with-rrdpath=/usr/local/rrdtool-1.2.27/make && make install

Download nfsen

wget http://downloads.sourceforge.net/nfsen/nfsen-1.3.tar.gztar zxvf nfsen-1.3.tar.gzcd nfsen-1.3

Make a copy of the default configuration file, and modify it as you 'd like. You shoshould look atthis digoal to get an overview of the directory structure of nfsen
Before you make your changes to the configuration file.

cp etc/nfsen-dist.conf etc/nfsen.confvi etc/nfsen.conf

Here are my configuration changes. If you have multiple flow sources, simply add more lines to the sources-array.

$BASEDIR = "/opt/nfsen";$HTMLDIR    = "/var/www/nfsen/";$PROFILEDATADIR="/var/log/netflow";$USER    = "www-data";$WWWUSER  = "www-data";$WWWGROUP = "www-data";%sources = (   'flowsource01'    => { 'port' => '9999', 'col' => '#ff0000', 'type' => 'netflow' },);

Now it's time to run the installation script and pointing to your Config File.

./install.pl etc/nfsen.conf

For some reason the nfsen frontend does not have an index file, so lets quickly create one

echo -e "<?php\n\theader(\"Location: nfsen.php\");\n?>" > /var/www/nfsen/index.php

Start nfsen

/opt/nfsen/bin/nfsen start

And the web-based frontend to nfsen is now available at http: // <your-Server>/nfsen/

Standalone Configuration

In some situations you may be want to run nfcapd collectors and nfsen on separate machines. To be able to do this you need to make some minor configuration changesnfsen.confIn addition to some OS-level modifications. One
nfsen.confI 've previusly used is the following:

$BASEDIR = "/opt/nfsen";$HTMLDIR    = "/var/www/html/nfsen/";$PROFILEDATADIR="/var/log/nfsen/profiles-data";$PROFILESTATDIR="/var/log/nfsen/profiles-stat"$USER     = "www-data";$WWWUSER  = "www-data";$WWWGROUP = "www-data";%sources = (   'router01'  => { 'port'    => '0', 'col' => '#ff0000' },   'router02'  => { 'port'    => '0', 'col' => '#00ff00' },   'router03'  => { 'port'    => '0', 'col' => '#0000ff' },);

Notice thatPortParameter is set0(Zero). This make sure that nfsen does not automatically initiate nfcapd processes for each flow source defined in the sources-array when starting nfsen.

One the collector-Machine/var/log/netflowIs a separate disk-partition which the nfcapd processes writes flow data to. This partition is also read-only mounted on the nfsen-machine/var/log/netflowOver the network. Exactly how
You do this depends on your system and setup. there are always ways to get the partition mounted on another server; NFS, common NAS, clustered filesystem like GFS or lustre, or other solutions. in any case you need to get the partition nfcapd stores flow data
On shared with the machine nfsen is running on. Also on the nfsen-machine the separate disk-partition/var/log/nfsenIs used by nfsen to store profile and plugin data.

In such a configuration, it is also imperative that the ident-string in the flow data stored by nfcapd is identical with the configuration innfsen.conf. This means that for instance if the router01 source is defined in the nfsen Configuration
As:

'router01'  => { 'port'    => '0', 'col' => '#ff0000' }

Then the nfcapd process for this flow source needs to use the following parameter:

-I router01

In this example the ident string "router01" set by nfcapd matches the nfsen configuration. If this doesn't match, nfprofile will not be able to process the flow data.

As the nfsen configuration expects to find the flow data inLiveDirectory under$PROFILEDATADIR(Defined in
nfsen.conf), Which in this example is/var/log/nfsen/profiles-data/, We Need To symlink to the actual data which is located on the read-only mounted Partition/var/log/netflow.

# pwd/var/log/nfsen/profiles-data/live/router01# ls -ltotal 0lrwxrwxrwx 1 root root 35 2008-05-22 13:36 2008 -> /var/log/netflow/router01/2008/

Here we have created the Directoryrouter01(Which complies with the ident-string for this source) under/var/log/nfsen/profiles-data/live. This is something that has to be done for each flow-source defined%sources
Innfsen.conf.

All this combined, We're re now able to run nfsen on a completely separate server.

PortTracker

Encoded with nfsen is the plugin calledPortTracker. This plugin tracks each TCP/UDP port and provides graphical insight to the activity observed on the most active ports.

cd /usr/local/src/nfsen/contrib/PortTracker/

Editdo_compileAnd defineNFDUMP,RRDINCLUDEAndRRDLIBVariables.

NFDUMP=/usr/local/src/nfdump-1.5.7RRDINCLUDE=/usr/local/rrdtool-1.2.27/includeLIBRRD=/usr/local/rrdtool-1.2.27/lib

Currently there is a small bug which prevents compilation. Therefore add
$NFDUMP/minilzo.oTo the definitionNFDUMP_OBJIndo_compile.

NFDUMP_OBJ="$NFDUMP/util.o $NFDUMP/nftree.o $NFDUMP/grammar.o $NFDUMP/scanner.o $NFDUMP/nffile.o $NFDUMP/flist.o $NFDUMP/nf_common.o $NFDUMP/panonymizer.o $NFDUMP/rijndael.o $NFDUMP/ipconv.o $NFDUMP/minilzo.o"

Executedo_compileWhich now will create the binary 'nftrack'. Install this binary in the same directory as the rest of nfdump binaries.

./do_compilecp nftrack /usr/local/bin/

Add the path to the RRD library to the LD configuration.

echo "/usr/local/rrdtool-1.2.27/lib" > /etc/ld.so.conf.d/librrd.confldconfig

Create a directory for PortTracker to store its port data (requires approximately 10 GiB). The user$USERFrom
nfsen.confHas to have write permission to this directory. In my case this user iswww-data-

mkdir /var/log/netflow/porttrackerchown www-data:www-data /var/log/netflow/porttrackerchmod 775 /var/log/netflow/porttracker

EditPortTracker.pmAnd set the path to the directory you just created.

my $PORTSDBDIR = "/var/log/netflow/porttracker";

Copy backend and frontend files

cp PortTracker.pm /opt/nfsen/plugins/cp PortTracker.php /var/www/nfsen/plugins/

Editnfsen.confAnd add the PortTracker plugin to the live Profile

vi /opt/nfsen/etc/nfsen.conf@plugins = (        [ 'live',   'PortTracker'],); 

Start nftrack$USER

sudo -u www-data nftrack -I -d /var/log/netflow/porttracker/

Restart nfsen

/opt/nfsen/bin/nfsen reload

ChecksyslogTo see if the plugin was successfully loaded

grep -i porttracker /var/log/syslog...Apr 13 16:48:13 hw01 nfsen[31474]: Loading plugin 'PortTracker': Success ...Apr 13 16:48:13 hw01 nfsen[31474]: Initializing plugin 'PortTracker': Success...

After the next 5 minute processing interval, PortTracker will generate data and graphs in the Web-interface.

Add a profile from CLI

One can maintain nfsen from the CLI as well as from the web-interface. Here follows an example on how to add a new profile.

List the profiles we have already defined.

root@server:/opt/nfsen/bin# ./nfsen -Alivessh

Now we'll add a new profile, called "TCP" with the start time at June 12 and set to as a continuous profile (I. e by not defining an End Time)

root@server:/opt/nfsen/bin# ./nfsen -a tcp -B 2008-06-12-00-00# # name    tcpgroup   (nogroup)tcreate Thu Jun 12 21:23:30 2008tstart  Thu Jun 12 00:00:00 2008tend    Thu Jun 12 21:15:00 2008updated Wed Jun 11 23:55:00 2008expire  0 hourssize    0maxsize 0type    continuouslocked  0status  newversion 130

Then add channels (flow sources). In my example here, I have only one single channel; router01. I also add my profile filter to extract only TCP traffic.

root@server:/opt/nfsen/bin# ./nfsen --add-channel tcp/router01 filter="proto 6"...channel router01     sign: + colour: #ff0000 order: 1        sourcelist: router01Files: 0 Size: 0

The profile has statusNew, I. e. it has not yet been activated.

root@server:/opt/nfsen/bin# ./nfsen --commit-profile tcp...status  built 0

If there is a lot of data to process for this profile,Status built <%>Line will show and gradually increase. You may view a profile summary by running:

root@server:/opt/nfsen/bin# ./nfsen -l tcp# # name    tcpgroup   (nogroup)tcreate Thu Jun 12 21:23:30 2008tstart  Thu Jun 12 00:00:00 2008tend    Thu Jun 12 21:15:00 2008updated Thu Jun 12 21:15:00 2008expire  0 hourssize    70.6 KBmaxsize 0type    continuouslocked  0status  OKversion 130channel router01     sign: + colour: #ff0000 order: 1        sourcelist: router01Files: 257       Size: 1052672

In this case we see that the profile is completely built (OK) and currently is 70 KiB in size.