NetFlow and nfsen Installation Process

Document directory

• Nfdump

Introduction NetFlow provides nearly real time traffic monitoring, smart Flow Filtration, aggregation and statistic evaluation, using source/destination IP addresses, protocols, etc. Web interface provides easy configuration of all components into NetFlow collectors, Devices, units etc. the NetFlow analyzer shoshould be able to deal with suspicous network activities (security attacks, routing troubles etc ). Nfsen with nfdump are one of the tools to monitor flows from Cisco routers. in this document, I have tried to approach the basic layout of nfsen installation. for more information regarding the installation of nfsen and nfdump, please visit the mailing list Of nfsen in sourceforge.net Dependencies and installationapache, PHP and Perl These three applications are usually installed by default. if they are not installed, then you can always install it using YaST (open SUSE) or using apt-Get (Debian or UBUNTU ). to install from YaST, go to YaST (need to be Root User)-software management And click on search. Might be you have to search each application to install. For more information visit Apache: http://httpd.apache.org/ PHP: http://www.php.net/ Perl: http://www.perl.org/ Rrdtool For more information on installing rrdtool, visit Http://oss.oetiker.ch/rrdtool/ Dependencies for rrdtoolzlib Download from Http://oss.oetiker.ch/rrdtool/pub/libs/zlib-LATESTVERSION.tar.gz more information onhttp: // www.zlib.net/ Libpng Download from Http://oss.oetiker.ch/rrdtool/pub/libs/libpng-LATESTVERSION.tar.gz more information onhttp: // www.libpng.org/pub/png/libpng.html Cairo Download from Http://oss.oetiker.ch/rrdtool/pub/libs/cairo-LATESTVERSION.tar.gz more information onhttp: // www.cairographics.org/ Glib Download from Http://oss.oetiker.ch/rrdtool/pub/libs/glib-LATESTVERSION.tar.gz more information onhttp: // www.gtk.org/ Pango Download from Http://oss.oetiker.ch/rrdtool/pub/libs/pango-latestversion.tar.gz more information onhttp: // www.pango.org/ Installation Download from: (download from Http://oss.oetiker.ch/rrdtool/pub/libs/rrdtool-LATESTVERSION.tar.gz) $ Tar zxvf rrdtool-LATESTVERSION.tar.gz $./Configure-Prefix =/usr/local/rrdtool-Disable-tcl $ Make Make install If there is error, then its much likely going to be the dependencies which are not installed. try to fix out the dependency which is/are not installed and try installing again. some of the dependencies can be installed using YaST, but its always good Install from a tar file, as you will be getting the latest version and good documentation about the dependencies. Nfdumpdownload site Download nfdump from Http://voxel.dl.sourceforge.net/sourceforge/nfdump/nfdump-1.5.7.tar.gz. also download flow-tools fromftp: // ftp.eng.oar.net/pub/flow-tools/flow-tools-0.68.tar.gz It is necessary while compiling nfdump. Compiling flow-Tools Download from Ftp://ftp.eng.oar.net/pub/flow-tools/flow-tools-0.68.tar.gz. Make sure you have TCP_WRAPPERS (ftp://ftp.porcupine.org/pub/security/tcp_wrappers_7.6.tar.gz) installed In your system before compiling. $ Tar flow-tools-0.68.tar.gz$./Configure$ Make Configure options -- Prefix =/usr/local/nfdump The directory where nfdump is going to be installed. If not given, the default will be installed in/usr/local. -- Enable-nfprofile Nfprofile is used by nfsen (frontend). By default it is not installed, so we have to enable it. -- With-rrdpath = Link Link to the rrdpath normally to the directory where RRD. h resides. normally it is on include folder of rrdpath. if you have gone through the installation technique as mentioned above, then the link wocould be/usr/local/rrdtool/include -- With-ftpath = Source We need the flow-tool library for nfdump. The flow-tool source is the directory of the flow-tool source after it is compiled. Installation Download nfdump from Http://voxel.dl.sourceforge.net/sourceforge/nfdump/nfdump-1.5.7.tar.gz $ Tar zxvf nfdump-1.5.7.tar.gz$./Configure-enable-nfprofile-with-rrdpath =/usr/local/rrdtool/include-with-ftpath = Source Path$ Make $ Make install Confirming files installed After installation make sure you have following files installed in your directory. in directory (if you have given prefix during installation, then it wocould be in/usr/local/nfdump/bin else it will be in/usr/local/bin) nfdump, nfcapd, nfexpire, nfprofile Nftrack. There wocould be other files too, but the above files are the basic files which you will need. Running nfcapd and nfdump Before Running nfdump commands, you must make sure that your router is sending the flows to your computer or not. to configure router go to section 4. I configured router to send flows on my system on port 9991 but it might be different for you based on The configuration of the router. Nfcapd Nfcapd can be run from any user; it just depends upon the output directory. make sure the output directory can be written by the user from whom you are running nfcapd. I have my output directory on/var/NetFlow/Data $ Nfcapd-z-w-T5-P 9991-L/var/NetFlow/Data Where-z => compress the files of the flows collected-W => creates new file each time after-T min-T => time interval-P => port number from where flows is collected (for me its 9991) -L => the output directory where the file is to be written There will be more options available. Just do $ Nfcapd -- Help for more options Tcpdump If you are not sure you are getting the flows on port 9991 or not, use tcpdump to check it. (Root User) Tcpdump-v udp port 9991. Point to remember: the flows collected from router will be UDP packets. Nfdump To view the captured data from nfcapd, use $ Nfdump-r/var/NetFlow/data/nfcapd. (timeslot) For more information about the options to use with nfdump you can visit Http://nfdump.sourceforge.net/page, or you can use $ Nfdump -- Help Command. Router You have to configure the Cisco router to export the flows before running the nfdump. there wocould be several documentations available for different Cisco models, so I have tried to explain the basic technique involved to configure router to export flows. Refer to Cisco router Model Description for more details. To enable NetFlow in Router Interface fastethernet 0/0 IP route-cache flow Send NetFlow data IP flow-export <ip-address> <udp-port> IP flow-export version 5 IP flow-Cache timeout active 5 Generally the timeout is given anytime between 1 and 60. But make sure that the timeout given in router is equal or less than the timeout provided in nfcapd. Usually by default both timeouts are 5 minutes. Extra configuration for busy Routers If you feel your router is handling loads of data and is relatively busy, then you can add following configuration line in your router MLS aging fast time 4 threshold 2 MLS aging normal 32 MLS aging long 900 Nfsen Nfsen is the frontend tool to visualize the flows collected using nfdump. Download Site Download from Http://internap.dl.sourceforge.net/sourceforge/nfsen/nfsen-1.3.tar.gz usually the snapshot is also available for nfsen, but it is always good way to go out with stable version. Creating user and having Apache permission A new user and group is created to allow the external commands run from the Web interface. You have to be root user to run following Commands /Usr/sbin/useradd-M nfsen Passwd nfsen /Usr/sbin/groupadd nfsen /Usr/sbin/usermod-G nfsen Create a new group and adding it to Apache user group. /Usr/sbin/groupadd nfsenadmin /Usr/sbin/usermod-a-G nfsenadmin nfsen (adding user nfsen to nfsenadmin user) /Usr/sbin/usermod-a-G nfsenadmin wwwrun (adding nfsenadmin to apache user group) Creating html dir Before we install nfsen, we need to create HTML dir. the Dir path shocould be same with the path given in nfsen. conf file. Mkdir-p/usr/local/nfsen/www/htdocs/nfsen Change the permission to nfsen: nagcmd so that you wont have to suffer permission issue later. Chown-R nfsen: nagcmd/usr/local/nfsen Configuration Basically when you are installing nfsen, You have to configure the nfsen. conf file located inside etc directory of nfsen source. $ Tar nfsen-1.3.tar.gz $ Cd nfsen-1.3/etc Edit nfsen. conf as follows: (leave other options as it is) $ BASEDIR = "/usr/local/nfsen" $ HTMLDIR = "$ {BASEDIR}/www/htdocs/nfsen" $ PREFIX = "$/usr/local/bin" (nfdump path Or can be/usr/local/nfdump/bin depending upon your installtion) $ USER = "nfsen" Make sure you create a user called nfsen and assigned the wwwgroup permission to it. Creating the user and assigning permission is explained in section 3.5.3. $ WWWUSER = "nfsen" $ WWWGROUP = "nagcmd" $ SUBDIRLAYOUT = 0 (if your router is very busy, otherwise 2 will give you lot of options to view the file) % Sources = ( 'Routername' => {'Port' => '000000', 'col' => '# 0000ff', 'type' => 'netflow '},); if you want to add more routers, then you can add more lines as above. for my configuration I used two routers, so my configuration was % Sources = ( 'Router1' => {'Port' => '000000', 'col' => '# 0000ff', 'type' => 'netflow '}, router2 '=> {'Port' => '000000', 'col' =>' # 0000ff ', 'type' => 'netflow '},); After editing abve options run ./Install. pl ETC/nfsen. conf And the nfsen will be installed on the system. Running nfsen After installing nfsen, to start nfsen, issue following command /Usr/local/nfsen/bin/nfsen start If you have changed anything on the nfsen. conf file after it has been installed always run /Usr/local/nfsen/bin/nfsen reload Always checkout nfsen errors in Syslog File. Processing Web Interface For this section You Need To Be root user. create a file nfsen. conf in directory/etc/apache2/CONF. D (path of Apache's conf. d ). add following lines in that file # vi/etc/apache2/CONF. d/nfsen. conf Lias/nfsen "/usr/local/nfsen/www/htdocs/nfsen" <Directory "/usr/local/nfsen/www/htdocs/nfsen"> Options none order allow, deny AllowOverride none Unknown end tag for </directory> Save the file and exit. Restart the webserver after that using command # service apache2 restart or #/etc/init. d/apache2 restart Correction on some files While running nfsen from Web interface for the first time, if you see the message "Error: no value for 'statvisable '! " Then you have to correct some lines in file details. php In lines 308 and 312 Line 308-If (isset ($Cookie 'statpref') {+ If (isset ($Cookie 'statpref') & $Cookie 'statpref '! = ''){ $Post 'statpref' = $Cookie 'statpref '; } Line 312 -If (isset ($Cookie 'statvisable') {+ If (isset ($Cookie 'statvisable') & $Cookie 'statvisable '! = ''){ $Post 'statvisable' = $Cookie 'statvisable '; } This error is due to some part of the Code which was developed in PhP4 and is not compatible with php5. Integration of existing data If you have existing NetFlow data, it can be added to the new nfsen installation by following steps 1. go to the profile directory (normally it is $ nfsen/profiles-data/live) 2. copy all the existing data to the folder or make sym link to the existing data 3. use command:. /usr/local/nfsen/bin/nfsen-r live 4. the Live profile is now setup with the existing data. you can always check it using. /usr/local/nfsen/bin/nfsen-l live Changing hierarchy If you check out nfsen. CONF file you can find, $ subdirlayout, which is used to change the hierarchy level of the files stored. if you want to change the way the file is stored in your system, you can change the number of this layout. the number corresponding To the hierarchy level is defined in nfsen. conf. After changing the layout number, just run the command /Usr/local/nfsen/bin/rebuildhierarchy. pl This command will change all your files into preferred layout.

Comment
Apt.hous...@gmail.com, Oct 4, 2009

Is there a package available for the same?

Comment
Jamie.ma...@gmail.com, Dec 12,200 9

Where it says:

"Chown-r nfsen: nagcmd/usr/local/nfsen"

And

"$ Wwwgroup =" nagcmd"

Shocould this be "nfsenadmin" instead of "nagcmd "?

Comment
Peterd...@gmail.com, Jan 24,201 1

The nagcmd I noticed was a group created for Nagios in another how to guide.

I am confused with this:

Shoshould Lias be alias and what is unknown end tag?

Processing web interface Logging ¶

For this section You Need To Be root user. create a file nfsen. conf in directory/etc/apache2/CONF. D (path of Apache's conf. d ). add following lines in that file # vi/etc/apache2/CONF. d/nfsen. conf

• Lias/nfsen "/usr/local/nfsen/www/htdocs/nfsen"

<Directory "/usr/local/nfsen/www/htdocs/nfsen">

Options none order allow, deny AllowOverride? None

Unknown end tag for </directory>