From yedeng0 at gmail.com  Fri Nov 20 07:07:33 2009
From: yedeng0 at gmail.com (Ye Deng)
Date: Thu, 19 Nov 2009 15:07:33 -0500
Subject: [netflow-tools] best linux netflow data collector?
Message-ID: <8deb0b5b0911191207m657b0ba3s74db647b1fa9ff1f@mail.gmail.com>

Hi all,

I am taking use of a Linux box with multiple NICs as a router&switch.
I may want to know what is the recommended tool to capture and record the
netflow data from the Linux box.
In my scenario, the Linux box is just connected to few hosts in a LAN,
and it does not carry heavy traffic.

I did some google searches, and found a lots of Linux netflow tools that are
not up-to-date.

Thanks a lot in advance.

Deng
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mindrot.org/pipermail/netflow-tools/attachments/20091119/2702736b/attachment.html>

From fweimer at bfk.de  Mon Nov 23 23:39:59 2009
From: fweimer at bfk.de (Florian Weimer)
Date: Mon, 23 Nov 2009 12:39:59 +0000
Subject: [netflow-tools] best linux netflow data collector?
In-Reply-To: <8deb0b5b0911191207m657b0ba3s74db647b1fa9ff1f@mail.gmail.com> (Ye
	Deng's message of "Thu\, 19 Nov 2009 15\:07\:33 -0500")
References: <8deb0b5b0911191207m657b0ba3s74db647b1fa9ff1f@mail.gmail.com>
Message-ID: <82r5rpe7og.fsf@mid.bfk.de>

* Ye Deng:

> I am taking use of a Linux box with multiple NICs as a router&switch.
> I may want to know what is the recommended tool to capture and record the
> netflow data from the Linux box.

Try nfdump.

-- 
Florian Weimer                <fweimer at bfk.de>
BFK edv-consulting GmbH       http://www.bfk.de/
Kriegsstra?e 100              tel: +49-721-96201-1
D-76133 Karlsruhe             fax: +49-721-96201-99

From mwlucas at blackhelicopters.org  Wed Nov 25 02:45:38 2009
From: mwlucas at blackhelicopters.org (Michael W. Lucas)
Date: Tue, 24 Nov 2009 10:45:38 -0500
Subject: [netflow-tools] reproducible flowd 0.9.1 crash
Message-ID: <20091124154538.GA17707@bewilderbeast.blackhelicopters.org>

Hi,

Flowd 0.9.1 consistently crashes on my system after only a few
minutes.  Platform is FreeBSD 9/i386, on VMWare, booting diskless off
of an OpenSolaris ZFS filesystem.  We're accepting v9 from an HP
Procurve switch.

I have a /var/empty/dev/log, but no messages logged from flowd.

Run in debugging mode, the program ends with:

...
98.22.63.158]:53 dst [192.221.138.129]:63705 gateway [0.0.0.0] packets 1 octets 125 in_if 29 out_if 32 sys_uptime_ms 6w4d11h43m11s.638 time_sec 2009-11-24T11:25:49 time_nanosec 0 netflow ver 9 flow_start 6w4d11h42m11s.308 flow_finish 6w4d11h42m11s.308 
output_flow_enqueue: offset 8948 alloc 16384
netflow v.9 data flowset (len 104) source 0x00000001
process_flow: ACCEPT flow FLOW recv_time 2009-11-24T10:40:01.716915 proto 17 tcpflags 00 tos 00 agent [198.22.63.129] src [192.167.90.2]:53 dst [198.22.63.130]:52715 packets 1 octets 244 in_if 32 out_if 0 sys_uptime_ms 6w4d11h43m11s.638 time_sec 2009-11-24T11:25:49 time_nanosec 0 netflow ver 9 flow_start 6w4d11h42m11s.325 flow_finish 6w4d11h42m11s.325 
output_flow_enqueue: offset 9032 alloc 16384
process_flow: ACCEPT flow FLOW recv_time 2009-11-24T10:40:01.716915 proto 17 tcpflags 00 tos 00 agent [198.22.63.129] src [192.35.51.30]:53 dst [198.22.63.130]:64923 packets 1 octets 157 in_if 32 out_if 0 sys_uptime_ms 6w4d11h43m11s.638 time_sec 2009-11-24T11:25:49 time_nanosec 0 netflow ver 9 flow_start 6w4d11h42m11s.326 flow_finish 6w4d11h42m11s.326 
output_flow_enqueue: offset 9116 alloc 16384
process_flow: ACCEPT flow FLOW recv_time 2009-11-24T10:40:01.716915 proto 17 tcpflags 00 tos 00 agent [198.22.63.129] src [192.35.51.30]:53 dst [198.22.63.130]:50591 packets 1 octets 141 in_if 32 out_if 0 sys_uptime_ms 6w4d11h43m11s.638 time_sec 2009-11-24T11:25:49 time_nanosec 0 netflow ver 9 flow_start 6w4d11h42m11s.327 flow_finish 6w4d11h42m11s.327 
output_flow_enqueue: offset 9200 alloc 16384
netflow v.9 data flowset (len 44) source 0x00000001
process_flow: ACCEPT flow FLOW recv_time 2009-11-24T10:40:01.716915 proto 17 tcpflags 00 tos 00 agent [198.22.63.129] src [198.22.63.130]:51669 dst [192.167.90.1]:53 gateway [0.0.0.0] packets 1 octets 69 in_if 29 out_if 32 sys_uptime_ms 6w4d11h43m11s.638 time_sec 2009-11-24T11:25:49 time_nanosec 0 netflow ver 9 flow_start 6w4d11h42m11s.328 flow_finish 6w4d11h42m11s.328 
output_flow_enqueue: offset 9288 alloc 16384
output_flow_flush: flushing output queue len 9288
flowd_mainloop: monitor closed
Bus error (core dumped)

Any suggestions, folks?

Thanks,
==ml

-- 
Michael W. Lucas 	mwlucas at BlackHelicopters.org
http://www.MichaelWLucas.com/
Latest book:  Cisco Routers for the Desperate, 2nd Edition
http://www.CiscoRoutersForTheDesperate.com/

From list2009 at lunch.za.net  Thu Nov 26 16:50:09 2009
From: list2009 at lunch.za.net (Andrew McGill)
Date: Thu, 26 Nov 2009 07:50:09 +0200
Subject: [netflow-tools] reproducible flowd 0.9.1 crash
In-Reply-To: <20091124154538.GA17707@bewilderbeast.blackhelicopters.org>
References: <20091124154538.GA17707@bewilderbeast.blackhelicopters.org>
Message-ID: <200911260750.09309.list2009@lunch.za.net>

On Tuesday 24 November 2009 17:45:38 Michael W. Lucas wrote:
> Hi,
> 
> Flowd 0.9.1 consistently crashes on my system after only a few
> minutes.  Platform is FreeBSD 9/i386, on VMWare, booting diskless off
> of an OpenSolaris ZFS filesystem.  We're accepting v9 from an HP
> Procurve switch.
> 
> I have a /var/empty/dev/log, but no messages logged from flowd.
> 
> Run in debugging mode, the program ends with:
...
> process_flow: ACCEPT flow FLOW recv_time 2009-11-24T10:40:01.716915 proto
>  17 tcpflags 00 tos 00 agent [198.22.63.129] src [198.22.63.130]:51669 dst
>  [192.167.90.1]:53 gateway [0.0.0.0] packets 1 octets 69 in_if 29 out_if 32
>  sys_uptime_ms 6w4d11h43m11s.638 time_sec 2009-11-24T11:25:49 time_nanosec
>  0 netflow ver 9 flow_start 6w4d11h42m11s.328 flow_finish 6w4d11h42m11s.328
>  output_flow_enqueue: offset 9288 alloc 16384
> output_flow_flush: flushing output queue len 9288
> flowd_mainloop: monitor closed
> Bus error (core dumped)
Hang, it sounds as if it crashed.  

> Any suggestions, folks?
Have a cup of tea.  You could debug the core file with gdb and get a backtrace 
(bt).  Alternatively, you could capture the netflow packets with tcpdump 
(tcpdump -s0 -w file ... and test on a more easily debugged system).

From mwlucas at blackhelicopters.org  Fri Nov 27 03:39:20 2009
From: mwlucas at blackhelicopters.org (Michael W. Lucas)
Date: Thu, 26 Nov 2009 11:39:20 -0500
Subject: [netflow-tools] reproducible flowd 0.9.1 crash
In-Reply-To: <200911260750.09309.list2009@lunch.za.net>
References: <20091124154538.GA17707@bewilderbeast.blackhelicopters.org>
	<200911260750.09309.list2009@lunch.za.net>
Message-ID: <20091126163920.GB21283@bewilderbeast.blackhelicopters.org>

On Thu, Nov 26, 2009 at 07:50:09AM +0200, Andrew McGill wrote:
> On Tuesday 24 November 2009 17:45:38 Michael W. Lucas wrote:
> > Hi,
> > 
> > Flowd 0.9.1 consistently crashes on my system after only a few
> > minutes.  Platform is FreeBSD 9/i386, on VMWare, booting diskless off
> > of an OpenSolaris ZFS filesystem.  We're accepting v9 from an HP
> > Procurve switch.
> > 
> > I have a /var/empty/dev/log, but no messages logged from flowd.
> > 
> > Run in debugging mode, the program ends with:
> ...
> > process_flow: ACCEPT flow FLOW recv_time 2009-11-24T10:40:01.716915 proto
> >  17 tcpflags 00 tos 00 agent [198.22.63.129] src [198.22.63.130]:51669 dst
> >  [192.167.90.1]:53 gateway [0.0.0.0] packets 1 octets 69 in_if 29 out_if 32
> >  sys_uptime_ms 6w4d11h43m11s.638 time_sec 2009-11-24T11:25:49 time_nanosec
> >  0 netflow ver 9 flow_start 6w4d11h42m11s.328 flow_finish 6w4d11h42m11s.328
> >  output_flow_enqueue: offset 9288 alloc 16384
> > output_flow_flush: flushing output queue len 9288
> > flowd_mainloop: monitor closed
> > Bus error (core dumped)
> Hang, it sounds as if it crashed.  
> 
> > Any suggestions, folks?
> Have a cup of tea.  You could debug the core file with gdb and get a backtrace 
> (bt).  Alternatively, you could capture the netflow packets with tcpdump 
> (tcpdump -s0 -w file ... and test on a more easily debugged system).

Unfortunately, flowd exceeds my minimal debugging abilities.  Building
a flowd with symbols and running it under gdb, I get:

netflow/usr/ports/net-mgmt/flowd/work/flowd-0.9.1;gdb ./flowd
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd"...
(gdb) run
Starting program: /usr/ports/net-mgmt/flowd/work/flowd-0.9.1/flowd 

Program exited normally.
(gdb) quit

Probably because it starts the flowd: monitor and flowd: net processes.

The crash leaves a /flowd.core file, but if I gdb that and run bt I get:

(gdb) bt
No stack.
(gdb) quit

Can anyone enlighten me as to how I should debug this?  I'm happy to
read the right documentation if someone can point me at it... The
Internet has innumerable tutorials, but most are obsolete, irrelevant,
or just plain wrong.

Thanks,
==ml

-- 
Michael W. Lucas 	mwlucas at BlackHelicopters.org
http://www.MichaelWLucas.com/
Latest book:  Cisco Routers for the Desperate, 2nd Edition
http://www.CiscoRoutersForTheDesperate.com/