[netflow-tools] Softflowd and rrd running on the same machine

[Bogdan Ghita]

Hello everybody

I've installed a few days ago softflowd and rrd-associated tools
(flow-capture/flowscan) on a monitoring machine connected to a local
network. First, I'll start with the praise - it's a great software, I've
been looking for a long time at netflow-related software, but couldn't
find anything that would allow me to implement it via a monitoring
machine. Everything seems to be working reasonably well at the moment,
but I still have a couple of problems that I can't find the solution
for:

- out-of-order packets. In order to get softflowd and rrd to work, I'm
sending packets via the local interface of the machine. I thought this
would work just fine but flow-capture reports continually 'lost'
packets: 

Apr 29 11:27:18 linux flow-capture[23080]: ftpdu_seq_check():
src_ip=xxx.xxx.xxx.xxx dst_ip=xxx.xxx.xxx.xxx d_version=5
expecting=28199800 received=28199770 lost=4294967265
Apr 29 11:27:18 linux flow-capture[23080]: ftpdu_seq_check():
src_ip=xxx.xxx.xxx.xxx dst_ip=xxx.xxx.xxx.xxx d_version=5
expecting=28199800 received=28199770 lost=4294967265
Apr 29 11:27:18 linux flow-capture[23080]: ftpdu_seq_check():
src_ip=xxx.xxx.xxx.xxx dst_ip=xxx.xxx.xxx.xxx d_version=5
expecting=28199790 received=28199934 lost=144
Apr 29 11:27:18 linux flow-capture[23080]: ftpdu_seq_check():
src_ip=xxx.xxx.xxx.xxx dst_ip=xxx.xxx.xxx.xxx d_version=5
expecting=28199935 received=28199936 lost=1
Apr 29 11:27:19 linux flow-capture[23080]: ftpdu_seq_check():
src_ip=xxx.xxx.xxx.xxx dst_ip=xxx.xxx.xxx.xxx d_version=5
expecting=28199969 received=28199940 lost=4294967266
Apr 29 11:27:19 linux last message repeated 2 times
Apr 29 11:27:19 linux flow-capture[23080]: ftpdu_seq_check():
src_ip=xxx.xxx.xxx.xxx dst_ip=xxx.xxx.xxx.xxx d_version=5
expecting=28199944 received=28200056 lost=112
Apr 29 11:27:20 linux flow-capture[23080]: ftpdu_seq_check():
src_ip=xxx.xxx.xxx.xxx dst_ip=xxx.xxx.xxx.xxx d_version=5
expecting=28200093 received=28200064 lost=4294967266
Apr 29 11:27:20 linux last message repeated 2 times
Apr 29 11:27:20 linux flow-capture[23080]: ftpdu_seq_check():
src_ip=xxx.xxx.xxx.xxx dst_ip=xxx.xxx.xxx.xxx d_version=5
expecting=28200068 received=28200174 lost=106

Is it possible (I've don very little in terms of socket programming, so
the answer might be straightforward) to pipe softflowd straight into
flow-capture, so that the communication will be (hopefully) smoother?

- CPU usage - softflowd seems to behave very strange when changing the
priority - with the default priority (0), it uses continuously about 80%
of the processor (yes, it is a big network and, again, yes, it is a slow
machine - P3-900MHz); when I renice it (via 'top') to lower priority
(e.g. 10), the utilisation drops to about 20%, although the processor
is, overall, only about 50% used. Is the collector dropping packets/flow
during that time? Is the machine that slow?
  
- Reports - I am not sure whether this is a softflowd problem or an
rrd-related problem. I've noticed a continuous udp flow running over the
network, quite considerable in terms of bandwidth. However, when drawing
the graphs, there is only an hourly spike, and nothing else. What could
be causing this type of reporting?

Thank you very much for your time.

Regards
Bogdan