From smajko at wp.pl Fri May 1 00:38:33 2009 From: smajko at wp.pl (Sebastian Majkowski) Date: Thu, 30 Apr 2009 16:38:33 +0200 Subject: [netflow-tools] flows timers control Message-ID: <49F9B7E9.8040400@wp.pl> Hi I use softflowd simultaneously with my router. I noticed that softflowd is generating almost twice more softflow info than my cisco. I guess thats because of timouts which I am able to set on cisco and use some cache before sending netflow information to my collector. Is it possible to set it also with softflowd? In other words: - generate netflow information when flow is not active for X seconds (I want to set X value, what is softflowd default?) (cisco: |ip flow-cache timeout inactive X)|** - generate netflow information when flow is active for Y seconds (and if still active repeat this after another Y seconds (what is the default?)) (cisco: *ip flow-cache active-timeout Y) I guess that softflowd timers are different than cisco ones, I probably get more detailed info with softflowd but I am running out of resources and just need to save on this. Any ideas? Regards S.M * From xela at mailinglist.at Fri May 1 03:15:44 2009 From: xela at mailinglist.at (alex k) Date: Thu, 30 Apr 2009 19:15:44 +0200 (CEST) Subject: [netflow-tools] flows timers control In-Reply-To: <49F9B7E9.8040400@wp.pl> References: <49F9B7E9.8040400@wp.pl> Message-ID: <91527fd9db6fcfcf06921381348aab8a.squirrel@www.mailinglist.at> > Hi > > I use softflowd simultaneously with my router. I noticed that softflowd > is generating almost twice more softflow info than my cisco. I guess > thats because of timouts which I am able to set on cisco and use some > cache before sending netflow information to my collector. > Is it possible to set it also with softflowd? In other words: > - generate netflow information when flow is not active for X seconds (I > want to set X value, what is softflowd default?) (cisco: |ip flow-cache > timeout inactive X)|** > - generate netflow information when flow is active for Y seconds (and if > still active repeat this after another Y seconds (what is the default?)) > (cisco: *ip flow-cache active-timeout Y) > I guess that softflowd timers are different than cisco ones, I probably > get more detailed info with softflowd but I am running out of resources > and just need to save on this. > > Any ideas? > > Regards > > S.M > * > _______________________________________________ > netflow-tools mailing list > netflow-tools at mindrot.org > https://lists.mindrot.org/mailman/listinfo/netflow-tools > Hi S., I'm not sure, if I understand your problem as I don't have experience with cisco routers. Running out of resources probably means memory (or cpu?). Softflowd has several timeouts. See "man softflowd", section "Timeouts". The one you are searching for might be "maxlife". Try something like "-t maxlife=5m". Then all flows will be expired after 5 minutes (sent to collector). Alternatively you could reduce the maximum number of flows to concurrently track with "-m" or the track_level with for instance "-T proto" (less detailed, but less resources needed). Hope this helps. xela From smajko at wp.pl Mon May 4 19:24:47 2009 From: smajko at wp.pl (Sebastian Majkowski) Date: Mon, 04 May 2009 11:24:47 +0200 Subject: [netflow-tools] flows timers control In-Reply-To: <91527fd9db6fcfcf06921381348aab8a.squirrel@www.mailinglist.at> References: <49F9B7E9.8040400@wp.pl> <91527fd9db6fcfcf06921381348aab8a.squirrel@www.mailinglist.at> Message-ID: <49FEB45F.5040009@wp.pl> alex k wrote: >> Hi >> >> I use softflowd simultaneously with my router. I noticed that softflowd >> is generating almost twice more softflow info than my cisco. I guess >> thats because of timouts which I am able to set on cisco and use some >> cache before sending netflow information to my collector. >> Is it possible to set it also with softflowd? In other words: >> - generate netflow information when flow is not active for X seconds (I >> want to set X value, what is softflowd default?) (cisco: |ip flow-cache >> timeout inactive X)|** >> - generate netflow information when flow is active for Y seconds (and if >> still active repeat this after another Y seconds (what is the default?)) >> (cisco: *ip flow-cache active-timeout Y) >> I guess that softflowd timers are different than cisco ones, I probably >> get more detailed info with softflowd but I am running out of resources >> and just need to save on this. >> >> Any ideas? >> >> Regards >> >> S.M >> * >> _______________________________________________ >> netflow-tools mailing list >> netflow-tools at mindrot.org >> https://lists.mindrot.org/mailman/listinfo/netflow-tools >> >> > > Hi S., > > I'm not sure, if I understand your problem as I don't have experience with > cisco routers. Running out of resources probably means memory (or cpu?). > Softflowd has several timeouts. See "man softflowd", section "Timeouts". > The one you are searching for might be "maxlife". > Try something like "-t maxlife=5m". Then all flows will be expired after 5 > minutes (sent to collector). > Alternatively you could reduce the maximum number of flows to concurrently > track with "-m" or the track_level with for instance "-T proto" (less > detailed, but less resources needed). > > Hope this helps. > > xela > > > > > Hi Alex, Thanks for this info. It looks that it works fine now, maxlife is what I needed (and expint). Files at collector seems to be similar in size. But I still have some performance issues. My server is dedicated only for softflowd, and I process around 300Mbits/s My CPu is AMD Opteron with 2 cores but only one core is used for softflowd process. My system is netflow:~# uname -a Linux netflow 2.6.26-2-amd64 #1 SMP Fri Mar 27 04:02:59 UTC 2009 x86_64 GNU/Linux Is it possible to use both cores?? Maybe different OS can do this? btw, I cant use -T proto because I need full netflow info (TCP/UDP ports also) Take a look at my top chart: Tasks: 51 total, 2 running, 49 sleeping, 0 stopped, 0 zombie Cpu0 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Cpu1 : 71.9%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 13.0%hi, 15.1%si, 0.0%st Mem: 4064312k total, 1020408k used, 3043904k free, 106800k buffers Swap: 9928128k total, 0k used, 9928128k free, 808820k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 25147 nobody 20 0 30036 16m 2748 R 100 0.4 19:56.48 softflowd 1 root 20 0 10316 756 628 S 0 0.0 0:02.08 init 2 root 15 -5 0 0 0 S 0 0.0 0:00.00 kthreadd 3 root RT -5 0 0 0 S 0 0.0 0:00.00 migration/0 4 root 15 -5 0 0 0 S 0 0.0 0:00.02 ksoftirqd/0 5 root RT -5 0 0 0 S 0 0.0 0:00.08 watchdog/0 6 root RT -5 0 0 0 S 0 0.0 0:00.08 migration/1 7 root 15 -5 0 0 0 S 0 0.0 0:14.76 ksoftirqd/1 8 root RT -5 0 0 0 S 0 0.0 0:00.16 watchdog/1 9 root 15 -5 0 0 0 S 0 0.0 0:05.56 events/0 10 root 15 -5 0 0 0 S 0 0.0 0:09.06 events/1 11 root 15 -5 0 0 0 S 0 0.0 0:00.00 khelper 46 root 15 -5 0 0 0 S 0 0.0 0:00.02 kblockd/0 47 root 15 -5 0 0 0 S 0 0.0 0:00.28 kblockd/1 49 root 15 -5 0 0 0 S 0 0.0 0:00.00 kacpid 50 root 15 -5 0 0 0 S 0 0.0 0:00.00 kacpi_notify 146 root 15 -5 0 0 0 S 0 0.0 0:00.04 ksuspend_usbd 152 root 15 -5 0 0 0 S 0 0.0 0:00.00 khubd 155 root 15 -5 0 0 0 S 0 0.0 0:00.00 kseriod 203 root 20 0 0 0 0 S 0 0.0 0:00.00 pdflush 204 root 20 0 0 0 0 S 0 0.0 0:02.52 pdflush 205 root 15 -5 0 0 0 S 0 0.0 0:00.00 kswapd0 206 root 15 -5 0 0 0 S 0 0.0 0:00.00 aio/0 207 root 15 -5 0 0 0 S 0 0.0 0:00.22 aio/1 743 root 15 -5 0 0 0 S 0 0.0 0:00.00 ata/0 744 root 15 -5 0 0 0 S 0 0.0 0:00.00 ata/1 745 root 15 -5 0 0 0 S 0 0.0 0:00.00 ata_aux 812 root 15 -5 0 0 0 S 0 0.0 0:00.00 scsi_eh_0 813 root 15 -5 0 0 0 S 0 0.0 0:00.00 scsi_eh_1 933 root 15 -5 0 0 0 S 0 0.0 0:09.20 kjournald 1008 root 16 -4 16512 756 488 S 0 0.0 0:00.12 udevd 1863 daemon 20 0 8024 520 404 S 0 0.0 0:00.00 portmap 1874 statd 20 0 10152 760 636 S 0 0.0 0:00.00 rpc.statd 2097 root 20 0 180m 1728 1032 S 0 0.0 0:39.62 rsyslogd 2111 root 20 0 3808 624 500 S 0 0.0 0:00.00 acpid 2121 messageb 20 0 21096 536 344 S 0 0.0 0:00.00 dbus-daemon 2137 root 20 0 48872 1180 676 S 0 0.0 0:02.60 sshd Regards Sebastian