From djm at mindrot.org Tue Jan 2 10:23:04 2007 From: djm at mindrot.org (Damien Miller) Date: Tue, 2 Jan 2007 10:23:04 +1100 (EST) Subject: [netflow-tools] softflowd and AS numbers? In-Reply-To: <671122F84800B54B9907F3D2157D714F5EF545@srv01.basis06.com> References: <671122F84800B54B9907F3D2157D714F5EF545@srv01.basis06.com> Message-ID: On Thu, 28 Dec 2006, Reto Burkhalter wrote: > Hello everybody > > As far as I remember, the netflow protocol should support > the extraction of ASN (autonomous system numbers). If I am > right, it would be nice to integrate softflowd with openbgpd > to deliver the ASN in the exported flows. > > I could imagine some sort of lookup via bgpctl read-only socket and > cache the result inside softflowd's memory structures for further > usage. > > Could be something for the TODO list, right? Absolutely - Mathieu Sauve-Frankel is already looking at implementing it for pfflowd first, and his implementation can be adapted for softflowd. -d From sschwerdhoefer at multamedio.de Tue Jan 16 19:12:56 2007 From: sschwerdhoefer at multamedio.de (Sebastian Schwerdhoefer) Date: Tue, 16 Jan 2007 09:12:56 +0100 Subject: [netflow-tools] softflowd under FreeBSD measures too much traffic Message-ID: <20070116081256.GC23597@localdomain> Dear List, I'm trying to set up an ip gateway with FreeBSD. For traffic accounting i'd like to use softflowd with pmacct (http://www.pmacct.net/). In my tests, softflowd seems to calculate other (false) values when it's directly sniffing on an interface (executed with "-i sis1"). The curious thing is, when softflowd analyzes a tcpdumped file, the values delivered by softflowd are correct! Can anyone understand and/or explain this strange behavior? I'm using the newest softflowd version. FreeBSD patches one file, softflowd.c. I appended the diff file ("patch-softflowd.c"). Could this be the fault? Thanks in advance! Sebastian Schwerdh?fer -------------- next part -------------- A non-text attachment was scrubbed... Name: patch-softflowd.c Type: text/x-csrc Size: 398 bytes Desc: not available Url : http://lists.mindrot.org/pipermail/netflow-tools/attachments/20070116/1748ec1e/attachment.bin From serge at drozilla.ch Tue Jan 16 20:40:09 2007 From: serge at drozilla.ch (Serge Droz) Date: Tue, 16 Jan 2007 10:40:09 +0100 Subject: [netflow-tools] Duration in v9 flows Message-ID: <45AC9D79.2070507@drozilla.ch> Hello all, there seems to be a problem with version 9 flows: The duration shows up way too long. We get the correct results by applying the appended patch. Cheers Serge -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: softflowd-0.9.8.patch Url: http://lists.mindrot.org/pipermail/netflow-tools/attachments/20070116/87e53181/attachment.ksh From djm at mindrot.org Wed Jan 17 14:48:29 2007 From: djm at mindrot.org (Damien Miller) Date: Wed, 17 Jan 2007 14:48:29 +1100 (EST) Subject: [netflow-tools] softflowd under FreeBSD measures too much traffic In-Reply-To: <20070116081256.GC23597@localdomain> References: <20070116081256.GC23597@localdomain> Message-ID: On Tue, 16 Jan 2007, Sebastian Schwerdhoefer wrote: > Dear List, > > I'm trying to set up an ip gateway with FreeBSD. For traffic > accounting i'd like to use softflowd with pmacct > (http://www.pmacct.net/). In my tests, softflowd seems to calculate > other (false) values when it's directly sniffing on an interface > (executed with "-i sis1"). The curious thing is, when softflowd > analyzes a tcpdumped file, the values delivered by softflowd are > correct! > > Can anyone understand and/or explain this strange behavior? Not without a debug trace from softflowd and a better description of how you calculated a variance. > I'm using the newest softflowd version. FreeBSD patches one file, softflowd.c. > I appended the diff file ("patch-softflowd.c"). Could this be the > fault? Probably not, at least if you aren't monitoring pfflowd. -d From sschwerdhoefer at multamedio.de Wed Jan 17 20:13:35 2007 From: sschwerdhoefer at multamedio.de (Sebastian Schwerdhoefer) Date: Wed, 17 Jan 2007 10:13:35 +0100 Subject: [netflow-tools] softflowd under FreeBSD measures too much traffic In-Reply-To: References: <20070116081256.GC23597@localdomain> Message-ID: <20070117091335.GH13985@localdomain> Damien Miller wrote: > Not without a debug trace from softflowd and a better description of > how you calculated a variance. Excuse my stupid question, but I must admit that I'm not familar with debugging. Also I'm not a C programmer, so I need additional help to create a debug trace. How can I create a debug trace? The other thing however, the better explanation should work :-) I'll do a bit of ascii art to show my test setup: sis0 sis1 +--------+ +----+ +--------+ | Host 1 |----| GW |----| Host 2 | +--------+ +----+ +--------+ | +------------+ `----| Accounting | +------------+ Host 1, Host 2 and the accounting machine are in seperate networks. GW is a pc that runs as ip gateway, to connect these networks. GW also runs softflowd which sends v5 flows to "accounting". The accounting machine uses pmacct to receive the netflow datagrams, calculate the total traffic between each host and save the values in a postgres database. Sis0 is gw's nic to host 1's network, Sis1 is gw's nic to host 2's network. Sis2 (not shown) is the nic to accounting's network. I did several test, where I started softflowd, ipfm (as third-party accounting tool, to compare the results) and tcpdump at (almost) the same time on the gateway. Therefore I have the following little bash script: startup.bash ipfm -c /tmp/ipfm.conf & softflowd -D -i sis1 -v 5 -n accounting:2100 & tcpdump -w /tmp/sis1.dump -i sis1 & On Host 1 I type "scp host2:/boot/vmlinuz* .", to create some traffic that the three apps shall measure. Of course I have to stop all three processes at once. I wait a few minutes (until softflowd detected expired flows and send the netflow datagrams to accounting) and stop the measurement with: shutdown.bash killall ipfm & killall softflowd & killall tcpdump & If I compare the traffic that ipfm calculated with the values that pmacct wrote to database, pmaccts values are always a little bit (not a data bit :-)) higher than ipfm's results. I don't suspect pmacct as failure, because softflowd's debug output shows the same values that pmacct writes to database. Now the mystic part: I clear the postgres database and advise softflowd to read the tcpdumped file with: softflowd -D -r /tmp/sis1.dump -v 5 -n accounting:2100. Acracadabra, the values in the postgres database are the same as the values from ipfm! I did this test several times. The exact values do always differ, but the final result (that softflowd measures more traffic when it's directly sniffing, but exact values when it analyzes the tcpdump file) is always the same. Thank you for reading and for your help! ... and excuse me for my skrewed up english :-/ I hope you can understand me... best regards, Sebastian Schwerdh?fer From djm at mindrot.org Thu Jan 18 18:51:50 2007 From: djm at mindrot.org (Damien Miller) Date: Thu, 18 Jan 2007 18:51:50 +1100 (EST) Subject: [netflow-tools] softflowd under FreeBSD measures too much traffic In-Reply-To: <20070117091335.GH13985@localdomain> References: <20070116081256.GC23597@localdomain> <20070117091335.GH13985@localdomain> Message-ID: On Wed, 17 Jan 2007, Sebastian Schwerdhoefer wrote: > Now the mystic part: I clear the postgres database and advise > softflowd to read the tcpdumped file with: softflowd -D -r > /tmp/sis1.dump -v 5 -n accounting:2100. Acracadabra, the values in the > postgres database are the same as the values from ipfm! Could you please send me (off-list) the acutal numbers that you are seeing, a debug trace from softflowd, along with the tcpdump trace that you are capturing. To get a debug trace from softflowd, just get modify your command above to record output to a file: softflowd -D -i sis1 -v 5 -n accounting:2100 >/tmp/softflowd.debug 2>&1 -d