From mccusker at sonalysts.com Thu Aug 16 00:54:10 2007 From: mccusker at sonalysts.com (Owen McCusker) Date: Wed, 15 Aug 2007 10:54:10 -0400 Subject: [netflow-tools] Repost: not seeing flows behind cable modem Message-ID: <4C9C3298-E963-4118-9FE3-F8127FBF4CA1@sonalysts.com> Greetings, Sorry for the repost, if it made, on this subject. I am running softflowd 0.9.8 on a FreeBSD box which sits right behind a cable modem. I am not seeing flows. The IPs are handed out via DHCP, and the name is fully qualified, host.my.domain. Do I need some special configurations to receive flows in this situation? Owen From doug at nakediron.com Fri Aug 17 07:36:19 2007 From: doug at nakediron.com (Douglas Choma) Date: Thu, 16 Aug 2007 14:36:19 -0700 Subject: [netflow-tools] softflowd questions Message-ID: Sorry if this has been answered elsewhere... I didn't find an mailing list archive. I'm trying to set up my Linux "router" to monitor Internet bandwidth usage (using Netflow). But I'm a little confused on a few issues: 1) Do I only need to monitor the external interface? Will that give me data about the source IP from internal requests? Or will the NAT'd packet contain the firewall's address as the source? 2) With the external interface in promiscuous mode, won't that open up the firewall to unwanted security risks? Please forgive my ignorance on this stuff. From cristi at net.utcluj.ro Fri Aug 17 09:10:40 2007 From: cristi at net.utcluj.ro (Cristian KLEIN) Date: Fri, 17 Aug 2007 02:10:40 +0300 Subject: [netflow-tools] softflowd questions In-Reply-To: References: Message-ID: <46C4D970.9000900@net.utcluj.ro> Douglas Choma wrote: > Sorry if this has been answered elsewhere... I didn't find an mailing > list archive. > > I'm trying to set up my Linux "router" to monitor Internet bandwidth > usage (using Netflow). But I'm a little confused on a few issues: > > 1) Do I only need to monitor the external interface? Will that give > me data about the source IP from internal requests? Or will the > NAT'd packet contain the firewall's address as the source? Softflowd uses pcap to get the packets which it then converts to flows. It essecially sees the same data that tcpdump would. On all systems (including Linux), pcap sees the packet immediately before being sent on the wire, or immediately after receiving it from the wire. If you use softflowd on the externat interface of a NAT, you will see the translated IPs and not the ones of your internal hosts. There are few cases in which you can't tell softflowd to monitor the internal interface. > 2) With the external interface in promiscuous mode, won't that open > up the firewall to unwanted security risks? Short: I, personally, haven't heard of such a thing. Long: A network card usually only captures the frame whose destination MAC address is either broadcast or the burned-in MAC address of the card. Promiscuous mode changes this behaviour and tells the NIC to capture all frames even if they have a different destination MAC than the burned-in one. In nowadays networks, this makes little difference, as switches usually filter such frames (this is in fact their very purpose). Anyway, supposing you have an old hub network and one of your interfaces is in promiscuous mode. The single consequence is that a lot of silly frames won't be dropped by the NIC, but by the operating system. From doug at nakediron.com Fri Aug 17 09:25:29 2007 From: doug at nakediron.com (Douglas Choma) Date: Thu, 16 Aug 2007 16:25:29 -0700 Subject: [netflow-tools] softflowd questions In-Reply-To: <46C4D970.9000900@net.utcluj.ro> References: <46C4D970.9000900@net.utcluj.ro> Message-ID: <412EDB44-8113-48F7-9E45-322845FE988B@nakediron.com> On Aug 16, 2007, at 4:10 PM, Cristian KLEIN wrote: > Softflowd uses pcap to get the packets which it then converts to > flows. > It essecially sees the same data that tcpdump would. On all systems > (including Linux), pcap sees the packet immediately before being > sent on > the wire, or immediately after receiving it from the wire. > > If you use softflowd on the externat interface of a NAT, you will see > the translated IPs and not the ones of your internal hosts. There are > few cases in which you can't tell softflowd to monitor the internal > interface. So then it makes more sense to use softflowd on the *internal* interface, and capture the packets (flows) to/from the Internet before the addresses are NAT'd? I'm guessing there are all sorts of possible uses for softflowd, but I just wanting to figure out a "best practice". Thanks for your help. :-) From djm at mindrot.org Mon Aug 20 20:11:31 2007 From: djm at mindrot.org (Damien Miller) Date: Mon, 20 Aug 2007 20:11:31 +1000 (EST) Subject: [netflow-tools] softflowd questions In-Reply-To: References: Message-ID: On Thu, 16 Aug 2007, Douglas Choma wrote: > Sorry if this has been answered elsewhere... I didn't find an mailing > list archive. > > I'm trying to set up my Linux "router" to monitor Internet bandwidth > usage (using Netflow). But I'm a little confused on a few issues: > > 1) Do I only need to monitor the external interface? Will that give > me data about the source IP from internal requests? Or will the > NAT'd packet contain the firewall's address as the source? No - if you are perfoming NAT then you will need to monitor the internal interface. Alternately, I believe that there is a pflowd[1] equivalent for Linux that exports flows directly when NAT/conntrack states expire. This is likely to be more efficient than softflowd for your use, but unfortunately I can't remember the name of the software. > 2) With the external interface in promiscuous mode, won't that open > up the firewall to unwanted security risks? It does increase your attack surface - any software that listens to the network does. On the other hand, softflowd is pretty simple and doesn't look past the packet headers. A conntrack-based exporter does not raise your attack surface as much as it only listens to kernel messages, which are hopefully more trusted. Hope this helps. -d [1] http://www.mindrot.org/projects/pfflowd/ From djm at mindrot.org Mon Aug 20 20:12:47 2007 From: djm at mindrot.org (Damien Miller) Date: Mon, 20 Aug 2007 20:12:47 +1000 (EST) Subject: [netflow-tools] Repost: not seeing flows behind cable modem In-Reply-To: <4C9C3298-E963-4118-9FE3-F8127FBF4CA1@sonalysts.com> References: <4C9C3298-E963-4118-9FE3-F8127FBF4CA1@sonalysts.com> Message-ID: On Wed, 15 Aug 2007, Owen McCusker wrote: > Greetings, > > Sorry for the repost, if it made, on this subject. > > I am running softflowd 0.9.8 on a FreeBSD box which sits right behind > a cable modem. > I am not seeing flows. > > The IPs are handed out via DHCP, and the name is fully qualified, > host.my.domain. > > Do I need some special configurations to receive flows in this > situation? Please post the commandline options that you are using to run softflowd, and some debug output (run with -D in addition to your usual options). Thanks, Damien From ben at tilderoot.com Wed Aug 29 15:34:52 2007 From: ben at tilderoot.com (Ben Lovett) Date: Tue, 28 Aug 2007 22:34:52 -0700 Subject: [netflow-tools] Python flowd module causing SIGBUS on OpenBSD/sparc64 Message-ID: <20070829053452.GA4905@selenium.tilderoot.com> I'm trying to get some network monitoring going, and am revisiting flowd with pfflowd. To try and make things a little more interesting I'm looking at using the python interface to flowd, but am running into a rather major problem, namely SIGBUS. This is only happening for me on my sparc64, i386 causes no troubles. Below is a backtrace generated after building the module with debugging symbols. Ideas? Ben --- Python 2.4.4 (#1, Aug 9 2007, 09:29:02) [GCC 3.3.5 (propolice)] on openbsd4 Type "help", "copyright", "credits" or "license" for more information. >>> import flowd >>> flows = flowd.FlowLog("/var/log/flowd", "rb") >>> for flow in flows: ... print flow.format() ... Program received signal SIGBUS, Bus error. 0x00000000501a3f1c in object_to_u64 (o=0x4e126150, u64=0x500230cc) at flowd_python.c:162 162 if (PyErr_Occurred()) (gdb) bt #0 0x00000000501a3f1c in object_to_u64 (o=0x4e126150, u64=0x500230cc) at flowd_python.c:162 #1 0x00000000501a3f7c in flowobj_normalise (f=0x50023000) at flowd_python.c:174 #2 0x00000000501a4560 in flow_format (self=0x50023000, args=0x50023000, kw_args=0x0) at flowd_python.c:271 #3 0x00000000455f7498 in PyCFunction_Call () from /usr/local/lib/libpython2.4.so.0.0 #4 0x0000000045639520 in call_function () from /usr/local/lib/libpython2.4.so.0.0 #5 0x0000000045636938 in PyEval_EvalFrame () from /usr/local/lib/libpython2.4.so.0.0 #6 0x00000000456375cc in PyEval_EvalCodeEx () from /usr/local/lib/libpython2.4.so.0.0 #7 0x0000000045634170 in PyEval_EvalCode () from /usr/local/lib/libpython2.4.so.0.0 #8 0x000000004565c91c in run_node () from /usr/local/lib/libpython2.4.so.0.0 #9 0x000000004565b230 in PyRun_InteractiveOneFlags () from /usr/local/lib/libpython2.4.so.0.0 #10 0x000000004565af80 in PyRun_InteractiveLoopFlags () from /usr/local/lib/libpython2.4.so.0.0 #11 0x000000004565aec8 in PyRun_AnyFileExFlags () from /usr/local/lib/libpython2.4.so.0.0 #12 0x0000000045663ccc in Py_Main () from /usr/local/lib/libpython2.4.so.0.0 #13 0x0000000000101118 in ___start () #14 0x000000004f802f24 in _dl_start () from /usr/libexec/ld.so #15 0x000000004f802f24 in _dl_start () from /usr/libexec/ld.so Previous frame identical to this frame (corrupt stack?) (gdb) From djm at fuyu.mindrot.org Fri Aug 31 13:11:03 2007 From: djm at fuyu.mindrot.org (Damien Miller) Date: Fri, 31 Aug 2007 13:11:03 +1000 (EST) Subject: [netflow-tools] CVS: fuyu.mindrot.org: softflowd Message-ID: <20070831031103.CFDB33C6B3@fuyu.mindrot.org> CVSROOT: /var/cvs Module name: softflowd Changes by: djm at fuyu.mindrot.org 07/08/31 13:11:03 Modified files: . : ChangeLog softflowd.c softflowd.h Log message: - (djm) Move max_flows into struct FLOWTRACK Diff commands: cvs -nQq rdiff -u -r1.93 -r1.94 softflowd/ChangeLog cvs -nQq rdiff -u -r1.96 -r1.97 softflowd/softflowd.c cvs -nQq rdiff -u -r1.10 -r1.11 softflowd/softflowd.h CVSWeb: http://cvsweb.mindrot.org/index.cgi/softflowd/ChangeLog?r1=1.93;r2=1.94 http://cvsweb.mindrot.org/index.cgi/softflowd/softflowd.c?r1=1.96;r2=1.97 http://cvsweb.mindrot.org/index.cgi/softflowd/softflowd.h?r1=1.10;r2=1.11 Please note that there may be a delay before commits are available on the public CVSWeb site.