From djm at mindrot.org Mon Sep 3 20:49:01 2007 From: djm at mindrot.org (Damien Miller) Date: Mon, 3 Sep 2007 20:49:01 +1000 (EST) Subject: [netflow-tools] Python flowd module causing SIGBUS on OpenBSD/sparc64 In-Reply-To: <20070829053452.GA4905@selenium.tilderoot.com> References: <20070829053452.GA4905@selenium.tilderoot.com> Message-ID: On Tue, 28 Aug 2007, Ben Lovett wrote: > I'm trying to get some network monitoring going, and am revisiting > flowd with pfflowd. To try and make things a little more > interesting I'm looking at using the python interface to flowd, but > am running into a rather major problem, namely SIGBUS. > > This is only happening for me on my sparc64, i386 causes no > troubles. Below is a backtrace generated after building the module > with debugging symbols. > > Ideas? You could try running with env MALLOC_OPTIONS=AFGJP python It looks like something is hosing memory, but the crash doesn't seem to be proximal to the actual corruption. Turning on those malloc options might expose the corruption closer to where it occurs. -d From djm at fuyu.mindrot.org Mon Sep 3 20:50:05 2007 From: djm at fuyu.mindrot.org (Damien Miller) Date: Mon, 3 Sep 2007 20:50:05 +1000 (EST) Subject: [netflow-tools] CVS: fuyu.mindrot.org: softflowd Message-ID: <20070903105005.5ED473C6B3@fuyu.mindrot.org> CVSROOT: /var/cvs Module name: softflowd Changes by: djm at fuyu.mindrot.org 07/09/03 20:50:05 Modified files: . : ChangeLog Makefile.in TODO softflowd.c softflowd.h Added files: . : freelist.c freelist.h Log message: - (djm) Implement a very simple freelist allocator for flows and expiry events Diff commands: cvs -nQq rdiff -u -r1.94 -r1.95 softflowd/ChangeLog cvs -nQq rdiff -u -r1.9 -r1.10 softflowd/Makefile.in cvs -nQq rdiff -u -r1.33 -r1.34 softflowd/TODO cvs -nQq rdiff -u -r1.97 -r1.98 softflowd/softflowd.c cvs -nQq rdiff -u -r1.11 -r1.12 softflowd/softflowd.h CVSWeb: http://cvsweb.mindrot.org/index.cgi/softflowd/ChangeLog?r1=1.94;r2=1.95 http://cvsweb.mindrot.org/index.cgi/softflowd/Makefile.in?r1=1.9;r2=1.10 http://cvsweb.mindrot.org/index.cgi/softflowd/TODO?r1=1.33;r2=1.34 http://cvsweb.mindrot.org/index.cgi/softflowd/softflowd.c?r1=1.97;r2=1.98 http://cvsweb.mindrot.org/index.cgi/softflowd/softflowd.h?r1=1.11;r2=1.12 Please note that there may be a delay before commits are available on the public CVSWeb site. From ben at tilderoot.com Fri Sep 7 14:29:46 2007 From: ben at tilderoot.com (Ben Lovett) Date: Thu, 6 Sep 2007 21:29:46 -0700 Subject: [netflow-tools] Python flowd module causing SIGBUS on OpenBSD/sparc64 In-Reply-To: References: <20070829053452.GA4905@selenium.tilderoot.com> Message-ID: On Sep 3, 2007, at 3:49 AM, Damien Miller wrote: > It looks like something is hosing memory, but the crash doesn't seem > to be proximal to the actual corruption. Turning on those malloc > options > might expose the corruption closer to where it occurs. Rebuilt python with debugging symbols for good measure. It doesn't make much sense to me that it would be crashing in a return.. But I suppose that is just another indication that something is going terribly wrong. selenium:~ % env MALLOC_OPTIONS=AFGJP python2.4 Python 2.4.4 (#1, Sep 3 2007, 13:24:30) [GCC 3.3.5 (propolice)] on openbsd4 Type "help", "copyright", "credits" or "license" for more information. >>> import flowd >>> flows = flowd.FlowLog("flowd-blah.log", "rb") >>> for flow in flows: ... print flow.format() ... zsh: 19122 bus error (core dumped) env MALLOC_OPTIONS=AFGJP python2.4 selenium:~ % gdb /usr/local/bin/python2.4 GNU gdb 6.3 Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "sparc64-unknown-openbsd4.2"... (no debugging symbols found) (gdb) core-file python2.4.core Core was generated by `python2.4'. Program terminated with signal 10, Bus error. Reading symbols from /usr/local/lib/libpython2.4.so.0.0...done. Loaded symbols for /usr/local/lib/libpython2.4.so.0.0 Reading symbols from /usr/lib/libutil.so.11.0...done. Loaded symbols for /usr/lib/libutil.so.11.0 Reading symbols from /usr/lib/libstdc++.so.42.0...done. Loaded symbols for /usr/lib/libstdc++.so.42.0 Reading symbols from /usr/lib/libm.so.2.3...done. Loaded symbols for /usr/lib/libm.so.2.3 Reading symbols from /usr/lib/libpthread.so.8.0...done. Loaded symbols for /usr/lib/libpthread.so.8.0 Reading symbols from /usr/lib/libc.so.41.0...done. Loaded symbols for /usr/lib/libc.so.41.0 Reading symbols from /usr/libexec/ld.so...done. Loaded symbols for /usr/libexec/ld.so Reading symbols from /usr/local/lib/python2.4/lib-dynload/ readline.so...done. Loaded symbols for /usr/local/lib/python2.4/lib-dynload/readline.so Reading symbols from /usr/lib/libreadline.so.3.0...done. Loaded symbols for /usr/lib/libreadline.so.3.0 Reading symbols from /usr/lib/libncurses.so.10.0...done. Loaded symbols for /usr/lib/libncurses.so.10.0 Reading symbols from /usr/local/lib/python2.4/site-packages/ flowd.so...done. Loaded symbols for /usr/local/lib/python2.4/site-packages/flowd.so #0 0x00000000481c4024 in object_to_u64 (o=0x4eb44150, u64=0x4281a4cc) at flowd_python.c:159 159 return (0); (gdb) bt #0 0x00000000481c4024 in object_to_u64 (o=0x4eb44150, u64=0x4281a4cc) at flowd_python.c:159 #1 0x00000000481c409c in flowobj_normalise (f=0x4281a400) at flowd_python.c:174 #2 0x00000000481c4680 in flow_format (self=0x4281a400, args=0x4281a400, kw_args=0x0) at flowd_python.c:271 #3 0x000000004b9af5b8 in PyCFunction_Call (func=0x4eb5fbd8, arg=0x4c086050, kw=0x4281a400) at /u/obj/ports/Python-2.4.4-no_tkinter/Python-2.4.4/Objects/ methodobject.c:77 #4 0x000000004b9f1640 in call_function (pp_stack=0xffffffffffffabf8, oparg=0) at /u/obj/ports/Python-2.4.4-no_tkinter/Python-2.4.4/Python/ ceval.c:3568 #5 0x000000004b9eea58 in PyEval_EvalFrame (f=0x4c1ab420) at /u/obj/ports/Python-2.4.4-no_tkinter/Python-2.4.4/Python/ ceval.c:2167 #6 0x000000004b9ef6ec in PyEval_EvalCodeEx (co=0x4eb3fdc0, globals=0x4eb5fd8c, locals=0x4c1ab400, args=0x0, argcount=0, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at /u/obj/ports/Python-2.4.4-no_tkinter/Python-2.4.4/Python/ ceval.c:2741 #7 0x000000004b9ec290 in PyEval_EvalCode (co=0x4eb3fdc0, globals=0x4c1ab5a8, locals=0x49b5da20) at /u/obj/ports/Python-2.4.4-no_tkinter/Python-2.4.4/Python/ ceval.c:484 #8 0x000000004ba14a3c in run_node (n=0x0, filename=0x49b5da20 "", globals=0x49b5da20, locals=0x49b5da20, flags=0x49b5da20) at /u/obj/ports/Python-2.4.4-no_tkinter/Python-2.4.4/Python/ pythonrun.c:1285---Type to continue, or q to quit--- #9 0x000000004ba13350 in PyRun_InteractiveOneFlags (fp=0xffffffffffffffff, filename=0x4bb3ea60 "", flags=0xffffffffffffb22c) at /u/obj/ports/Python-2.4.4-no_tkinter/Python-2.4.4/Python/ pythonrun.c:771 #10 0x000000004ba130a0 in PyRun_InteractiveLoopFlags (fp=0x413217a0, filename=0x4bb3ea60 "", flags=0xffffffffffffb22c) at /u/obj/ports/Python-2.4.4-no_tkinter/Python-2.4.4/Python/ pythonrun.c:704 #11 0x000000004ba12fe8 in PyRun_AnyFileExFlags (fp=0x413217a0, filename=0x4bb3ea60 "", closeit=0, flags=0xffffffffffffb22c) at /u/obj/ports/Python-2.4.4-no_tkinter/Python-2.4.4/Python/ pythonrun.c:667 #12 0x000000004ba1bdec in Py_Main (argc=1, argv=0xffffffffffffb428) at /u/obj/ports/Python-2.4.4-no_tkinter/Python-2.4.4/Modules/ main.c:493 #13 0x0000000000101118 in ___start () #14 0x0000000045402f24 in _dl_start () from /usr/libexec/ld.so #15 0x0000000045402f24 in _dl_start () from /usr/libexec/ld.so Previous frame identical to this frame (corrupt stack?) From jkempf at davisvision.com Mon Sep 17 06:52:39 2007 From: jkempf at davisvision.com (Jesse Kempf) Date: Sun, 16 Sep 2007 16:52:39 -0400 Subject: [netflow-tools] Flowd-0.9 Python Library Problem Message-ID: <20070916165239.09a6cfeb.jkempf@davisvision.com> Hi, So I'm writing a small python program that massages duplicate flows out of a stream of NetFlow exports and allows one to redirect these flows to arbitrary locations. I'm using softflowd on FreeBSD to monitor several links, and export in v9 format to a different FreeBSD machine. I'm using the flowd python module to parse the netflow records. They come in on a UDP port, I pass them to flowd.Flow()...and that's where everything explodes. Softflowd is set to export v9 flows. Wireshark says these are v9 flows. flowd.Flow() explodes with: Traceback (most recent call last): File "nfagro.py", line 105, in ? main() File "nfagro.py", line 84, in main msg = NetflowRecord(msg) File "nfagro.py", line 24, in __init__ self.nf = flowd.Flow(blob=msg) ValueError: Unsupported version And this is being pulled from (msg, sndaddr) = listensocket.recvfrom(10240). When I look at the data in msg, too, the first two octets are 0x0009. So...could this be an endianness issue? Some other crazy thing? Cheers, -Jesse Kempf ------------------------------------------------------------------------ The information contained in this communication is intended only for the use of the recipient(s) named above. It may contain information that is privileged or confidential, and may be protected by State and/or Federal Regulations. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication, or any of its contents, is strictly prohibited. If you have received this communication in error, please return it to the sender immediately and delete the original message and any copy of it from your computer system. If you have any questions concerning this message, please contact the sender. ------------------------------------------------------------------------ From djm at mindrot.org Mon Sep 17 16:18:44 2007 From: djm at mindrot.org (Damien Miller) Date: Mon, 17 Sep 2007 16:18:44 +1000 (EST) Subject: [netflow-tools] Flowd-0.9 Python Library Problem In-Reply-To: <20070916165239.09a6cfeb.jkempf@davisvision.com> References: <20070916165239.09a6cfeb.jkempf@davisvision.com> Message-ID: On Sun, 16 Sep 2007, Jesse Kempf wrote: > Hi, > So I'm writing a small python program that massages duplicate flows > out of a stream of NetFlow exports and allows one to redirect these > flows to arbitrary locations. I'm using softflowd on FreeBSD to > monitor several links, and export in v9 format to a different FreeBSD > machine. I'm using the flowd python module to parse the netflow > records. They come in on a UDP port, I pass them to flowd.Flow()...and > that's where everything explodes. > > Softflowd is set to export v9 flows. > Wireshark says these are v9 flows. > flowd.Flow() explodes with: > Traceback (most recent call last): > File "nfagro.py", line 105, in ? > main() > File "nfagro.py", line 84, in main > msg = NetflowRecord(msg) > File "nfagro.py", line 24, in __init__ > self.nf = flowd.Flow(blob=msg) > ValueError: Unsupported version > > And this is being pulled from (msg, sndaddr) = > listensocket.recvfrom(10240). When I look at the data in msg, too, the > first two octets are 0x0009. So...could this be an endianness issue? > Some other crazy thing? Are you trying to parse netflow records directly with the flowd Python module? That won't work - the Python module is to read logs written by flowd. flowd writes its own NetFlow version independant log format. I agree that a making a lightweight NetFlow parser library out of flowd's guts would be a good thing though :) -d From djm at mindrot.org Mon Sep 17 16:20:17 2007 From: djm at mindrot.org (Damien Miller) Date: Mon, 17 Sep 2007 16:20:17 +1000 (EST) Subject: [netflow-tools] Python flowd module causing SIGBUS on OpenBSD/sparc64 In-Reply-To: References: <20070829053452.GA4905@selenium.tilderoot.com> Message-ID: On Thu, 6 Sep 2007, Ben Lovett wrote: > On Sep 3, 2007, at 3:49 AM, Damien Miller wrote: > > It looks like something is hosing memory, but the crash doesn't seem > > to be proximal to the actual corruption. Turning on those malloc > > options > > might expose the corruption closer to where it occurs. > > Rebuilt python with debugging symbols for good measure. > > It doesn't make much sense to me that it would be crashing in a > return.. But I suppose that is just another indication that something > is going terribly wrong. Yes, it looks like something has hosed the stack, or gdb is misbehaving. Either way, it doesn't give much to go on. > selenium:~ % env MALLOC_OPTIONS=AFGJP python2.4 > Python 2.4.4 (#1, Sep 3 2007, 13:24:30) > [GCC 3.3.5 (propolice)] on openbsd4 > Type "help", "copyright", "credits" or "license" for more information. > >>> import flowd > >>> flows = flowd.FlowLog("flowd-blah.log", "rb") > >>> for flow in flows: > ... print flow.format() Could you try printing out the flow fields manually and seeing if you can elicit a crash? -d From jkempf at davisvision.com Tue Sep 18 00:34:16 2007 From: jkempf at davisvision.com (Jesse Kempf) Date: Mon, 17 Sep 2007 10:34:16 -0400 Subject: [netflow-tools] Flowd-0.9 Python Library Problem In-Reply-To: References: <20070916165239.09a6cfeb.jkempf@davisvision.com> Message-ID: <5830CDF1-74FD-4134-B756-BDFA43432E2E@davisvision.com> On Sep 17, 2007, at 2:18 AM, Damien Miller wrote: > Are you trying to parse netflow records directly with the flowd Python > module? Well, yes. > That won't work - the Python module is to read logs written by > flowd. flowd writes its own NetFlow version independant log format. That would certainly explain why things weren't working properly. > I agree that a making a lightweight NetFlow parser library out of > flowd's > guts would be a good thing though :) I find myself wondering whether it would be better from a system building perspective to build this thing as a backend for flowd by way of the Unix domain socket export, or actually break down and make a libnetflow and then create a python wrapper around it. Cheers, -Jesse Kempf ------------------------------------------------------------------------ The information contained in this communication is intended only for the use of the recipient(s) named above. It may contain information that is privileged or confidential, and may be protected by State and/or Federal Regulations. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication, or any of its contents, is strictly prohibited. If you have received this communication in error, please return it to the sender immediately and delete the original message and any copy of it from your computer system. If you have any questions concerning this message, please contact the sender. ------------------------------------------------------------------------ From hugo.rebello at dhl.com Tue Sep 18 05:42:25 2007 From: hugo.rebello at dhl.com (Hugo Rebello) Date: Mon, 17 Sep 2007 16:42:25 -0300 Subject: [netflow-tools] Flowd Message-ID: <46EED8A1.3070902@dhl.com> Hello Guys, I have installed the Flowd and some doubts shew up. Is it possible to use the logs created by Flowd to make a web page with graphics ? How can I do that ? Can anybody send me an example of flowd.conf ? Thank you. Cheers, Hugo From jkempf at davisvision.com Tue Sep 18 07:35:43 2007 From: jkempf at davisvision.com (Jesse Kempf) Date: Mon, 17 Sep 2007 17:35:43 -0400 Subject: [netflow-tools] Flowd-0.9 Python Library Problem In-Reply-To: <5830CDF1-74FD-4134-B756-BDFA43432E2E@davisvision.com> References: <20070916165239.09a6cfeb.jkempf@davisvision.com> <5830CDF1-74FD-4134-B756-BDFA43432E2E@davisvision.com> Message-ID: <20070917173543.4a9db770.jkempf@davisvision.com> On Mon, 17 Sep 2007 10:34:16 -0400 Jesse Kempf wrote: > > I agree that a making a lightweight NetFlow parser library out of > > flowd's > > guts would be a good thing though :) > > I find myself wondering whether it would be better from a system > building perspective to build this thing as a backend for flowd by > way of the Unix domain socket export, or actually break down and make > a libnetflow and then create a python wrapper around it. Having given this more than three or four seconds of thought now, the insanity of this suggestion is obvious to me. I'm looking at extracting the individual netflow parsers from flowd.c, but it seems like each of the process_netflow_v?() functions are tightly bound to flowd's tracking of the sanity of the sending agents. It shouldn't be too unreasonable to generalize this with a struct of function pointers for handlers for each of the error conditions checked for in process_netflow_v?(). Then the caller could have his functions handled however he may please, as in the incrementing of peer->ninvalid in flowd.c, or throwing exceptions in the case of a Python library. Cheers, -Jesse Kempf ------------------------------------------------------------------------ The information contained in this communication is intended only for the use of the recipient(s) named above. It may contain information that is privileged or confidential, and may be protected by State and/or Federal Regulations. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication, or any of its contents, is strictly prohibited. If you have received this communication in error, please return it to the sender immediately and delete the original message and any copy of it from your computer system. If you have any questions concerning this message, please contact the sender. ------------------------------------------------------------------------ From mjammann at yahoo.com Wed Sep 19 04:25:02 2007 From: mjammann at yahoo.com (Matt Ammann) Date: Tue, 18 Sep 2007 11:25:02 -0700 (PDT) Subject: [netflow-tools] Analysis tools/re export flows Message-ID: <22531.41978.qm@web36505.mail.mud.yahoo.com> Hello, I've collected a passel of flow data as part of a stress test on my network. Now I need an analysis package which can make sense of the data, with a minimum of report writing and etc on my part. What analysis package is recommended which can work with flowd data? I had the idea to nfsen, but can't figure out how to get the flowd data into nfdump format. Tips would be greatly appreciated, as I need to report on this test by tomorrow. Matt ____________________________________________________________________________________ Luggage? GPS? Comic books? Check out fitting gifts for grads at Yahoo! Search http://search.yahoo.com/search?fr=oni_on_mail&p=graduation+gifts&cs=bz From mjammann at yahoo.com Wed Sep 19 04:52:19 2007 From: mjammann at yahoo.com (Matt Ammann) Date: Tue, 18 Sep 2007 11:52:19 -0700 (PDT) Subject: [netflow-tools] Analysis tools/re export flows Message-ID: <158283.12734.qm@web36501.mail.mud.yahoo.com> Hello, I've collected a passel of flow data as part of a stress test on my network. Now I need an analysis package which can make sense of the data, with a minimum of report writing and etc on my part. What analysis package is recommended which can work with flowd data? I had the idea to nfsen, but can't figure out how to get the flowd data into nfdump format. Tips would be greatly appreciated, as I need to report on this test by tomorrow. Matt ____________________________________________________________________________________ Take the Internet to Go: Yahoo!Go puts the Internet in your pocket: mail, news, photos & more. http://mobile.yahoo.com/go?refer=1GNXIC From cristi at net.utcluj.ro Wed Sep 19 06:20:30 2007 From: cristi at net.utcluj.ro (Cristian KLEIN) Date: Tue, 18 Sep 2007 23:20:30 +0300 Subject: [netflow-tools] Analysis tools/re export flows In-Reply-To: <22531.41978.qm@web36505.mail.mud.yahoo.com> References: <22531.41978.qm@web36505.mail.mud.yahoo.com> Message-ID: <46F0330E.7030007@net.utcluj.ro> Matt Ammann wrote: > Hello, > > I've collected a passel of flow data as part of a stress test on my network. > > Now I need an analysis package which can make sense of the data, with a minimum of report writing and etc on my part. > > What analysis package is recommended which can work with flowd data? I had the idea to nfsen, but can't figure out how to get the flowd data into nfdump format. > > Tips would be greatly appreciated, as I need to report on this test by tomorrow. > > Matt Have you tried flow-tools? From mjammann at yahoo.com Wed Sep 19 07:31:04 2007 From: mjammann at yahoo.com (Matt Ammann) Date: Tue, 18 Sep 2007 14:31:04 -0700 (PDT) Subject: [netflow-tools] Analysis tools/re export flows Message-ID: <466746.72452.qm@web36501.mail.mud.yahoo.com> Matt Ammann wrote: > Hello, > > I've collected a passel of flow data as part of a stress test on my network. > > Now I need an analysis package which can make sense of the data, with a minimum of report writing and etc on my part. > > What analysis package is recommended which can work with flowd data? I had the idea to nfsen, but can't figure out how to get the flowd data into nfdump format. > > Tips would be greatly appreciated, as I need to report on this test by tomorrow. > > Matt Have you tried flow-tools? Maybe I'm missing something, but it seems flow-tools won't read flowd logs directly - I need to flow-reader to ascii, then import into flow-tools us flow-import. flow-import requires csv input, and a mask specifying which fields are included in the import. So to make this work I need to 0. figure out which fields I need for flow-tools netflow V5 1. write a script to format flow-reader output to csv with right fields. 2. write a script to figure how how the "right fields" map to flow-tools field masks 3. import the csv file.. Any suggestions to make this easier? thanks, Matt ____________________________________________________________________________________ Take the Internet to Go: Yahoo!Go puts the Internet in your pocket: mail, news, photos & more. http://mobile.yahoo.com/go?refer=1GNXIC From Sameka.S.Prather at noaa.gov Thu Sep 20 04:43:35 2007 From: Sameka.S.Prather at noaa.gov (Sameka S. Prather) Date: Wed, 19 Sep 2007 14:43:35 -0400 Subject: [netflow-tools] flowd.conf Message-ID: <46F16DD7.7020406@noaa.gov> Hello Guys, I just installed flowd and would like to know if someone could provide an example of the flowd.conf file. I am having a problem with setting up a the bit counters. Can anyone help me out? -- Thank you, Sameka S. Prather Aster Engineering, Inc. Contractor 301-713-3333 x141 From djm at mindrot.org Sun Sep 23 19:46:51 2007 From: djm at mindrot.org (Damien Miller) Date: Sun, 23 Sep 2007 19:46:51 +1000 (EST) Subject: [netflow-tools] Flowd-0.9 Python Library Problem In-Reply-To: <20070917173543.4a9db770.jkempf@davisvision.com> References: <20070916165239.09a6cfeb.jkempf@davisvision.com> <5830CDF1-74FD-4134-B756-BDFA43432E2E@davisvision.com> <20070917173543.4a9db770.jkempf@davisvision.com> Message-ID: On Mon, 17 Sep 2007, Jesse Kempf wrote: > I'm looking at extracting the individual netflow parsers from flowd.c, > but it seems like each of the process_netflow_v?() functions are > tightly bound to flowd's tracking of the sanity of the sending agents. Yes, Netflow v.9 requires collectors to hold state. > It shouldn't be too unreasonable to generalize this with a struct > of function pointers for handlers for each of the error conditions > checked for in process_netflow_v?(). Then the caller could have his > functions handled however he may please, as in the incrementing of > peer->ninvalid in flowd.c, or throwing exceptions in the case of a > Python library. It might be cleaner to make these function return an integer error code and propogate these back up to process_packet() (or higher). -d From djm at mindrot.org Sun Sep 23 19:48:37 2007 From: djm at mindrot.org (Damien Miller) Date: Sun, 23 Sep 2007 19:48:37 +1000 (EST) Subject: [netflow-tools] Flowd In-Reply-To: <46EED8A1.3070902@dhl.com> References: <46EED8A1.3070902@dhl.com> Message-ID: On Mon, 17 Sep 2007, Hugo Rebello wrote: > Hello Guys, > > I have installed the Flowd and some doubts shew up. > > Is it possible to use the logs created by Flowd to make a web page with > graphics ? How can I do that ? Flowd doesn't have a out-of-the-box charting application. There is some stuff in the tools/ subdirectory, but it would probably need customisation to your particular environment. > Can anybody send me an example of flowd.conf ? Quite a few people have asked for example flowd.conf files - what is wrong with the one that ships in the distribtion? -d From kempfj2 at cs.rpi.edu Sun Sep 30 11:42:39 2007 From: kempfj2 at cs.rpi.edu (Jesse Kempf) Date: Sat, 29 Sep 2007 21:42:39 -0400 Subject: [netflow-tools] CVS Snapshot Breakage Message-ID: <6CEB7AA2-C7B2-41E5-8799-F6705CA366D9@cs.rpi.edu> Hi, At the end of the email, I've attached the full session. In short, compiling the CVS snapshot on FreeBSD 6.2/i386 fails with: waffle% make gcc -g -O2 -fPIC -c flowd.c flowd.c: In function `usage': flowd.c:1381: error: `PROGVER' undeclared (first use in this function) flowd.c:1381: error: (Each undeclared identifier is reported only once flowd.c:1381: error: for each function it appears in.) flowd.c:1386: error: `SYSCONFDIR' undeclared (first use in this function) flowd.c:1386: error: syntax error before string constant flowd.c: In function `main': flowd.c:1396: error: `SYSCONFDIR' undeclared (first use in this function) flowd.c:1396: error: syntax error before string constant *** Error code 1 Stop in /tmp/flowd/flowd. It took me a little while to figure out that this wasn't my fault (I did a little mucking around in Makefile.in, and the autotools and I have never gotten along very well). The preprocessor magic seems to be broken. I ran autoconf-2.59 and 2.61 on configure.ac, and this happens with configure scripts generated by both versions. While I'd normally have no problem going in, fixing things, and submitting a patch, I'm not sure exactly what the reason is for what's lead to the breakage. Cheers, -Jesse Kempf waffle% wget http://www.mindrot.org/flowd_snap/flowd- SNAP-20070924.tar.gz --22:44:50-- http://www.mindrot.org/flowd_snap/flowd- SNAP-20070924.tar.gz => `flowd-SNAP-20070924.tar.gz' Resolving www.mindrot.org... 203.209.195.154 Connecting to www.mindrot.org|203.209.195.154|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 184,605 (180K) [application/x-tar] 100% [====================================================================>] 184,605 93.33K/s 22:44:53 (93.11 KB/s) - `flowd-SNAP-20070924.tar.gz' saved [184605/184605] waffle% tar -xzf flowd-SNAP-20070924.tar.gz waffle% cd flowd waffle% ./configure checking build system type... i386-unknown-freebsd6.2 checking host system type... i386-unknown-freebsd6.2 checking for gcc... gcc checking for C compiler default output file name... a.out checking whether the C compiler works... yes checking whether we are cross compiling... no checking for suffix of executables... checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking for a BSD-compatible install... /usr/bin/install -c checking for ranlib... ranlib checking for special C compiler options needed for large files... no checking for _FILE_OFFSET_BITS value needed for large files... no checking how to run the C preprocessor... gcc -E checking for grep that handles long lines and -e... /usr/bin/grep checking for egrep... /usr/bin/grep -E checking for ANSI C header files... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking whether byte ordering is bigendian... no checking for yacc... /usr/bin/yacc checking for byacc... /usr/bin/byacc checking dirent.h usability... yes checking dirent.h presence... yes checking for dirent.h... yes checking sys/ndir.h usability... no checking sys/ndir.h presence... no checking for sys/ndir.h... no checking sys/dir.h usability... yes checking sys/dir.h presence... yes checking for sys/dir.h... yes checking ndir.h usability... no checking ndir.h presence... no checking for ndir.h... no checking sys/pstat.h usability... no checking sys/pstat.h presence... no checking for sys/pstat.h... no checking endian.h usability... no checking endian.h presence... no checking for endian.h... no checking sys/cdefs.h usability... yes checking sys/cdefs.h presence... yes checking for sys/cdefs.h... yes checking paths.h usability... yes checking paths.h presence... yes checking for paths.h... yes checking for strings.h... (cached) yes checking sys/time.h usability... yes checking sys/time.h presence... yes checking for sys/time.h... yes checking time.h usability... yes checking time.h presence... yes checking for time.h... yes checking for struct sockaddr.sa_len... yes checking for library containing daemon... none required checking for library containing socket... none required checking for closefrom... no checking for betoh64... no checking for htobe64... no checking for daemon... yes checking for setresuid... yes checking for setreuid... yes checking for setresgid... yes checking for setregid... yes checking for sysconf... yes checking for setproctitle... yes checking for dirfd... no checking for sendmsg... yes checking for recvmsg... yes checking for tzset... yes checking for strlcpy... yes checking for strlcat... yes checking for u_int64_t... yes checking for int64_t... yes checking for uint64_t... yes checking for u_int32_t... yes checking for int32_t... yes checking for uint32_t... yes checking for u_int16_t... yes checking for int16_t... yes checking for uint16_t... yes checking for u_int8_t... yes checking for int8_t... yes checking for uint8_t... yes checking for u_int... yes checking for char... yes checking size of char... 1 checking for unsigned char... yes checking size of unsigned char... 1 checking for short int... yes checking size of short int... 2 checking for unsigned short int... yes checking size of unsigned short int... 2 checking for int... yes checking size of int... 4 checking for unsigned int... yes checking size of unsigned int... 4 checking for long int... yes checking size of long int... 4 checking for unsigned long int... yes checking size of unsigned long int... 4 checking for long long int... yes checking size of long long int... 8 checking for unsigned long long int... yes checking size of unsigned long long int... 8 checking for sa_family_t... yes checking size of sa_family_t... 1 checking for /proc/pid/fd directory... no checking for msg_accrights field in struct msghdr... no checking for msg_control field in struct msghdr... yes checking if libc defines __progname... yes configure: creating ./config.status config.status: creating Makefile config.status: WARNING: Makefile.in seems to ignore the -- datarootdir setting config.status: creating flowd.8 config.status: creating flowd-reader.8 config.status: creating flowd.conf.5 config.status: creating flowd-pytypes.h config.status: creating flowd-config.h waffle% make gcc -g -O2 -fPIC -c flowd.c flowd.c: In function `usage': flowd.c:1381: error: `PROGVER' undeclared (first use in this function) flowd.c:1381: error: (Each undeclared identifier is reported only once flowd.c:1381: error: for each function it appears in.) flowd.c:1386: error: `SYSCONFDIR' undeclared (first use in this function) flowd.c:1386: error: syntax error before string constant flowd.c: In function `main': flowd.c:1396: error: `SYSCONFDIR' undeclared (first use in this function) flowd.c:1396: error: syntax error before string constant *** Error code 1 From kempfj2 at cs.rpi.edu Sun Sep 30 11:44:39 2007 From: kempfj2 at cs.rpi.edu (Jesse Kempf) Date: Sat, 29 Sep 2007 21:44:39 -0400 Subject: [netflow-tools] Netflow Library Message-ID: <4B99C25B-3EAE-42EA-8C21-9A7FD143E1C7@cs.rpi.edu> Hi, So far I've moved all the parsers into their own file, and have wired up flowd, at least, to use the library. Since I fell asleep trying to read the IPFIX RFC, how much work would need to be done to support IPFIX itself in the parser library? It might not be a bad idea to combine the parser library from flowd with the encoding library for softflowd to provide an alternative to libfixbuf or whatever CMU is calling their IPFIX library. Cheers, -Jesse Kempf