From mwlucas at blackhelicopters.org Sun May 1 03:13:01 2005 From: mwlucas at blackhelicopters.org (Michael W. Lucas) Date: Sat, 30 Apr 2005 13:13:01 -0400 Subject: [netflow-tools] problems with pfflowd that don't happen with softflowd In-Reply-To: <427381DA.3000706@mindrot.org> References: <20050430114744.GC60645@bewilderbeast.blackhelicopters.org> <427372E6.2090005@mindrot.org> <20050430121557.GD60645@bewilderbeast.blackhelicopters.org> <427381DA.3000706@mindrot.org> Message-ID: <20050430171301.GA62209@bewilderbeast.blackhelicopters.org> On Sat, Apr 30, 2005 at 11:02:18PM +1000, Damien Miller wrote: > Michael W. Lucas wrote: > >>That is possible: softflowd's timeouts are pretty conservative, > >>especially for TCP - 30 minutes post-FIN. You can tune these on the > >>commandline though :) > > > > > >Ah, you learn something every day. > > > >That 30 minutes can be important when you want to, say, grovel through > >netflow data looking for port scans and worms and whatnot. :-) > > gah, it it not 30 minutes, it is really 5 minutes post-FIN. I probably > started out writing 300 seconds, but botched changing the units. My > apologies. OK, no sweat. That's better to work with, anyway. I will do the below during my next maintenance window (tonight). > > >>When you run tcpdump, did you try the "-T cnfp" to get it to parse the > >>NetFlow packets? What collector are you using? > > > >I'm using flow-tools' flow-capture. > > > >Under softflowd, -T cnfp on the collector shows: > > > >08:08:01.118622 IP a.b.c.d.52011 > w.x.y.z.9318: NetFlow v5, 179.007 > >uptime, 1114862881.120821000, #0, 29 recs > >08:08:01.118770 IP a.b.c.d.52011 > w.x.y.z.9318: NetFlow v5, 179.007 > >uptime, 1114862881.120821000, #0, 30 recs > >08:08:01.118918 IP a.b.c.d.52011 > w.x.y.z.9318: NetFlow v5, 179.007 > >uptime, 1114862881.120821000, #0, 29 recs > >08:08:01.118923 IP a.b.c.d.52011 > w.x.y.z.9318: NetFlow v5, 179.007 > >uptime, 1114862881.120821000, #0, 29 recs > > OK, that looks like a softflowd bug there - it is not updating the > sequence number correctly. I can see one bug there (fix attached), but > I don't see how it could give rise to all the sequence numbers being > zero like this... > > >with pfflowd, we see: > > > >08:10:20.540514 IP a.b.c.d.57523 > w.x.y.z.9318: NetFlow v5, 8.741 uptime, > >1114863020.542900000, #0, 12 recs > >08:10:20.540519 IP a.b.c.d.57523 > w.x.y.z.9318: NetFlow v5, 8.741 uptime, > >1114863020.542953000, #12, 12 recs > >08:10:20.540662 IP a.b.c.d.57523 > w.x.y.z.9318: NetFlow v5, 8.741 uptime, > >1114863020.542971000, #24, 12 recs > >08:10:20.540668 IP a.b.c.d.57523 > w.x.y.z.9318: NetFlow v5, 8.741 uptime, > >1114863020.542987000, #36, 11 recs > > > >The only difference I see is that with softflowd, the # doesn't > >increment. > > There might be more to it. Could you try capturing with > "tcpdump -s1500 -vvvTcnfp"? It will show some more details. > > >I'm willing to accept that there's something in flow-capture that's > >choking on something with pfflowd. Still, it seems that there will be > >more people on this list using flow-capture than there will be on the > >flow-capture list using pfflowd and softflowd. :-) > > > >I'm also using the same flow-capture binary to collect netflow from a > >Cisco router, so I somehow think that there's some detail with pfflowd > >that I just don't have right. > > pfflowd doesn't fill in all the NetFlow fields - it just doesn't get > enough information for the pfsync interface to do it. IIRC some programs > may not like getting a 0 interface index. > > You could try the second attached patch, which is a very crude hack that > might fool it (just hardcode a couple of interface numbers). > > -d > Index: netflow1.c > =================================================================== > RCS file: /var/cvs/softflowd/netflow1.c,v > retrieving revision 1.1 > diff -u -p -r1.1 netflow1.c > --- netflow1.c 13 Sep 2004 02:25:09 -0000 1.1 > +++ netflow1.c 30 Apr 2005 12:54:20 -0000 > @@ -65,7 +65,7 @@ struct NF1_FLOW { > */ > int > send_netflow_v1(struct FLOW **flows, int num_flows, int nfsock, > - u_int64_t flows_exported, struct timeval *system_boot_time, > + u_int64_t *flows_exported, struct timeval *system_boot_time, > int verbose_flag) > { > struct timeval now; > @@ -90,6 +90,7 @@ send_netflow_v1(struct FLOW **flows, int > &err, &errsz); /* Clear ICMP errors */ > if (send(nfsock, packet, (size_t)offset, 0) == -1) > return (-1); > + *flows_exported += j; > j = 0; > num_packets++; > } > @@ -162,5 +163,6 @@ send_netflow_v1(struct FLOW **flows, int > num_packets++; > } > > + *flows_exported += j; > return (num_packets); > } > Index: netflow5.c > =================================================================== > RCS file: /var/cvs/softflowd/netflow5.c,v > retrieving revision 1.2 > diff -u -p -r1.2 netflow5.c > --- netflow5.c 29 Sep 2004 03:57:10 -0000 1.2 > +++ netflow5.c 30 Apr 2005 12:54:20 -0000 > @@ -63,7 +63,7 @@ struct NF5_FLOW { > */ > int > send_netflow_v5(struct FLOW **flows, int num_flows, int nfsock, > - u_int64_t flows_exported, struct timeval *system_boot_time, > + u_int64_t *flows_exported, struct timeval *system_boot_time, > int verbose_flag) > { > struct timeval now; > @@ -78,7 +78,7 @@ send_netflow_v5(struct FLOW **flows, int > uptime_ms = timeval_sub_ms(&now, system_boot_time); > > hdr = (struct NF5_HEADER *)packet; > - for(num_packets = offset = j = i = 0; i < num_flows; i++) { > + for (num_packets = offset = j = i = 0; i < num_flows; i++) { > if (j >= NF5_MAXFLOWS - 1) { > if (verbose_flag) > logit(LOG_DEBUG, "Sending flow packet len = %d", offset); > @@ -88,6 +88,7 @@ send_netflow_v5(struct FLOW **flows, int > &err, &errsz); /* Clear ICMP errors */ > if (send(nfsock, packet, (size_t)offset, 0) == -1) > return (-1); > + *flows_exported += j; > j = 0; > num_packets++; > } > @@ -98,7 +99,7 @@ send_netflow_v5(struct FLOW **flows, int > hdr->uptime_ms = htonl(uptime_ms); > hdr->time_sec = htonl(now.tv_sec); > hdr->time_nanosec = htonl(now.tv_usec * 1000); > - hdr->flow_sequence = htonl(flows_exported); > + hdr->flow_sequence = htonl(*flows_exported); > /* Other fields are left zero */ > offset = sizeof(*hdr); > } > @@ -163,6 +164,7 @@ send_netflow_v5(struct FLOW **flows, int > num_packets++; > } > > + *flows_exported += j; > return (num_packets); > } > > Index: netflow9.c > =================================================================== > RCS file: /var/cvs/softflowd/netflow9.c,v > retrieving revision 1.6 > diff -u -p -r1.6 netflow9.c > --- netflow9.c 10 Jan 2005 01:02:34 -0000 1.6 > +++ netflow9.c 30 Apr 2005 12:54:20 -0000 > @@ -248,7 +248,7 @@ nf_flow_to_flowset(const struct FLOW *fl > */ > int > send_netflow_v9(struct FLOW **flows, int num_flows, int nfsock, > - u_int64_t flows_exported, struct timeval *system_boot_time, > + u_int64_t *flows_exported, struct timeval *system_boot_time, > int verbose_flag) > { > struct NF9_HEADER *nf9; > @@ -275,7 +275,7 @@ send_netflow_v9(struct FLOW **flows, int > nf9->flows = 0; /* Filled as we go, htons at end */ > nf9->uptime_ms = htonl(timeval_sub_ms(&now, system_boot_time)); > nf9->time_sec = htonl(time(NULL)); > - nf9->package_sequence = htonl(flows_exported); > + nf9->package_sequence = htonl(*flows_exported + j); > nf9->source_id = 0; > offset = sizeof(*nf9); > > @@ -365,5 +365,6 @@ send_netflow_v9(struct FLOW **flows, int > j += i; > } > > + *flows_exported += j; > return (num_packets); > } > Index: softflowd.c > =================================================================== > RCS file: /var/cvs/softflowd/softflowd.c,v > retrieving revision 1.80 > diff -u -p -r1.80 softflowd.c > --- softflowd.c 10 Jan 2005 01:50:07 -0000 1.80 > +++ softflowd.c 30 Apr 2005 12:54:20 -0000 > @@ -92,7 +92,7 @@ static const struct DATALINK lt[] = { > }; > > /* Netflow send functions */ > -typedef int (netflow_send_func_t)(struct FLOW **, int, int, u_int64_t, > +typedef int (netflow_send_func_t)(struct FLOW **, int, int, u_int64_t *, > struct timeval *, int); > struct NETFLOW_SENDER { > int version; > @@ -774,12 +774,11 @@ check_expired(struct FLOWTRACK *ft, stru > if (num_expired > 0) { > if (target != NULL && target->fd != -1) { > r = target->dialect->func(expired_flows, num_expired, > - target->fd, ft->flows_exported, > + target->fd, &ft->flows_exported, > &ft->system_boot_time, verbose_flag); > if (verbose_flag) > logit(LOG_DEBUG, "sent %d netflow packets", r); > if (r > 0) { > - ft->flows_exported += num_expired * 2; > ft->packets_sent += r; > /* XXX what if r < num_expired * 2 ? */ > } else { > Index: softflowd.h > =================================================================== > RCS file: /var/cvs/softflowd/softflowd.h,v > retrieving revision 1.7 > diff -u -p -r1.7 softflowd.h > --- softflowd.h 30 Sep 2004 04:12:36 -0000 1.7 > +++ softflowd.h 30 Apr 2005 12:54:20 -0000 > @@ -189,13 +189,13 @@ u_int32_t timeval_sub_ms(const struct ti > > /* Prototypes for functions to send NetFlow packets, from netflow*.c */ > int send_netflow_v1(struct FLOW **flows, int num_flows, int nfsock, > - u_int64_t flows_exported, struct timeval *system_boot_time, > + u_int64_t *flows_exported, struct timeval *system_boot_time, > int verbose_flag); > int send_netflow_v5(struct FLOW **flows, int num_flows, int nfsock, > - u_int64_t flows_exported, struct timeval *system_boot_time, > + u_int64_t *flows_exported, struct timeval *system_boot_time, > int verbose_flag); > int send_netflow_v9(struct FLOW **flows, int num_flows, int nfsock, > - u_int64_t flows_exported, struct timeval *system_boot_time, > + u_int64_t *flows_exported, struct timeval *system_boot_time, > int verbose_flag); > > #endif /* _SOFTFLOWD_H */ > ? pfflowd > Index: pfflowd.c > =================================================================== > RCS file: /var/cvs/pfflowd/pfflowd.c,v > retrieving revision 1.18 > diff -u -p -r1.18 pfflowd.c > --- pfflowd.c 6 Sep 2004 12:25:57 -0000 1.18 > +++ pfflowd.c 30 Apr 2005 13:00:41 -0000 > @@ -319,6 +319,9 @@ send_netflow_v1(const struct _PFSYNC_STA > flw->flow_finish = htonl(uptime_ms); > flw->protocol = st[i].proto; > flw->tcp_flags = 0; > + /* XXX: hack */ > + flw->if_index_in = htons(1); > + flw->if_index_out = htons(2); > offset += sizeof(*flw); > j++; > hdr->flows++; > @@ -335,6 +338,9 @@ send_netflow_v1(const struct _PFSYNC_STA > flw->flow_finish = htonl(uptime_ms); > flw->protocol = st[i].proto; > flw->tcp_flags = 0; > + /* XXX: hack */ > + flw->if_index_in = htons(2); > + flw->if_index_out = htons(1); > offset += sizeof(*flw); > j++; > hdr->flows++; > @@ -491,6 +497,9 @@ send_netflow_v5(const struct _PFSYNC_STA > flw->flow_finish = htonl(uptime_ms); > flw->tcp_flags = 0; > flw->protocol = st[i].proto; > + /* XXX: hack */ > + flw->if_index_in = htons(1); > + flw->if_index_out = htons(2); > offset += sizeof(*flw); > j++; > hdr->flows++; > @@ -507,6 +516,9 @@ send_netflow_v5(const struct _PFSYNC_STA > flw->flow_finish = htonl(uptime_ms); > flw->tcp_flags = 0; > flw->protocol = st[i].proto; > + /* XXX: hack */ > + flw->if_index_in = htons(2); > + flw->if_index_out = htons(1); > offset += sizeof(*flw); > j++; > hdr->flows++; -- Michael W. Lucas mwlucas at FreeBSD.org, mwlucas at BlackHelicopters.org http://www.BlackHelicopters.org/~mwlucas/ Latest book: Cisco Routers for the Desperate http://www.CiscoRoutersForTheDesperate.com From mwlucas at blackhelicopters.org Sun May 1 22:46:40 2005 From: mwlucas at blackhelicopters.org (Michael W. Lucas) Date: Sun, 1 May 2005 08:46:40 -0400 Subject: [netflow-tools] problems with pfflowd that don't happen with softflowd In-Reply-To: <427381DA.3000706@mindrot.org> References: <20050430114744.GC60645@bewilderbeast.blackhelicopters.org> <427372E6.2090005@mindrot.org> <20050430121557.GD60645@bewilderbeast.blackhelicopters.org> <427381DA.3000706@mindrot.org> Message-ID: <20050501124640.GA66898@bewilderbeast.blackhelicopters.org> On Sat, Apr 30, 2005 at 11:02:18PM +1000, Damien Miller wrote: > Michael W. Lucas wrote: ... > > There might be more to it. Could you try capturing with > "tcpdump -s1500 -vvvTcnfp"? It will show some more details. with pfflowd and the patch, no records appear. Here's the tcpdump: #tcpdump -n -i bge0 -s1500 -vvvTcnfp | grep a.b.c.d tcpdump: listening on bge0, link-type EN10MB (Ethernet), capture size 1500 bytes 08:25:26.482635 IP (tos 0x0, ttl 63, id 12722, offset 0, flags [none], length: 628) a.b.c.d.56113 > w.x.y.z.port: NetFlow v5, 122.768 uptime, 1114950326.485754000, #761, 12 recs 08:25:26.482642 IP (tos 0x0, ttl 63, id 12723, offset 0, flags [none], length: 628) a.b.c.d.56113 > w.x.y.z.port: NetFlow v5, 122.768 uptime, 1114950326.485780000, #773, 12 recs 08:25:26.482783 IP (tos 0x0, ttl 63, id 12724, offset 0, flags [none], length: 628) a.b.c.d.56113 > w.x.y.z.port: NetFlow v5, 122.768 uptime, 1114950326.485793000, #785, 12 recs 08:25:26.482789 IP (tos 0x0, ttl 63, id 12725, offset 0, flags [none], length: 628) a.b.c.d.56113 > w.x.y.z.port: NetFlow v5, 122.768 uptime, 1114950326.485805000, #797, 12 recs 08:25:26.482929 IP (tos 0x0, ttl 63, id 12726, offset 0, flags [none], length: 484) a.b.c.d.56113 > w.x.y.z.port: NetFlow v5, 122.768 uptime, 1114950326.485815000, #809, 9 recs 08:25:32.833883 IP (tos 0x0, ttl 63, id 12741, offset 0, flags [none], length: 628) a.b.c.d.56113 > w.x.y.z.port: NetFlow v5, 129.119 uptime, 1114950332.837027000, #818, 12 recs 08:25:32.834031 IP (tos 0x0, ttl 63, id 12742, offset 0, flags [none], length: 628) a.b.c.d.56113 > w.x.y.z.port: NetFlow v5, 129.119 uptime, 1114950332.837054000, #830, 12 recs 08:25:32.834036 IP (tos 0x0, ttl 63, id 12743, offset 0, flags [none], length: 628) a.b.c.d.56113 > w.x.y.z.port: NetFlow v5, 129.119 uptime, 1114950332.837067000, #842, 12 recs 08:25:32.834178 IP (tos 0x0, ttl 63, id 12744, offset 0, flags [none], length: 628) a.b.c.d.56113 > w.x.y.z.port: NetFlow v5, 129.119 uptime, 1114950332.837079000, #854, 12 recs 08:25:38.378356 IP (tos 0x0, ttl 63, id 12755, offset 0, flags [none], length: 340) a.b.c.d.56113 > w.x.y.z.port: NetFlow v5, 134.663 uptime, 1114950338.381608000, #866, 6 recs 08:25:42.921488 IP (tos 0x0, ttl 63, id 12766, offset 0, flags [none], length: 580) a.b.c.d.56113 > w.x.y.z.port: NetFlow v5, 139.206 uptime, 1114950342.924569000, #872, 11 recs 08:25:42.921494 IP (tos 0x0, ttl 63, id 12767, offset 0, flags [none], length: 628) a.b.c.d.56113 > w.x.y.z.port: NetFlow v5, 139.206 uptime, 1114950342.924595000, #883, 12 recs 08:25:42.921634 IP (tos 0x0, ttl 63, id 12768, offset 0, flags [none], length: 436) a.b.c.d.56113 > w.x.y.z.port: NetFlow v5, 139.206 uptime, 1114950342.924607000, #895, 8 recs With softflowd and the patch, it appears that flow-capture stops receiving records. I started the tcpdump on the collector and ran "softflowctl expire-all" on the shaper, then checked the flow-capture file contents: #flowdumper tmp-v05.2005-05-01.083501-0400 | wc -l 0 # Mind you, once I do a "softflowctl shutdown" and restart with the unpatched softflowd binary, I abruptly see: #flowdumper tmp-v05.2005-05-01.083501-0400 | wc -l 26151 # So, could the flows just be resent upon shutdown? Or perhaps flow-capture somehow cached them before writing them to disk. :-( I can rerun the test early tomorrow morning. tcpdump: listening on bge0, link-type EN10MB (Ethernet), capture size 1500 bytes 08:38:01.723537 IP (tos 0x0, ttl 63, id 16481, offset 0, flags [none], length: 1492) a.b.c.d.51473 > w.x.y.z.port: NetFlow v5, 230.801 uptime, 1114951081.726391000, #1538, 30 recs 08:38:01.723685 IP (tos 0x0, ttl 63, id 16482, offset 0, flags [none], length: 1492) a.b.c.d.51473 > w.x.y.z.port: NetFlow v5, 230.801 uptime, 1114951081.726391000, #1568, 30 recs 08:38:01.723831 IP (tos 0x0, ttl 63, id 16483, offset 0, flags [none], length: 1444) a.b.c.d.51473 > w.x.y.z.port: NetFlow v5, 230.801 uptime, 1114951081.726391000, #1598, 29 recs 08:38:01.723838 IP (tos 0x0, ttl 63, id 16484, offset 0, flags [none], length: 1444) a.b.c.d.51473 > w.x.y.z.port: NetFlow v5, 230.801 uptime, 1114951081.726391000, #1627, 29 recs 08:38:01.723980 IP (tos 0x0, ttl 63, id 16485, offset 0, flags [none], length: 1492) a.b.c.d.51473 > w.x.y.z.port: NetFlow v5, 230.801 uptime, 1114951081.726391000, #1656, 30 recs 08:38:01.724126 IP (tos 0x0, ttl 63, id 16486, offset 0, flags [none], length: 1444) a.b.c.d.51473 > w.x.y.z.port: NetFlow v5, 230.801 uptime, 1114951081.726391000, #1686, 29 recs 08:38:01.724274 IP (tos 0x0, ttl 63, id 16487, offset 0, flags [none], length: 1444) a.b.c.d.51473 > w.x.y.z.port: NetFlow v5, 230.801 uptime, 1114951081.726391000, #1715, 29 recs 08:38:01.724421 IP (tos 0x0, ttl 63, id 16488, offset 0, flags [none], length: 1492) a.b.c.d.51473 > w.x.y.z.port: NetFlow v5, 230.801 uptime, 1114951081.726391000, #1744, 30 recs 08:38:01.724568 IP (tos 0x0, ttl 63, id 16489, offset 0, flags [none], length: 1492) a.b.c.d.51473 > w.x.y.z.port: NetFlow v5, 230.801 uptime, 1114951081.726391000, #1774, 30 recs 08:38:01.724575 IP (tos 0x0, ttl 63, id 16490, offset 0, flags [none], length: 1444) a.b.c.d.51473 > w.x.y.z.port: NetFlow v5, 230.801 uptime, 1114951081.726391000, #1804, 29 recs 08:38:01.724715 IP (tos 0x0, ttl 63, id 16491, offset 0, flags [none], length: 1492) a.b.c.d.51473 > w.x.y.z.port: NetFlow v5, 230.801 uptime, 1114951081.726391000, #1833, 30 recs 08:38:01.724862 IP (tos 0x0, ttl 63, id 16492, offset 0, flags [none], length: 1492) a.b.c.d.51473 > w.x.y.z.port: NetFlow v5, 230.801 uptime, 1114951081.726391000, #1863, 30 recs 08:38:01.725010 IP (tos 0x0, ttl 63, id 16493, offset 0, flags [none], length: 1492) a.b.c.d.51473 > w.x.y.z.port: NetFlow v5, 230.801 uptime, 1114951081.726391000, #1893, 30 recs 08:38:01.725156 IP (tos 0x0, ttl 63, id 16494, offset 0, flags [none], length: 1444) a.b.c.d.51473 > w.x.y.z.port: NetFlow v5, 230.801 uptime, 1114951081.726391000, #1923, 29 recs 08:38:01.725163 IP (tos 0x0, ttl 63, id 16495, offset 0, flags [none], length: 1492) a.b.c.d.51473 > w.x.y.z.port: NetFlow v5, 230.801 uptime, 1114951081.726391000, #1952, 30 recs 08:38:01.725304 IP (tos 0x0, ttl 63, id 16496, offset 0, flags [none], length: 1492) a.b.c.d.51473 > w.x.y.z.port: NetFlow v5, 230.801 uptime, 1114951081.726391000, #1982, 30 recs 08:38:01.725452 IP (tos 0x0, ttl 63, id 16497, offset 0, flags [none], length: 772) a.b.c.d.51473 > w.x.y.z.port: NetFlow v5, 230.801 uptime, 1114951081.726391000, #2012, 15 recs Thanks! ==ml -- Michael W. Lucas mwlucas at FreeBSD.org, mwlucas at BlackHelicopters.org http://www.BlackHelicopters.org/~mwlucas/ Latest book: Cisco Routers for the Desperate http://www.CiscoRoutersForTheDesperate.com From mwlucas at blackhelicopters.org Mon May 2 22:43:43 2005 From: mwlucas at blackhelicopters.org (Michael W. Lucas) Date: Mon, 2 May 2005 08:43:43 -0400 Subject: [netflow-tools] softflowd timeouts Message-ID: <20050502124343.GA76097@bewilderbeast.blackhelicopters.org> I'm looking to adjust the timeouts of softflowd so that I can get "closer to real-time" detection of port scans, etc. 99.99% of my flows on this web server farm are short-lived, so it appears that the TCP timeout of 3600s is a little high. What sort of negative effects could I expect if I set the TCP timeout to, say, 300s? Surely something drove setting the TCP timeout to 1 hour? ==ml -- Michael W. Lucas mwlucas at FreeBSD.org, mwlucas at BlackHelicopters.org http://www.BlackHelicopters.org/~mwlucas/ Latest book: Cisco Routers for the Desperate http://www.CiscoRoutersForTheDesperate.com From djm at mindrot.org Mon May 2 22:55:14 2005 From: djm at mindrot.org (Damien Miller) Date: Mon, 02 May 2005 22:55:14 +1000 Subject: [netflow-tools] softflowd timeouts In-Reply-To: <20050502124343.GA76097@bewilderbeast.blackhelicopters.org> References: <20050502124343.GA76097@bewilderbeast.blackhelicopters.org> Message-ID: <42762332.6090605@mindrot.org> Michael W. Lucas wrote: > I'm looking to adjust the timeouts of softflowd so that I can get > "closer to real-time" detection of port scans, etc. 99.99% of my > flows on this web server farm are short-lived, so it appears that the > TCP timeout of 3600s is a little high. > > What sort of negative effects could I expect if I set the TCP timeout > to, say, 300s? Surely something drove setting the TCP timeout to 1 > hour? The 1 hour timeout is for established TCP connections and should be long so it doesn't time out quiescent sessions (e.g. long lived FTP or ssh sessions) For portscan detection, you should probably adjust the TCP FIN and RST timeouts. I should add a timeout for "unanswered" connections, which would be useful for hosts that are packet filtered - this is already in the TODO. -d From mwlucas at blackhelicopters.org Mon May 2 22:59:42 2005 From: mwlucas at blackhelicopters.org (Michael W. Lucas) Date: Mon, 2 May 2005 08:59:42 -0400 Subject: [netflow-tools] softflowd timeouts In-Reply-To: <20050502124343.GA76097@bewilderbeast.blackhelicopters.org> References: <20050502124343.GA76097@bewilderbeast.blackhelicopters.org> Message-ID: <20050502125942.GA76220@bewilderbeast.blackhelicopters.org> On Mon, May 02, 2005 at 08:43:43AM -0400, Michael W. Lucas wrote: > I'm looking to adjust the timeouts of softflowd so that I can get > "closer to real-time" detection of port scans, etc. 99.99% of my > flows on this web server farm are short-lived, so it appears that the > TCP timeout of 3600s is a little high. > > What sort of negative effects could I expect if I set the TCP timeout > to, say, 300s? Surely something drove setting the TCP timeout to 1 > hour? And sorry to follow up on myself: Damien said the timeouts are 30min, but on a default install on FreeBSD I see: # softflowctl timeouts softflowd[57604]: Printing timeouts: TCP timeout: 3600s TCP post-RST timeout: 120s TCP post-FIN timeout: 300s UDP timeout: 300s ICMP timeout: 300s General timeout: 3600s Maximum lifetime: 604800s Expiry interval: 60s -- Michael W. Lucas mwlucas at FreeBSD.org, mwlucas at BlackHelicopters.org http://www.BlackHelicopters.org/~mwlucas/ Latest book: Cisco Routers for the Desperate http://www.CiscoRoutersForTheDesperate.com From b.ghita at jack.see.plymouth.ac.uk Tue May 3 01:02:20 2005 From: b.ghita at jack.see.plymouth.ac.uk (Bogdan Ghita) Date: Mon, 2 May 2005 16:02:20 +0100 Subject: [netflow-tools] Softflowd and rrd running on the same machine In-Reply-To: <4272BF35.7060902@mindrot.org> Message-ID: <007701c54f27$f06a3be0$8360a38d@tweak> Hello Damien Thank you very much for your response. Although I've still got a few unclear issues, at least now I cleared some of them and I've got plenty to try for the next few days:). See a few more comments below. Best regards Bogdan > -----Original Message----- > From: Damien Miller [mailto:djm at mindrot.org] > Sent: 30 April 2005 00:12 > To: Bogdan Ghita > Cc: netflow-tools at mindrot.org > Subject: Re: [netflow-tools] Softflowd and rrd running on the same machine > > Bogdan Ghita wrote: > > Hello everybody > > > > I've installed a few days ago softflowd and rrd-associated tools > > (flow-capture/flowscan) on a monitoring machine connected to a local > > network. First, I'll start with the praise - it's a great software, I've > > been looking for a long time at netflow-related software, but couldn't > > find anything that would allow me to implement it via a monitoring > > machine. Everything seems to be working reasonably well at the moment, > > but I still have a couple of problems that I can't find the solution > > for: > > > > - out-of-order packets. In order to get softflowd and rrd to work, I'm > > sending packets via the local interface of the machine. I thought this > > would work just fine but flow-capture reports continually 'lost' > > packets: > > > > Apr 29 11:27:18 linux flow-capture[23080]: ftpdu_seq_check(): > > src_ip=xxx.xxx.xxx.xxx dst_ip=xxx.xxx.xxx.xxx d_version=5 > > expecting=28199800 received=28199770 lost=4294967265 > > Apr 29 11:27:18 linux flow-capture[23080]: ftpdu_seq_check(): > > src_ip=xxx.xxx.xxx.xxx dst_ip=xxx.xxx.xxx.xxx d_version=5 > > expecting=28199800 received=28199770 lost=4294967265 > > This might be a bug in the sequence number generation of softflowd, or a > bug in flow-capture - flow-capture is certainly printing negative > sequence number offsets incorrectly. > > Can you capture some softflowd output packages with another netflow > capable tool to manually check the sequence numbers? E.g. tcpdump's > cnfp mode, flowd, etc. Thank you for the tcdpdump tip - although I'm using it quite often, I wasn't aware that it can decode Netflow packets. I've looked at it and I got the following trace: 15:00:51.234336 IP (tos 0x0, ttl 64, id 37603, offset 0, flags [DF], length: 1444) tester.32782 > tester.2000: NetFlow v5, 261557.484 uptime, 1115042451.234285000, #92222382, 29 recs 15:00:51.235204 IP (tos 0x0, ttl 64, id 37604, offset 0, flags [DF], length: 1444) tester.32782 > tester.2000: NetFlow v5, 261557.484 uptime, 1115042451.234285000, #92222382, 29 recs [lines deleted...] 15:00:51.277266 IP (tos 0x0, ttl 64, id 37656, offset 0, flags [DF], length: 1492) tester.32782 > tester.2000: NetFlow v5, 261557.484 uptime, 1115042451.234285000, #92222382, 30 recs 15:00:51.277933 IP (tos 0x0, ttl 64, id 37657, offset 0, flags [DF], length: 1492) tester.32782 > tester.2000: NetFlow v5, 261557.484 uptime, 1115042451.234285000, #92222382, 30 recs 15:01:01.242400 IP (tos 0x0, ttl 64, id 37718, offset 0, flags [DF], length: 1444) tester.32782 > tester.2000: NetFlow v5, 261567.493 uptime, 1115042461.242348000, #92227192, 29 recs 15:01:01.243251 IP (tos 0x0, ttl 64, id 37719, offset 0, flags [DF], length: 1444) tester.32782 > tester.2000: NetFlow v5, 261567.493 uptime, 1115042461.242348000, #92227192, 29 recs [lines deleted...] 15:01:01.322057 IP (tos 0x0, ttl 64, id 37835, offset 0, flags [DF], length: 1492) tester.32782 > tester.2000: NetFlow v5, 261567.493 uptime, 1115042461.242348000, #92227192, 30 recs 15:01:01.322110 IP (tos 0x0, ttl 64, id 37836, offset 0, flags [DF], length: 1444) tester.32782 > tester.2000: NetFlow v5, 261567.493 uptime, 1115042461.242348000, #92227192, 29 recs 15:01:01.322157 IP (tos 0x0, ttl 64, id 37837, offset 0, flags [DF], length: 772) tester.32782 > tester.2000: NetFlow v5, 261567.493 uptime, 1115042461.242348000, #92227192, 15 recs 15:01:11.230365 IP (tos 0x0, ttl 64, id 37838, offset 0, flags [DF], length: 1444) tester.32782 > tester.2000: NetFlow v5, 261577.480 uptime, 1115042471.230315000, #92231968, 29 recs 15:01:11.231270 IP (tos 0x0, ttl 64, id 37839, offset 0, flags [DF], length: 1444) tester.32782 > tester.2000: NetFlow v5, 261577.480 uptime, 1115042471.230315000, #92231968, 29 recs 15:01:11.232049 IP (tos 0x0, ttl 64, id 37840, offset 0, flags [DF], length: 1444) tester.32782 > tester.2000: NetFlow v5, 261577.480 uptime, 1115042471.230315000, #92231968, 29 recs [lines deleted...] 15:01:11.276308 IP (tos 0x0, ttl 64, id 37952, offset 0, flags [DF], length: 1444) tester.32782 > tester.2000: NetFlow v5, 261577.480 uptime, 1115042471.230315000, #92231968, 29 recs 15:01:11.276335 IP (tos 0x0, ttl 64, id 37953, offset 0, flags [DF], length: 1492) tester.32782 > tester.2000: NetFlow v5, 261577.480 uptime, 1115042471.230315000, #92231968, 30 recs 15:01:11.276353 IP (tos 0x0, ttl 64, id 37954, offset 0, flags [DF], length: 340) tester.32782 > tester.2000: NetFlow v5, 261577.480 uptime, 1115042471.230315000, #92231968, 6 recs 15:01:21.225781 IP (tos 0x0, ttl 64, id 37955, offset 0, flags [DF], length: 1444) tester.32782 > tester.2000: NetFlow v5, 261587.476 uptime, 1115042481.225734000, #92236610, 29 recs 15:01:21.226594 IP (tos 0x0, ttl 64, id 37956, offset 0, flags [DF], length: 1444) tester.32782 > tester.2000: NetFlow v5, 261587.476 uptime, 1115042481.225734000, #92236610, 29 recs 15:01:21.227292 IP (tos 0x0, ttl 64, id 37957, offset 0, flags [DF], length: 1444) tester.32782 > tester.2000: NetFlow v5, 261587.476 uptime, 1115042481.225734000, #92236610, 29 recs I've looked at print-cnfp.c within tcpdump and the # values are the sequence numbers within Netflow. I am not sure why they remain the same between datagrams (are they supposed to? the specification says that they should be increased with the corresponding number of flows every time a new datagram arrives), but one thing is sure - there is no gap in the id numbers, so no datagrams are lost in the process. Related to this, I've changed the expiry interval to 10 seconds, hoping to get more granularity (and less loss, due to reduced bursts) - has anybody obtained better/worse results while varying the expiry interval? > > > Is it possible (I've don very little in terms of socket programming, so > > the answer might be straightforward) to pipe softflowd straight into > > flow-capture, so that the communication will be (hopefully) smoother? > > It is highly unlikely that the packets are acually getting dropped. It > is more likely a bug in one/both packages. Judging by the trace above (no gaps in the id numbers), I am more tempted to believe there is (at least) some sort of misinterpretation, if not a bug. > > > - CPU usage - softflowd seems to behave very strange when changing the > > priority - with the default priority (0), it uses continuously about 80% > > of the processor (yes, it is a big network and, again, yes, it is a slow > > machine - P3-900MHz); when I renice it (via 'top') to lower priority > > (e.g. 10), the utilisation drops to about 20%, although the processor > > is, overall, only about 50% used. Is the collector dropping packets/flow > > during that time? Is the machine that slow? > > It is difficult to tell whether or not you are dropping packets - it > depend a lot on what happens on the kernel side too. I'm not sure what > information on packet drops libpcap makes available, but the attached > patch adds printing of the statistics that it does provide to those > printable using "softflowctl statistics". > > I'm not sure what a "dropped packet" in libpcap's parlance is - it may > be a drop because the client couldn't keep up, or a drop from a bpf > filter program. It should be obvious once you start playing with it > though :) Thank you very much for the patch. I will apply it on later today and come back with the details. > > > - Reports - I am not sure whether this is a softflowd problem or an > > rrd-related problem. I've noticed a continuous udp flow running over the > > network, quite considerable in terms of bandwidth. However, when drawing > > the graphs, there is only an hourly spike, and nothing else. What could > > be causing this type of reporting? > > That depends on how the data is being presented - if you are showing > flow records vs time, then a long UDP conversation may be represented by > only a single flow record. Retrospectively producing throughput vs time > representations of long-lived flow conversations is a little tricky and > requires some suppositions, as the flow record summarises away any > dynamics in the conversation. > I got this one, understood. > -d From djm at mindrot.org Tue May 3 10:03:17 2005 From: djm at mindrot.org (Damien Miller) Date: Tue, 03 May 2005 10:03:17 +1000 Subject: [netflow-tools] Softflowd and rrd running on the same machine In-Reply-To: <007701c54f27$f06a3be0$8360a38d@tweak> References: <007701c54f27$f06a3be0$8360a38d@tweak> Message-ID: <4276BFC5.2010209@mindrot.org> Bogdan Ghita wrote: > 15:01:21.226594 IP (tos 0x0, ttl 64, id 37956, offset 0, flags [DF], > length: 1444) tester.32782 > tester.2000: NetFlow v5, 261587.476 uptime, > 1115042481.225734000, #92236610, 29 recs > 15:01:21.227292 IP (tos 0x0, ttl 64, id 37957, offset 0, flags [DF], > length: 1444) tester.32782 > tester.2000: NetFlow v5, 261587.476 uptime, > 1115042481.225734000, #92236610, 29 recs > > I've looked at print-cnfp.c within tcpdump and the # values are the > sequence numbers within Netflow. I am not sure why they remain the same > between datagrams (are they supposed to? the specification says that > they should be increased with the corresponding number of flows every > time a new datagram arrives), but one thing is sure - there is no gap in > the id numbers, so no datagrams are lost in the process. Related to > this, I've changed the expiry interval to 10 seconds, hoping to get more > granularity (and less loss, due to reduced bursts) - has anybody > obtained better/worse results while varying the expiry interval? This is a bug in softflowd. The fix is attached :) Please test and let me know whether it helps. -d -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: softflowd-seqno.diff Url: http://lists.mindrot.org/pipermail/netflow-tools/attachments/20050503/450c924f/attachment.ksh From djm at mindrot.org Tue May 3 11:53:36 2005 From: djm at mindrot.org (Damien Miller) Date: Tue, 03 May 2005 11:53:36 +1000 Subject: [netflow-tools] problems with pfflowd that don't happen with softflowd In-Reply-To: <20050501124640.GA66898@bewilderbeast.blackhelicopters.org> References: <20050430114744.GC60645@bewilderbeast.blackhelicopters.org> <427372E6.2090005@mindrot.org> <20050430121557.GD60645@bewilderbeast.blackhelicopters.org> <427381DA.3000706@mindrot.org> <20050501124640.GA66898@bewilderbeast.blackhelicopters.org> Message-ID: <4276D9A0.4010003@mindrot.org> Michael W. Lucas wrote: > with pfflowd and the patch, no records appear. Here's the tcpdump: OK, so it is not the interface index. I'll have to install flow-tools to see what the problem is. > With softflowd and the patch, it appears that flow-capture stops > receiving records. I started the tcpdump on the collector and ran > "softflowctl expire-all" on the shaper, then checked the flow-capture > file contents: > > #flowdumper tmp-v05.2005-05-01.083501-0400 | wc -l > 0 > # > > Mind you, once I do a "softflowctl shutdown" and restart with the > unpatched softflowd binary, I abruptly see: > > #flowdumper tmp-v05.2005-05-01.083501-0400 | wc -l > 26151 > # > > So, could the flows just be resent upon shutdown? Or perhaps > flow-capture somehow cached them before writing them to disk. :-( That is most likely: softflowd sends immediately on expiry - it doesn't queue export packets. > I can rerun the test early tomorrow morning. > > tcpdump: listening on bge0, link-type EN10MB (Ethernet), capture size 1500 bytes > 08:38:01.723537 IP (tos 0x0, ttl 63, id 16481, offset 0, flags [none], length: 1492) a.b.c.d.51473 > w.x.y.z.port: NetFlow v5, 230.801 uptime, 1114951081.726391000, #1538, 30 recs > 08:38:01.723685 IP (tos 0x0, ttl 63, id 16482, offset 0, flags [none], length: 1492) a.b.c.d.51473 > w.x.y.z.port: NetFlow v5, 230.801 uptime, 1114951081.726391000, #1568, 30 recs The sequence numbers look OK at least :) -d From b.ghita at jack.see.plymouth.ac.uk Wed May 4 01:25:11 2005 From: b.ghita at jack.see.plymouth.ac.uk (Bogdan Ghita) Date: Tue, 3 May 2005 16:25:11 +0100 Subject: [netflow-tools] Pcap dropping packets In-Reply-To: <4272BF35.7060902@mindrot.org> Message-ID: <00bb01c54ff4$4c049370$8360a38d@tweak> Hello again I'm aware that the subject is rather off-topic for the list, but it seems that either nobody has had problems before with it or it's my machine that really needs upgrading. I've installed the statistics patch from Damien and now I get the full picture - "pcap packets dropped" are about 10% of "pcap packets received". An example below: --- Number of active flows: 550792 Packets processed: 468154791 Fragments: 1823 Ignored packets: 40871 (40871 non-IP, 0 too short) Flows expired: 7131866 (0 forced) Flows exported: 10501294 in 358453 packets (0 failures) pcap packets received: 3767680 pcap packets dropped: 373100 pcap packets dropped by interface: 0 --- (the percentage does vary considerably, the number of lost packets is not precisely 10%) which, according to my understanding :), means that my network card driver is dropping packets...I've got an onboard Intel Ethernet, so I've looked then on the eepro100 mailing list but it looks rather quiet. So, anybody has had any experience tweaking the buffers(?) on a/the eepro100 network card driver? Best regards Bogdan P.S. the other softflowd patch work like a charm - no more daily archives of /var/log/messages , due to flow-capture errors. From djm at mindrot.org Wed May 4 08:10:56 2005 From: djm at mindrot.org (Damien Miller) Date: Wed, 04 May 2005 08:10:56 +1000 Subject: [netflow-tools] Re: Pcap dropping packets In-Reply-To: <00bb01c54ff4$4c049370$8360a38d@tweak> References: <00bb01c54ff4$4c049370$8360a38d@tweak> Message-ID: <4277F6F0.3000104@mindrot.org> Bogdan Ghita wrote: > Hello again > > I'm aware that the subject is rather off-topic for the list, but it > seems that either nobody has had problems before with it or it's my > machine that really needs upgrading. > > I've installed the statistics patch from Damien and now I get the full > picture - "pcap packets dropped" are about 10% of "pcap packets > received". Are you using a BPF filter? E.g. "tcp and port http" -d From b.ghita at jack.see.plymouth.ac.uk Wed May 4 22:10:38 2005 From: b.ghita at jack.see.plymouth.ac.uk (Bogdan Ghita) Date: Wed, 4 May 2005 13:10:38 +0100 Subject: [netflow-tools] RE: Pcap dropping packets In-Reply-To: <4277F6F0.3000104@mindrot.org> Message-ID: <003d01c550a2$49b7e640$8360a38d@tweak> Not that I'm aware of :) softflowd -i eth0 -m 3000000 -v 5 -n tester:2000 /usr/local/netflow/bin/flow-capture -w /var/netflow/ft tester/tester/2000 - 5 -V5 -E1G -n 287 -N 0 -R/usr/local/netflow/bin/linkme (is there anything else that is setting any filters?) Regards Bogdan > -----Original Message----- > From: Damien Miller [mailto:djm at mindrot.org] > Sent: 03 May 2005 23:11 > To: Bogdan Ghita > Cc: netflow-tools at mindrot.org > Subject: Re: Pcap dropping packets > > Bogdan Ghita wrote: > > Hello again > > > > I'm aware that the subject is rather off-topic for the list, but it > > seems that either nobody has had problems before with it or it's my > > machine that really needs upgrading. > > > > I've installed the statistics patch from Damien and now I get the full > > picture - "pcap packets dropped" are about 10% of "pcap packets > > received". > > Are you using a BPF filter? E.g. "tcp and port http" > > -d From djm at mindrot.org Sat May 14 16:45:41 2005 From: djm at mindrot.org (Damien Miller) Date: Sat, 14 May 2005 16:45:41 +1000 Subject: [netflow-tools] softflowd NetFlow v.9 flow times bug Message-ID: <42859E95.10203@mindrot.org> Hi, I just discovered a dumb bug in softflowd's NetFlow v.9 flow times: the first_switched and the last_switched times are reversed. Those of you using softflowd's NetFlow v.9 export may want to apply the attached patch until the next version is released. Apologies for any inconvenience. Regards, Damien Miller -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: softflowd-netflow9-time-bug.diff Url: http://lists.mindrot.org/pipermail/netflow-tools/attachments/20050514/f2a1534c/attachment.ksh From djm at mindrot.org Sat May 14 17:57:32 2005 From: djm at mindrot.org (Damien Miller) Date: Sat, 14 May 2005 17:57:32 +1000 Subject: [netflow-tools] Announce: flowd-0.8.5 Message-ID: <4285AF6C.1040902@mindrot.org> flowd-0.8.5 has just been released. It is available from http://www.mindrot.org/flowd.html flowd is a small, fast and secure NetFlow collector that supports NetFlow versions 1, 5, 7 and 9. It is fully IPv6 capable and supports capture of flows sent to multicast groups. flowd is privilege separated to limit the impact of any security vulnerability. Netflow logs are stored in a compact and flexible binary format that supports selection of which flow fields are stored. The flowd distribution provides C, Perl and Python APIs to read these logs. Changes since flowd-0.8.5: ======================== * Improved filters. Filter expressions may now match on: - TCP flags (e.g. match SYN-only flows) - Address family (IPv4 vs. IPv6 flows) - Time of day range (e.g. from 9am to 5pm) - Day(s) of the week (e.g. Monday-Friday) * Extended Python API to allow it to write binary flow logs * Allow store.c routines (and thus the flowd-reader tool) to read from and write to pipes. * Bug fixes to souce port filters and FILTER_DEBUG code from Martynov Nikolay Checksums: ========== - MD5 (flowd-0.8.5.tar.gz) = b747abad67db7da2e0ce903ba4f61905 Reporting Bugs: =============== - Please report bugs using http://bugzilla.mindrot.org/ and/or the netflow-tools at mindrot.org mailing list. To subscribe to this mailing list, please use the web interface at http://www.mindrot.org/mailman/listinfo/netflow-tools Damien Miller From djm at mindrot.org Mon May 16 16:47:05 2005 From: djm at mindrot.org (Damien Miller) Date: Mon, 16 May 2005 16:47:05 +1000 Subject: [netflow-tools] Re: Pcap dropping packets In-Reply-To: <003d01c550a2$49b7e640$8360a38d@tweak> References: <003d01c550a2$49b7e640$8360a38d@tweak> Message-ID: <428841E9.3000906@mindrot.org> Bogdan Ghita wrote: > Not that I'm aware of :) > > softflowd -i eth0 -m 3000000 -v 5 -n tester:2000 > > /usr/local/netflow/bin/flow-capture -w /var/netflow/ft > tester/tester/2000 - 5 -V5 -E1G -n 287 -N 0 > -R/usr/local/netflow/bin/linkme > > (is there anything else that is setting any filters?) I have had a chance to test this a bit now. Either your libpcap reports different things than mine in the statistics (possible) or you are indeed overrunning your box's ability to process packets. -d From jason at dixongroup.net Mon May 16 21:36:07 2005 From: jason at dixongroup.net (Jason Dixon) Date: Mon, 16 May 2005 07:36:07 -0400 Subject: [netflow-tools] Flowd unable to use FIFO Message-ID: Forwarding to netflow-tools where it belongs... I'm trying to get flowd to write to FIFO so I can read it in with a perl script and output to my choice of storage (file, db, etc). The patch below from djm allows flowd to write to fifo, but Flowd.pm fails on init() with a "bad magic" error. Thanks, Jason > From: Damien Miller > Date: May 16, 2005 2:42:59 AM EDT > To: ports at openbsd.org > Subject: Re: Netflow collector on OpenBSD > > Jason Dixon wrote: >> I'm looking for a Netflow collector on OpenBSD that can dump to >> database. I would prefer to use djm's flowd, but it doesn't support >> database output. I've tried having it log to fifo for reading in >> with a Perl script, but it dies trying to perform a seek on startup: > > This is probably best taken to the flowd mailing list[1], but > interested > users can try the attached patch. > > -d > > [1] http://www.mindrot.org/mailman/listinfo/netflow-tools > ? build > ? buildit.sh > ? filter.day > ? flowd.conf.test > ? flowd.log > ? flowd.log.v46 > ? flowd.pyc > ? netflow-v9 > ? testwrite.py > ? x.conf > ? xxx.conf > ? tools/stats.py > ? tools/stats.pyc > Index: flowd.c > =================================================================== > RCS file: /var/cvs/flowd/flowd.c,v > retrieving revision 1.56 > diff -u -p -r1.56 flowd.c > --- flowd.c 28 Apr 2005 09:02:58 -0000 1.56 > +++ flowd.c 16 May 2005 04:16:30 -0000 > @@ -121,9 +121,19 @@ start_log(int monitor_fd) > int fd; > off_t pos; > char ebuf[512]; > + struct stat sb; > > if ((fd = client_open_log(monitor_fd)) == -1) > logerrx("Logfile open failed, exiting"); > + > + if (fstat(fd, &sb) == -1) > + logerr("log fstat"); > + > + /* Don't bother writing header to FIFOs */ > + if (S_ISFIFO(sb.st_mode)) { > + logit(LOG_DEBUG, "logfile is FIFO, skipping header write"); > + return (fd); > + } > > /* Only write out the header if we are at the start of the file */ > switch ((pos = lseek(fd, 0, SEEK_END))) { > Index: privsep.c > =================================================================== > RCS file: /var/cvs/flowd/privsep.c,v > retrieving revision 1.26 > diff -u -p -r1.26 privsep.c > --- privsep.c 14 May 2005 06:04:18 -0000 1.26 > +++ privsep.c 16 May 2005 04:16:30 -0000 > @@ -712,12 +712,22 @@ static int > answer_open_log(struct flowd_config *conf, int client_fd) > { > int fd; > + struct stat sb; > > logit(LOG_DEBUG, "%s: entering", __func__); > > fd = open(conf->log_file, O_RDWR|O_APPEND|O_CREAT, 0600); > if (fd == -1) { > - logitm(LOG_ERR, "%s: open", __func__); > + logitm(LOG_ERR, "%s: open(%.100s)", __func__, conf->log_file); > + return (-1); > + } > + if (fstat(fd, &sb) == -1) { > + logitm(LOG_ERR, "%s: fstat(%.100s)", __func__, conf->log_file); > + return (-1); > + } > + if (!S_ISREG(sb.st_mode) && !S_ISFIFO(sb.st_mode)) { > + logit(LOG_ERR, "%s: log file \"%.100s\" is neither a regular " > + "file nor a FIFO", __func__, conf->log_file); > return (-1); > } > if (send_fd(client_fd, fd) == -1) > -- Jason Dixon DixonGroup Consulting http://www.dixongroup.net From djm at mindrot.org Mon May 16 23:36:36 2005 From: djm at mindrot.org (Damien Miller) Date: Mon, 16 May 2005 23:36:36 +1000 Subject: [netflow-tools] Flowd unable to use FIFO In-Reply-To: References: Message-ID: <4288A1E4.7060106@mindrot.org> Jason Dixon wrote: > Forwarding to netflow-tools where it belongs... > > I'm trying to get flowd to write to FIFO so I can read it in with a perl > script and output to my choice of storage (file, db, etc). The patch > below from djm allows flowd to write to fifo, but Flowd.pm fails on > init() with a "bad magic" error. Please try the attached (completely untested) patch. flowd (with the FIFO patch) won't write a header when it sends records to a FIFO, so the Perl and Python APIs have to be adjusted to cope. Maybe it should just send a header after all, but I need to think about it some more. -d -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: flowd-logfifo-pp.diff Url: http://lists.mindrot.org/pipermail/netflow-tools/attachments/20050516/5b49d8f6/attachment.ksh From jason at dixongroup.net Mon May 16 23:47:43 2005 From: jason at dixongroup.net (Jason Dixon) Date: Mon, 16 May 2005 09:47:43 -0400 Subject: [netflow-tools] Flowd unable to use FIFO In-Reply-To: <4288A1E4.7060106@mindrot.org> References: <4288A1E4.7060106@mindrot.org> Message-ID: <115d5b74543dcdf90d8c986f76fb4ec1@dixongroup.net> On May 16, 2005, at 9:36 AM, Damien Miller wrote: > Please try the attached (completely untested) patch. > > flowd (with the FIFO patch) won't write a header when it sends records > to a FIFO, so the Perl and Python APIs have to be adjusted to cope. > Maybe it should just send a header after all, but I need to think about > it some more. Missing semicolon. --- Flowd-perl/lib/Flowd.pm.old Mon May 16 10:12:38 2005 +++ Flowd-perl/lib/Flowd.pm Mon May 16 10:16:50 2005 @@ -144,7 +144,7 @@ open($fhandle, "<$filename") or die "open($filename): $!"; $self->{handle} = $fhandle; - @st = stat($self->{handle}) + @st = stat($self->{handle}); if (($st[2] & 0170000) == 0010000) { # If reading from FIFO, assume that version matches $self->{version} = 0x00000002; Looks like it's working now! Thanks! -- Jason Dixon DixonGroup Consulting http://www.dixongroup.net From gniewko at aci.com.pl Mon May 16 23:58:11 2005 From: gniewko at aci.com.pl (gniewko) Date: Mon, 16 May 2005 15:58:11 +0200 Subject: [netflow-tools] softflowd Message-ID: <1116251891.1146.30.camel@morgue> Hello there; I have a problem with probing promisc interface; In my situation there is a PC which interface is connected to span port of the switch (Vlan tagged); I've created vlan interface and run softflowd with appropriate switch (-i eth0.10); SWITCH (mirroring port) ----- eth0.10 | eth1 ------ collector Everything works smoothly and collector (another PC) is receiving netflow v5 packets; The problem is that the ammount of traffic which is collected is much more smaller then the real interface load; there is about 100 mbps on the eth0.10 but flowtools on the collector shows that there is not more then ~30mbps; PC's are Linux with Intel Gigabit (eth0), Intel FastEth (eth1) and Intel Gigabit (collector). i'd be thankfull for any hint; thanks in advance, gniewko From djm at mindrot.org Tue May 17 11:03:22 2005 From: djm at mindrot.org (Damien Miller) Date: Tue, 17 May 2005 11:03:22 +1000 Subject: [netflow-tools] softflowd In-Reply-To: <1116251891.1146.30.camel@morgue> References: <1116251891.1146.30.camel@morgue> Message-ID: <428942DA.7060502@mindrot.org> gniewko wrote: > Hello there; > > I have a problem with probing promisc interface; > In my situation there is a PC which interface is connected to span port > of the switch (Vlan tagged); I've created vlan interface and run > softflowd with appropriate switch (-i eth0.10); > > > SWITCH (mirroring port) ----- eth0.10 | eth1 ------ collector > > > Everything works smoothly and collector (another PC) is receiving > netflow v5 packets; The problem is that the ammount of traffic which is > collected is much more smaller then the real interface load; > there is about 100 mbps on the eth0.10 but flowtools on the collector > shows that there is not more then ~30mbps; What does "softflowctl statistics" say? You might also want to try the attached patch which adds some stats from libpcap itself. Remember that softflowd doesn't (yet) understand 802.1q, so if the frames will be ignored if they are still tagged when it sees them. -d -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: softflowd-pcap-stats.diff Url: http://lists.mindrot.org/pipermail/netflow-tools/attachments/20050517/672d2267/attachment.ksh From jason at dixongroup.net Sat May 21 04:02:29 2005 From: jason at dixongroup.net (Jason Dixon) Date: Fri, 20 May 2005 14:02:29 -0400 Subject: [netflow-tools] Duplicate flow entry Message-ID: I'm working on a script which uses Flowd::read_flow (0.8.5 w/FIFO patches) to read in and then dump everything to a database. Everything looks fine, except I noticed a duplication of one entry during testing. The flow was a 10MB zeroes file scp'd from my laptop (192.168.0.14) to a server (10.0.0.104) binatted behind a PF box (192.168.0.22). You can see the duplication of this flow on lines 13-16 of the ascii table at: http://www.dixongroup.net/netmon.txt Any idea what might have caused this duplication? I see no other signs of duplication in the database. Thanks, -- Jason Dixon DixonGroup Consulting http://www.dixongroup.net From jason at dixongroup.net Sat May 21 07:06:22 2005 From: jason at dixongroup.net (Jason Dixon) Date: Fri, 20 May 2005 17:06:22 -0400 Subject: [netflow-tools] Duplicate flow entry In-Reply-To: References: Message-ID: On May 20, 2005, at 2:02 PM, Jason Dixon wrote: > I'm working on a script which uses Flowd::read_flow (0.8.5 w/FIFO > patches) to read in and then dump everything to a database. > Everything looks fine, except I noticed a duplication of one entry > during testing. The flow was a 10MB zeroes file scp'd from my laptop > (192.168.0.14) to a server (10.0.0.104) binatted behind a PF box > (192.168.0.22). You can see the duplication of this flow on lines > 13-16 of the ascii table at: > > http://www.dixongroup.net/netmon.txt > > Any idea what might have caused this duplication? I see no other > signs of duplication in the database. I've updated the page to reflect my more recent findings. It appears that this behavior has something to do with state being created on both interfaces. That is to say, for connections that do NOT get routed through the firewall (in this case, binat), I am only seeing one set of flows (in/out) for each connection. However, if the connection is passing from one network to the other, I see duplicate entries for each flow. Obviously, a "SELECT DISTINCT" is a sufficient workaround, but I would like to understand why this is happening. http://www.dixongroup.net/netmon.txt (updated) P.S. DJM is probably en route to the hackathon, so I'd be curious if anyone else in the community has any ideas. Thanks, -- Jason Dixon DixonGroup Consulting http://www.dixongroup.net From djm at mindrot.org Sat May 21 08:26:55 2005 From: djm at mindrot.org (Damien Miller) Date: Sat, 21 May 2005 08:26:55 +1000 Subject: [netflow-tools] Duplicate flow entry In-Reply-To: References: Message-ID: <428E642F.7060700@mindrot.org> Jason Dixon wrote: > I've updated the page to reflect my more recent findings. It appears > that this behavior has something to do with state being created on both > interfaces. That is to say, for connections that do NOT get routed > through the firewall (in this case, binat), I am only seeing one set of > flows (in/out) for each connection. However, if the connection is > passing from one network to the other, I see duplicate entries for each > flow. Obviously, a "SELECT DISTINCT" is a sufficient workaround, but I > would like to understand why this is happening. You are correct: pfflowd is reporting what pfsync gives it, so if pf creates state in two places, then pfflowd will see two state entries You can work around this by: 1) not creating state on certain traffic 2) fiddling with "set state-policy" (maybe) 3) using pfflowd's -S option Eliminating this dupes within flowd might be tricky, because they might not always be obvious. Consider the case of a firewall running a dynamic routing protocol that changes the outbound interface of a flow part way through its lifetime. Try running tcpdump on the pfsync interface and seeing if there is any other distinguishing features in the raw pfsync frames. pfflowd could probably be taught to filter on these (and it does need filtering support), > P.S. DJM is probably en route to the hackathon, so I'd be curious if > anyone else in the community has any ideas. Unfortunately I can't make it this year :( However, I will be badgering the pf guys about making life easier for pfflowd. Starting with 64-bit packet and octet counters in pfsync... -d From jason at dixongroup.net Sat May 21 13:31:41 2005 From: jason at dixongroup.net (Jason Dixon) Date: Fri, 20 May 2005 23:31:41 -0400 Subject: [netflow-tools] Duplicate flow entry In-Reply-To: <428E642F.7060700@mindrot.org> References: <428E642F.7060700@mindrot.org> Message-ID: On May 20, 2005, at 6:26 PM, Damien Miller wrote: > Jason Dixon wrote: >> I've updated the page to reflect my more recent findings. It appears >> that this behavior has something to do with state being created on >> both interfaces. That is to say, for connections that do NOT get >> routed through the firewall (in this case, binat), I am only seeing >> one set of flows (in/out) for each connection. However, if the >> connection is passing from one network to the other, I see duplicate >> entries for each flow. Obviously, a "SELECT DISTINCT" is a >> sufficient workaround, but I would like to understand why this is >> happening. > > You are correct: pfflowd is reporting what pfsync gives it, so if pf > creates state in two places, then pfflowd will see two state entries > > You can work around this by: > > 1) not creating state on certain traffic Messy, doesn't scale well for a 3rd party application. I don't want to have to force design requirements on my users. > 2) fiddling with "set state-policy" (maybe) No good, tried it already with if-bound. > 3) using pfflowd's -S option Not a solution, we definitely want to track both inbound and outbound traffic. > Eliminating this dupes within flowd might be tricky, because they might > not always be obvious. Consider the case of a firewall running a > dynamic > routing protocol that changes the outbound interface of a flow part way > through its lifetime. Agreed. > Try running tcpdump on the pfsync interface and seeing if there is any > other distinguishing features in the raw pfsync frames. pfflowd could > probably be taught to filter on these (and it does need filtering > support), What tcpdump flags do you suggest for this? I've tried "tcpdump -vvvnettti" on the physical interface, but I don't see anything revealing: ay 20 23:27:05.468242 0:c0:4f:46:94:48 1:0:5e:0:0:f0 0800 262: 10.255.255.2 > 224.0.0.240: PFSYNCv2 count 1: INS ST: (DF) [tos 0x10] (ttl 255, id 57925) May 20 23:27:06.467300 0:c0:4f:46:94:48 1:0:5e:0:0:f0 0800 214: 10.255.255.2 > 224.0.0.240: PFSYNCv2 count 2: UPD ST COMP: (DF) [tos 0x10] (ttl 255, id 35988) May 20 23:27:07.467299 0:c0:4f:46:94:48 1:0:5e:0:0:f0 0800 214: 10.255.255.2 > 224.0.0.240: PFSYNCv2 count 2: UPD ST COMP: (DF) [tos 0x10] (ttl 255, id 62175) > Unfortunately I can't make it this year :( Very sorry to hear it. Wish we'd know beforehand, perhaps we could've organized a DJM hackathon drive. :-P > However, I will be badgering the pf guys about making life easier for > pfflowd. Starting with 64-bit packet and octet counters in pfsync... For the time being, "SELECT DISTINCT" will be fine. But in the long run, it would be nice if PF didn't have to create dedicated state entries for each interface. Sounds good from a dreamer's perspective, but not very practical in real life. Thanks, -- Jason Dixon DixonGroup Consulting http://www.dixongroup.net From jason at dixongroup.net Sun May 22 14:36:41 2005 From: jason at dixongroup.net (Jason Dixon) Date: Sun, 22 May 2005 00:36:41 -0400 Subject: [netflow-tools] Flow direction Message-ID: <006bf132e305f8ddcceed9326351c952@dixongroup.net> Still working on the same flowd->db application, looks like I might have found an interface/direction problem with either pfsync or pfflowd. I noticed that no matter what type of traffic I push through my PF firewalls, "inbound" traffic is always identified with the external interface and "outbound" with the internal. Even when I attempt outbound connections from internal hosts (or the firewall itself), outbound traffic is always assigned to the internal interface. An example flow can be seen in line 1 of the output at http://www.dixongroup.net/netmon.txt. I have enabled if-bound state-policy, but this has had no effect. Is this another pfsync->pfflowd issue? Thanks, -- Jason Dixon DixonGroup Consulting http://www.dixongroup.net From jason at dixongroup.net Sun May 22 22:42:06 2005 From: jason at dixongroup.net (Jason Dixon) Date: Sun, 22 May 2005 08:42:06 -0400 Subject: [netflow-tools] Flow direction In-Reply-To: <006bf132e305f8ddcceed9326351c952@dixongroup.net> References: <006bf132e305f8ddcceed9326351c952@dixongroup.net> Message-ID: <3ffa604b19c57e1f5eb32bf965c8854d@dixongroup.net> On May 22, 2005, at 12:36 AM, Jason Dixon wrote: > Still working on the same flowd->db application, looks like I might > have found an interface/direction problem with either pfsync or > pfflowd. I noticed that no matter what type of traffic I push through > my PF firewalls, "inbound" traffic is always identified with the > external interface and "outbound" with the internal. Even when I > attempt outbound connections from internal hosts (or the firewall > itself), outbound traffic is always assigned to the internal > interface. > > An example flow can be seen in line 1 of the output at > http://www.dixongroup.net/netmon.txt. I have enabled if-bound > state-policy, but this has had no effect. Is this another > pfsync->pfflowd issue? Would softflowd be a better probe to use during development/testing of this application? It's a netflow application, so it's not tied to PF for any reason. Thanks, -- Jason Dixon DixonGroup Consulting http://www.dixongroup.net From jason at dixongroup.net Mon May 23 13:07:26 2005 From: jason at dixongroup.net (Jason Dixon) Date: Sun, 22 May 2005 23:07:26 -0400 Subject: [netflow-tools] Softflowd not recognizing interface index Message-ID: I've dropped pfflowd to try out softflowd as it appears to be a bit more "compliant" than the pfflowd exports. However, I notice that my flows still don't seem to be reporting the correct index numbers for the interfaces. I'm not very SNMP savvy, so perhaps it's my own inexperience showing. Can someone please explain to me why the following output all have 0 as the if_index_in and if_index_out for all flows? http://www.dixongroup.net/softflowd.txt A good example is seen on lines 1-2 and 8-9. Each pair represents the same SSH session but they appear to come from different softflowd agents (first from the external interface, then the internal). Regardless, they all report an if_index_* of 0, no matter the direction or agent. Thanks, -- Jason Dixon DixonGroup Consulting http://www.dixongroup.net From djm at mindrot.org Mon May 23 13:11:13 2005 From: djm at mindrot.org (Damien Miller) Date: Mon, 23 May 2005 13:11:13 +1000 Subject: [netflow-tools] Softflowd not recognizing interface index In-Reply-To: References: Message-ID: <429149D1.1060102@mindrot.org> Jason Dixon wrote: > I've dropped pfflowd to try out softflowd as it appears to be a bit more > "compliant" than the pfflowd exports. However, I notice that my flows > still don't seem to be reporting the correct index numbers for the > interfaces. I'm not very SNMP savvy, so perhaps it's my own > inexperience showing. Can someone please explain to me why the > following output all have 0 as the if_index_in and if_index_out for all > flows? > > http://www.dixongroup.net/softflowd.txt > > A good example is seen on lines 1-2 and 8-9. Each pair represents the > same SSH session but they appear to come from different softflowd agents > (first from the external interface, then the internal). Regardless, > they all report an if_index_* of 0, no matter the direction or agent. Yes, softflowd doesn't get enough information from bpf to fill these in. If you can figure out a way to get interface names of indices out of libpcap I can fix it easily. -d From jason at dixongroup.net Mon May 23 13:20:04 2005 From: jason at dixongroup.net (Jason Dixon) Date: Sun, 22 May 2005 23:20:04 -0400 Subject: [netflow-tools] Softflowd not recognizing interface index In-Reply-To: <429149D1.1060102@mindrot.org> References: <429149D1.1060102@mindrot.org> Message-ID: <2fa6110b000b7e8c7fd82358c15d6dd7@dixongroup.net> On May 22, 2005, at 11:11 PM, Damien Miller wrote: > Jason Dixon wrote: >> I've dropped pfflowd to try out softflowd as it appears to be a bit >> more "compliant" than the pfflowd exports. However, I notice that my >> flows still don't seem to be reporting the correct index numbers for >> the interfaces. I'm not very SNMP savvy, so perhaps it's my own >> inexperience showing. Can someone please explain to me why the >> following output all have 0 as the if_index_in and if_index_out for >> all flows? >> http://www.dixongroup.net/softflowd.txt >> A good example is seen on lines 1-2 and 8-9. Each pair represents >> the same SSH session but they appear to come from different softflowd >> agents (first from the external interface, then the internal). >> Regardless, they all report an if_index_* of 0, no matter the >> direction or agent. > > Yes, softflowd doesn't get enough information from bpf to fill these > in. > If you can figure out a way to get interface names of indices out of > libpcap I can fix it easily. Hell, I don't even grok the significance of the if_index_* stuff, other than to assume that it's somehow valuable. I would think that the reporting agent would always report its own interface as 0, but what do I know. :) Unfortunately, what I know about pcap could fit into a bottlecap. -- Jason Dixon DixonGroup Consulting http://www.dixongroup.net From bastien.rozenzwejg at enstimac.fr Tue May 24 23:05:16 2005 From: bastien.rozenzwejg at enstimac.fr (Rozen) Date: Tue, 24 May 2005 15:05:16 +0200 Subject: [netflow-tools] Flowd error Message-ID: <4293268C.1040508@enstimac.fr> Hi, I'm sending netflow Tickets with pfflowd to 127.0.0.1:12345 and I would recover them with flowd but there is a snag I don't understand. rozen at 2727[rozen]# flowd -d read_config: entering child_get_config: entering recv_config: entering fd = 3 recv_config: ready to receive config send_config: entering fd = 4 send_config: done child_get_config: child config done bind: Address already in use Listener setup of [127.0.0.1]:12345 failed and my flowd.conf : logfile "/var/log/flowd" listen on 127.0.0.1:12345 flow source 127.0.0.1 store ALL discard all accept agent 127.0.0.1 accept agent [::1] Thank you for answers From jason at dixongroup.net Tue May 24 23:09:15 2005 From: jason at dixongroup.net (Jason Dixon) Date: Tue, 24 May 2005 09:09:15 -0400 Subject: [netflow-tools] Flowd error In-Reply-To: <4293268C.1040508@enstimac.fr> References: <4293268C.1040508@enstimac.fr> Message-ID: <5af3390a0024103945517229e817f49e@dixongroup.net> On May 24, 2005, at 9:05 AM, Rozen wrote: > Hi, > > I'm sending netflow Tickets with pfflowd to 127.0.0.1:12345 and I > would recover them with flowd but there is a snag I don't understand. > > > rozen at 2727[rozen]# flowd -d > > read_config: entering > child_get_config: entering > recv_config: entering fd = 3 > recv_config: ready to receive config > send_config: entering fd = 4 > send_config: done > child_get_config: child config done > bind: Address already in use > Listener setup of [127.0.0.1]:12345 failed There is already something listening at 127.0.0.1:12345. -- Jason Dixon DixonGroup Consulting http://www.dixongroup.net From bastien.rozenzwejg at enstimac.fr Tue May 24 23:35:51 2005 From: bastien.rozenzwejg at enstimac.fr (Rozen) Date: Tue, 24 May 2005 15:35:51 +0200 Subject: [netflow-tools] Flowd error In-Reply-To: <5af3390a0024103945517229e817f49e@dixongroup.net> References: <4293268C.1040508@enstimac.fr> <5af3390a0024103945517229e817f49e@dixongroup.net> Message-ID: <42932DB7.7090100@enstimac.fr> Jason Dixon wrote: > On May 24, 2005, at 9:05 AM, Rozen wrote: > >> Hi, >> >> I'm sending netflow Tickets with pfflowd to 127.0.0.1:12345 and I >> would recover them with flowd but there is a snag I don't understand. >> >> >> rozen at 2727[rozen]# flowd -d >> >> read_config: entering >> child_get_config: entering >> recv_config: entering fd = 3 >> recv_config: ready to receive config >> send_config: entering fd = 4 >> send_config: done >> child_get_config: child config done >> bind: Address already in use >> Listener setup of [127.0.0.1]:12345 failed > > > There is already something listening at 127.0.0.1:12345. > > -- > Jason Dixon > DixonGroup Consulting > http://www.dixongroup.net > > > > Thank you for this quick answer, but there is a new problem; I don't understand what file does it want when I do flowd in debugging mode ? ---------------------terminal----------------------------------------- rozen at 2727[rozen]# flowd -d read_config: entering child_get_config: entering recv_config: entering fd = 3 recv_config: ready to receive config send_config: entering fd = 4 send_config: done child_get_config: child config done Listener for [127.0.0.1]:12345 fd = 3 privsep_init: entering chdir(no): No such file or directory privsep_master: child exited child exited with status 1 ------------------------------------------------------------------------------ From jason at dixongroup.net Tue May 24 23:55:40 2005 From: jason at dixongroup.net (Jason Dixon) Date: Tue, 24 May 2005 09:55:40 -0400 Subject: [netflow-tools] Flowd error In-Reply-To: <42932DB7.7090100@enstimac.fr> References: <4293268C.1040508@enstimac.fr> <5af3390a0024103945517229e817f49e@dixongroup.net> <42932DB7.7090100@enstimac.fr> Message-ID: <3baa2a443059c16862005be6a6b623a3@dixongroup.net> On May 24, 2005, at 9:35 AM, Rozen wrote: > Thank you for this quick answer, but there is a new problem; I don't > understand what file does it want when I do flowd in debugging mode ? > > > ---------------------terminal----------------------------------------- > rozen at 2727[rozen]# flowd -d > > read_config: entering > child_get_config: entering > recv_config: entering fd = 3 > recv_config: ready to receive config > send_config: entering fd = 4 > send_config: done > child_get_config: child config done > Listener for [127.0.0.1]:12345 fd = 3 > privsep_init: entering > chdir(no): No such file or directory > privsep_master: child exited > child exited with status 1 > ----------------------------------------------------------------------- > ------- No clue. I would look to make sure that you have a valid entry for _flowd in your /etc/passwd and that /var/empty exists. It looks like it's trying to privsep to the _flowd user and failing. -- Jason Dixon DixonGroup Consulting http://www.dixongroup.net From bastien.rozenzwejg at enstimac.fr Wed May 25 00:04:08 2005 From: bastien.rozenzwejg at enstimac.fr (Rozen) Date: Tue, 24 May 2005 16:04:08 +0200 Subject: [netflow-tools] Flowd error In-Reply-To: <3baa2a443059c16862005be6a6b623a3@dixongroup.net> References: <4293268C.1040508@enstimac.fr> <5af3390a0024103945517229e817f49e@dixongroup.net> <42932DB7.7090100@enstimac.fr> <3baa2a443059c16862005be6a6b623a3@dixongroup.net> Message-ID: <42933458.5060004@enstimac.fr> Jason Dixon wrote: > On May 24, 2005, at 9:35 AM, Rozen wrote: > >> Thank you for this quick answer, but there is a new problem; I don't >> understand what file does it want when I do flowd in debugging mode ? >> >> >> ---------------------terminal----------------------------------------- >> rozen at 2727[rozen]# flowd -d >> >> read_config: entering >> child_get_config: entering >> recv_config: entering fd = 3 >> recv_config: ready to receive config >> send_config: entering fd = 4 >> send_config: done >> child_get_config: child config done >> Listener for [127.0.0.1]:12345 fd = 3 >> privsep_init: entering >> chdir(no): No such file or directory >> privsep_master: child exited >> child exited with status 1 >> ----------------------------------------------------------------------- >> ------- > > > No clue. I would look to make sure that you have a valid entry for > _flowd in your /etc/passwd and that /var/empty exists. It looks like > it's trying to privsep to the _flowd user and failing. > > -- > Jason Dixon > DixonGroup Consulting > http://www.dixongroup.net > > > > So, It can be that I have not well created my _flowd user; In my /etc/passwd, the _flowd entry is : _flowd:*:1010:1010:User &:no:/usr/sbin/nologin and /var/empty exists. What parameters for _flowd in adduser ? From jason at dixongroup.net Wed May 25 00:07:44 2005 From: jason at dixongroup.net (Jason Dixon) Date: Tue, 24 May 2005 10:07:44 -0400 Subject: [netflow-tools] Flowd error In-Reply-To: <42933458.5060004@enstimac.fr> References: <4293268C.1040508@enstimac.fr> <5af3390a0024103945517229e817f49e@dixongroup.net> <42932DB7.7090100@enstimac.fr> <3baa2a443059c16862005be6a6b623a3@dixongroup.net> <42933458.5060004@enstimac.fr> Message-ID: On May 24, 2005, at 10:04 AM, Rozen wrote: > So, It can be that I have not well created my _flowd user; > In my /etc/passwd, the _flowd entry is : > _flowd:*:1010:1010:User &:no:/usr/sbin/nologin > > and /var/empty exists. > What parameters for _flowd in adduser ? # grep flow /etc/passwd _flowd:*:77:77:flowd privsep:/var/empty:/sbin/nologin # ls -l /var/ | grep empty drwxr-xr-x 3 root wheel 512 Mar 29 2004 empty -- Jason Dixon DixonGroup Consulting http://www.dixongroup.net From bastien.rozenzwejg at enstimac.fr Wed May 25 00:24:52 2005 From: bastien.rozenzwejg at enstimac.fr (Rozen) Date: Tue, 24 May 2005 16:24:52 +0200 Subject: [netflow-tools] Flowd error In-Reply-To: References: <4293268C.1040508@enstimac.fr> <5af3390a0024103945517229e817f49e@dixongroup.net> <42932DB7.7090100@enstimac.fr> <3baa2a443059c16862005be6a6b623a3@dixongroup.net> <42933458.5060004@enstimac.fr> Message-ID: <42933934.1040003@enstimac.fr> Jason Dixon wrote: > On May 24, 2005, at 10:04 AM, Rozen wrote: > >> So, It can be that I have not well created my _flowd user; >> In my /etc/passwd, the _flowd entry is : >> _flowd:*:1010:1010:User &:no:/usr/sbin/nologin >> >> and /var/empty exists. >> What parameters for _flowd in adduser ? > > > # grep flow /etc/passwd > _flowd:*:77:77:flowd privsep:/var/empty:/sbin/nologin > > # ls -l /var/ | grep empty > drwxr-xr-x 3 root wheel 512 Mar 29 2004 empty > > > -- > Jason Dixon > DixonGroup Consulting > http://www.dixongroup.net > > > I've set up as you my user, and my directory... and it doesn't work !! I have theses messages: ----------------terminal----------------------------------------- rozen at 2727[rozen]# flowd -d read_config: entering child_get_config: entering recv_config: entering fd = 3 recv_config: ready to receive config send_config: entering fd = 4 send_config: done child_get_config: child config done Listener for [127.0.0.1]:12345 fd = 3 Listener for [::1]:12345 fd = 4 privsep_init: entering init_pfd: entering (num_fds = 0) init_pfd: done (num_fds = 3) client_open_log: entering answer_open_log: entering answer_open_log: open: Is a directory rozen at 2727[rozen]# receive_fd: recvmsg: expected received 1 got 0 ----------------------------------------------------------------------- From jason at dixongroup.net Wed May 25 00:28:51 2005 From: jason at dixongroup.net (Jason Dixon) Date: Tue, 24 May 2005 10:28:51 -0400 Subject: [netflow-tools] Flowd error In-Reply-To: <42933934.1040003@enstimac.fr> References: <4293268C.1040508@enstimac.fr> <5af3390a0024103945517229e817f49e@dixongroup.net> <42932DB7.7090100@enstimac.fr> <3baa2a443059c16862005be6a6b623a3@dixongroup.net> <42933458.5060004@enstimac.fr> <42933934.1040003@enstimac.fr> Message-ID: On May 24, 2005, at 10:24 AM, Rozen wrote: > I've set up as you my user, and my directory... and it doesn't work !! > I have theses messages: > > ----------------terminal----------------------------------------- > rozen at 2727[rozen]# flowd -d > read_config: entering > child_get_config: entering > recv_config: entering fd = 3 > recv_config: ready to receive config > send_config: entering fd = 4 > send_config: done > child_get_config: child config done > Listener for [127.0.0.1]:12345 fd = 3 > Listener for [::1]:12345 fd = 4 > privsep_init: entering > init_pfd: entering (num_fds = 0) > init_pfd: done (num_fds = 3) > client_open_log: entering > answer_open_log: entering > answer_open_log: open: Is a directory > rozen at 2727[rozen]# receive_fd: recvmsg: expected received 1 got 0 > > ----------------------------------------------------------------------- No clue, sorry. Perhaps you should try backing out the install and this time just do a "make install" instead of whatever you tried. Good luck. -- Jason Dixon DixonGroup Consulting http://www.dixongroup.net From bastien.rozenzwejg at enstimac.fr Wed May 25 01:23:18 2005 From: bastien.rozenzwejg at enstimac.fr (Rozen) Date: Tue, 24 May 2005 17:23:18 +0200 Subject: [netflow-tools] Flowd error In-Reply-To: References: <4293268C.1040508@enstimac.fr> <5af3390a0024103945517229e817f49e@dixongroup.net> <42932DB7.7090100@enstimac.fr> <3baa2a443059c16862005be6a6b623a3@dixongroup.net> <42933458.5060004@enstimac.fr> <42933934.1040003@enstimac.fr> Message-ID: <429346E6.4000801@enstimac.fr> Is there anyone who has installed flowd on FreeBSD 5.4 ? How ? Thank you for answer From djm at mindrot.org Wed May 25 07:23:08 2005 From: djm at mindrot.org (Damien Miller) Date: Wed, 25 May 2005 07:23:08 +1000 Subject: [netflow-tools] Flowd error In-Reply-To: <42933934.1040003@enstimac.fr> References: <4293268C.1040508@enstimac.fr> <5af3390a0024103945517229e817f49e@dixongroup.net> <42932DB7.7090100@enstimac.fr> <3baa2a443059c16862005be6a6b623a3@dixongroup.net> <42933458.5060004@enstimac.fr> <42933934.1040003@enstimac.fr> Message-ID: <42939B3C.3080504@mindrot.org> Rozen wrote: > ----------------terminal----------------------------------------- > rozen at 2727[rozen]# flowd -d > read_config: entering > child_get_config: entering > recv_config: entering fd = 3 > recv_config: ready to receive config > send_config: entering fd = 4 > send_config: done > child_get_config: child config done > Listener for [127.0.0.1]:12345 fd = 3 > Listener for [::1]:12345 fd = 4 > privsep_init: entering > init_pfd: entering (num_fds = 0) > init_pfd: done (num_fds = 3) > client_open_log: entering > answer_open_log: entering > answer_open_log: open: Is a directory You have configured a directory instead of a file as the logfile in flowd.conf. I think I need to improve these error messages... -d From bastien.rozenzwejg at enstimac.fr Wed May 25 16:58:24 2005 From: bastien.rozenzwejg at enstimac.fr (Rozen) Date: Wed, 25 May 2005 08:58:24 +0200 Subject: [netflow-tools] Flowd error In-Reply-To: <42939B3C.3080504@mindrot.org> References: <4293268C.1040508@enstimac.fr> <5af3390a0024103945517229e817f49e@dixongroup.net> <42932DB7.7090100@enstimac.fr> <3baa2a443059c16862005be6a6b623a3@dixongroup.net> <42933458.5060004@enstimac.fr> <42933934.1040003@enstimac.fr> <42939B3C.3080504@mindrot.org> Message-ID: <42942210.1070808@enstimac.fr> Damien Miller wrote: > Rozen wrote: > >> ----------------terminal----------------------------------------- >> rozen at 2727[rozen]# flowd -d >> read_config: entering >> child_get_config: entering >> recv_config: entering fd = 3 >> recv_config: ready to receive config >> send_config: entering fd = 4 >> send_config: done >> child_get_config: child config done >> Listener for [127.0.0.1]:12345 fd = 3 >> Listener for [::1]:12345 fd = 4 >> privsep_init: entering >> init_pfd: entering (num_fds = 0) >> init_pfd: done (num_fds = 3) >> client_open_log: entering >> answer_open_log: entering >> answer_open_log: open: Is a directory > > > You have configured a directory instead of a file as the logfile in > flowd.conf. > > I think I need to improve these error messages... > > -d > > Thank you very much, all work well !!! A simple question : When pfflowd and flowd are running on the same machine which is a PF firewall with NAT and Authpf, can we see who is connected or the ip address before NAT ? Because I haven't seen any address type of my intern network...