[netflow-tools] softflowd and pflog

[Cristian KLEIN]

Damien Miller wrote:
> On Thu, 26 Oct 2006, Cristian KLEIN wrote:
> 
>> I found it useful to log packets from a FreeBSD / OpenBSD pflog
>> interface. This way, you may fine-tune the traffic you want to export.
> 
> Have you tried pfflowd[1]? It does something very similar to what
> you want.

I found that using pfsync for flows export might be in contradiction,
for example, if somebody wants NAT states to be distributed, but does
not want to export them.

> Well, using pflog means that many fields in the flow will not be 
> accurate, especially since pflog typically records only the first 
> packet matching a state entry. You can use the "log (all)" modifier,
> but then you are back to having softflowd look at every packet.

Imagine the following scenario: you have em0 and em1, having private
IPs, doing Gigabit routing, and fxp0 going out the Internet. Obviously,
you will do NAT on fxp0. If you install softflowd on fxp0 you will only
see the translated packets, which is not very useful. Installing
softflowd on em0 and/or em1 will inject 1Gbps into a poor user-space
daemon. I found that using "pass log from <campus> to !<campus>" and
softflowd solves the problem.

> What advantages do you see in using pflog instead of pfsync (which is
> what pfflowd uses)?

I couldn't state *the* reason why pflog is better that pfsync, but I
have the following arguments for pflog:

1) One might not want to use states
For example, I don't like to use keep-state in pf, because reaching the
state limit makes my clients very unhappy. On the other hand, if
softflowd exports flows too early (either because of small timeout
values, or because the state limit has been reached), that is quite okey
with me.

2) pfsync and flow export might be in contradiction
(already explained above)

3) Easier verification
I think that doing a tcpdump in pflog0 to find out which flows are being
exported is easier than watching pfsync messages.

However, the ultimate reason is that I found it useful and it was easy
to implement, so, why not have it?

> I think it would be better to detect the presence of the net/if_pflog.h
> header in configure and use PFLOG_HDRLEN directly so softflowd will 
> automatically pick up changes in that file.

I must admit, I didn't bother too much, because I saw that all other
encapsulations are using hardcoded values too. However, I have the
following dilema: should only PFLOG_HDRLEN be taken from net/if_pflog.h,
or all values, including the position / size of the address family?