[netflow-tools] Duplicate flow entry

[Jason Dixon]

On May 20, 2005, at 6:26 PM, Damien Miller wrote:

> Jason Dixon wrote:
>> I've updated the page to reflect my more recent findings.  It appears 
>> that this behavior has something to do with state being created on 
>> both interfaces.  That is to say, for connections that do NOT get 
>> routed through the firewall (in this case, binat), I am only seeing 
>> one set of flows (in/out) for each connection.  However, if the 
>> connection is passing from one network to the other, I see duplicate 
>> entries for each flow.  Obviously, a "SELECT DISTINCT" is a 
>> sufficient workaround, but I would like to understand why this is 
>> happening.
>
> You are correct: pfflowd is reporting what pfsync gives it, so if pf
> creates state in two places, then pfflowd will see two state entries
>
> You can work around this by:
>
> 1) not creating state on certain traffic

Messy, doesn't scale well for a 3rd party application.  I don't want to 
have to force design requirements on my users.

> 2) fiddling with "set state-policy" (maybe)

No good,  tried it already with if-bound.

> 3) using pfflowd's -S option

Not a solution, we definitely want to track both inbound and outbound 
traffic.

> Eliminating this dupes within flowd might be tricky, because they might
> not always be obvious. Consider the case of a firewall running a 
> dynamic
> routing protocol that changes the outbound interface of a flow part way
> through its lifetime.

Agreed.

> Try running tcpdump on the pfsync interface and seeing if there is any
> other distinguishing features in the raw pfsync frames. pfflowd could
> probably be taught to filter on these (and it does need filtering
> support),

What tcpdump flags do you suggest for this?  I've tried "tcpdump 
-vvvnettti" on the physical interface, but I don't see anything 
revealing:

<snip>
ay 20 23:27:05.468242 0:c0:4f:46:94:48 1:0:5e:0:0:f0 0800 262: 
10.255.255.2 > 224.0.0.240: PFSYNCv2 count 1: INS ST:
  (DF) [tos 0x10] (ttl 255, id 57925)
May 20 23:27:06.467300 0:c0:4f:46:94:48 1:0:5e:0:0:f0 0800 214: 
10.255.255.2 > 224.0.0.240: PFSYNCv2 count 2: UPD ST COMP:
  (DF) [tos 0x10] (ttl 255, id 35988)
May 20 23:27:07.467299 0:c0:4f:46:94:48 1:0:5e:0:0:f0 0800 214: 
10.255.255.2 > 224.0.0.240: PFSYNCv2 count 2: UPD ST COMP:
  (DF) [tos 0x10] (ttl 255, id 62175)
</snip>

> Unfortunately I can't make it this year :(

Very sorry to hear it.  Wish we'd know beforehand, perhaps we could've 
organized a DJM hackathon drive.  :-P

> However, I will be badgering the pf guys about making life easier for
> pfflowd. Starting with 64-bit packet and octet counters in pfsync...

For the time being, "SELECT DISTINCT" will be fine.  But in the long 
run, it would be nice if PF didn't have to create dedicated state 
entries for each interface.  Sounds good from a dreamer's perspective, 
but not very practical in real life.

Thanks,

--
Jason Dixon
DixonGroup Consulting
http://www.dixongroup.net