[netflow-tools] softflowctl expire-all

[Robin Breathe]

Damien Miller wrote:
> Robin Breathe wrote:
>> I use softflowd and flowd together on a Solaris 9 host talking NetFlow
>> v5.
>>
>> I seem to be seeing an inconsistency between the output of `softflowctl
>> statistics` and the results of issuing a `softflowctl expire-all`.
> 
> hm, this is weird...
> 
>> Is my understanding of the way netflow, and in particular netflow-tools,
>> works flawed? Any ideas on how to proceed in working out what's going
>> wrong?
> 
> Could you try running tcpdump to capture the export packets to see which
> number is correct?

Damien,

I actually did some fairly extensive testing earlier, I simply couldn't
decide how best to report the results.

Running tcpdump shows that there really are far more active flows than
end up in my flowd logs.

Running both softflowd and flowd in debug mode, it becomes apparent that
softflowd *thinks* that it is expiring all the flows (it prints the
"EXPIRED:" line in check_expired()), but some never arrive with flowd.
If I initiate a test flow (e.g. complete ssh session), then if I do an
`expire-all` after a few seconds, it won't get exported; if I wait for
it to naturally expire, it is exported. At least this is easily
reproduceable :)

The end result of this is that taking "snapshots" with `expire-all`
every 5 minutes *usually* gives me only the first 3 minutes worth of
flows (assuming the real flow-start-time algorithm we discussed earlier
is sane). The exact period which gets exported seems to vary a little
(+/-90s), though this is likely due to varying traffic patterns (running
on a network with >1500 active hosts).

Your help is really appreciated, this was an unexpected surprise drawing
close to the end of an otherwise successful project :) I was going to
trawl through the code myself tomorrow, but you already know it; I'd be
even more delighted if we could nail this bug down together.

Kind regards,
Robin


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
Url : http://lists.mindrot.org/pipermail/netflow-tools/attachments/20050927/baab7979/attachment.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 187 bytes
Desc: OpenPGP digital signature
Url : http://lists.mindrot.org/pipermail/netflow-tools/attachments/20050927/baab7979/attachment-0001.bin