[netflow-tools] Thanks and request for flowd

[Josef Fortier]

Hi Damien:

Thanks for the reply.

>    >     1) Is there a better way to pipe ad-hoc filters to flowd-reader (or
>    >        another API).
>    
>    What sort of filters are you after? I wouldn't oppose allowing some basic
>    commandline switches to filter by source/destination address or port.

First, I should clarify my assumptions....

I'm rotating flowd files once a week. When I run the tools/ scripts
(wormsuspects.pl etc) they take a while. My assumption is that the time
reflects a linear search on the flowd file. Is this correct?

The flowd-reader lets me specify a config file, which in turn can
specify filters. This seems to have much better performance. Looking at
Flowd.pm it looks like there is no facility for filtering. In practice,
this appears to give me a large speed boost.

I'd suspect this because flowd-reader can throw away things much faster
then an external program...

Right now, I'm using short shell script to drive flowd-reader, and then
pump the output into something else (perl, awk) for further processing.

Example:

    echo "
	# Comment regarding filters
	discard quick before date ..........
	discard quick src ! [local-address-range]
	" | flowd-reader /dev/stdin	\
	  | perl -e '
		# a program to do further processing
	    '

This works OK, but the syntax is a little gunky (in particular the
/dev/stdin bothers me). 

One options is to attach stdin and read filters from there (a flag
maybe).

For the speed reasons, I'd like to be able to pass ad-hoc filter
statement lists into whatever reads the flowd file. My wish is to get
fast, relatively straightforward scripted reports out. It's all about
reporting...



>    
>    >     2) Can tagging improve filtering. It appears that tagging is a way
>    >        to create meta-information for reporting, but I keep wondering if
>    >        I can use it to create positive additive filters ("find me all
>    >        the http traffic, then find me the https")  rather then negative
>    >        filters (discard work fine cumulatively).
>    
>    Yes, there have been other requesting that too. I haven't been able to
>    figure out a syntax for flowd.conf that works well for additive/cumulative
>    tagging. The problem is that the filters are now "one match wins", but
>    cumulative tagging breaks that a little. Suggestions welcome!


I've worked out how to get some of this from the filter statements. Basically,

    1) Filter out all the negative stuff
    2) use "accept quick" to add to the positive filter
    3) discard everything

Example

    discard quick before date [date_number]
    accept quick src [address_range_ONE]
    accept quick src [address_range_TWO]
    discard any

This is limited in that the accept statements still need to be atomic,
no cumulative statements.


Thanks

Joe
-- 

_______________________________________________________________________

Josef Fortier
Network Administrator
_______________________________________________________________________