[netflow-tools] flowd benchmark
gijs at looze.net
Wed Jul 13 23:39:14 EST 2005
Damien Miller wrote:
> Damien Miller wrote:
>> flowd is always going to have to do a little more work, because the set
>> of fields that it stores is variable. That being said, it should be
>> possible to speed up the reader function by moving more it from the pure
>> python part of the module to the C implementation.
> OK, I moved all of the flow reding into the C part of the Python module
> and it didn't help much.
That's fast! : )
I did a little more research. Because I thought the python API of flowd
was slow, I wanted to write a flowd-reader parser in python. I tried 3
flowcapture programs and their readers.
The test is the most basic operation, just read out the flow log file
and print the fields
$ time flowd-reader ./flowd-log | wc -l
$ time flow-export -f2 < ./flowtools-log | wc -l
flow-export: Exported 931711 records
Much, faster but not variable field length.
$ time ./ipflow grep ./netflow.log | wc -l
This is a new one I tried, supporting v9. It isn't that much faster than
flowd. So it really is the variable field length thing that makes it slow.
All tests where done with v5 cisco flows, and on a 2 proccessor system.
More information about the netflow-tools