From vsdssd at gmail.com  Mon Sep  1 14:14:54 2014
From: vsdssd at gmail.com (Varun Sharma)
Date: Mon, 1 Sep 2014 09:44:54 +0530
Subject: [netflow-tools] Softflowd IPFIX date and time problem.
In-Reply-To: <53FF26DD.3020106@sfc.wide.ad.jp>
References: <CAFat2-j_Af_o1b8ZukCGuFq2uXjXwU4=W-Sffa=+b06gs4n-_Q@mail.gmail.com>
 <53FF26DD.3020106@sfc.wide.ad.jp>
Message-ID: <CAFat2-jaaGbUbpxWGaZe+OkrvnTDP2GK1ZRfKO9mYmORT3Uf1g@mail.gmail.com>

Hi Hitoshi,

Test environment :

Two 16 core machines are connected back to back using dual port 10G
card. CentOS release 6.2 (Final) 64bit version install on both
machines.
$ uname -a
Linux hwcentos 2.6.32-220.el6.x86_64 #1 SMP Tue Dec 6 19:48:22 GMT
2011 x86_64 x86_64 x86_64 GNU/Linux

On one of machine I install Softflowd(Revision:80aac3b2fec3).
Softflowd observed packet sent by iperf client.


I go through the ipfix.c code. If I comment line number 409 and 426 in
ipfix.c file.

409 : //#if defined (_BSD_SOURCE) && defined (HAVE_ENDIAN_H) ||
defined (HAVE_HTOBE64) || defined (HAVE_HTONLL)

426 : //#endif

Now Exported IPFIX flow records include accurate flow end time in
milliseconds format and also read properly on collector(nfdump) side.
I think  (HAVE_ENDIAN_H) is not defined  that?s why problem persist.

How to resolve this problem ?

Thanks in advance .

Regards,
Varun

On Thu, Aug 28, 2014 at 6:25 PM, Hitoshi Irino <irino at sfc.wide.ad.jp> wrote:
> Hello Varun,
>
> I tested on Ubuntu Linux 14.04.1 64bit version.
> In my test environment, softflowd observed packets sent by nmap -sU(UDP port
> scan). It works well. Exported IPFIX flow records include accurate flow end
> time.
>
> Could you teach me your environment?
>
> Regards,
> Hitoshi
>
>
> On 2014/08/28 14:22, Varun Sharma wrote:
>>
>> Hi ,
>>
>> I am using Softflowd IPFIX supported version ( Revision : 80aac3b2fec3
>> ) downloaded from google code. I export flows in IPFIX format to
>> collector server ( NFDUMP 1.6.10 ) . I am seeing  issue  with date and
>> time field when I am reading nfdump logs .
>>
>> Whereas In case of Netflow v5 and v9 it is working fine means proper
>> date and time comes in nfdump logs.
>>
>> Command run :
>>
>> softflowd  -i eth3 -n 192.168.50.2:9995 -v 10 -d -t maxlife=30  -D -A
>> milli
>>
>> nfdump log :
>>
>> $ nfdump -r nfcapd.201408280909
>>
>> Date first seen          Duration Proto      Src IP Addr:Port
>> Dst IP Addr:Port   Packets    Bytes Flows
>>
>> 2005-04-02 04:35:37.968 1970-01-01 05:30:00.000 3182570558.032 TCP
>>    192.168.50.1:43241 ->     192.168.50.2:5001  .AP.SF   0    17405
>> 823.5 M     1
>>
>> 2005-04-02 04:35:37.968 1970-01-01 05:30:00.000 3182570558.032 TCP
>>    192.168.50.2:5001  ->     192.168.50.1:43241 .A..SF   0    15470
>> 711626     1
>>
>> 2005-04-02 04:35:37.968 1970-01-01 05:30:00.000 3182570558.032 TCP
>>    192.168.50.1:43242 ->     192.168.50.2:5001  .AP.SF   0    20138
>> 928.1 M     1
>>
>> 2005-04-02 04:35:37.968 1970-01-01 05:30:00.000 3182570558.032 TCP
>>    192.168.50.2:5001  ->     192.168.50.1:43242 .A..SF   0    20814
>> 957450     1
>>
>> 2005-04-02 04:35:37.967 1970-01-01 05:30:00.000 3182570558.033 TCP
>>    192.168.50.1:43243 ->     192.168.50.2:5001  .AP.SF   0    20031
>> 925.8 M     1
>>
>> .......
>>
>> 2015-06-17 11:04:55.259 1970-01-01 05:30:00.000 2860448000.741 TCP
>>    192.168.50.1:43257 ->     192.168.50.2:5001  .AP.SF   0     7235
>> 348.3 M     1
>>
>> 2015-06-17 11:04:55.259 1970-01-01 05:30:00.000 2860448000.741 TCP
>>    192.168.50.2:5001  ->     192.168.50.1:43257 .A..SF   0    10138
>> 466354     1
>>
>> 2015-06-17 11:04:55.248 1970-01-01 05:30:00.000 2860448000.752 TCP
>>    192.168.50.1:43258 ->     192.168.50.2:5001  .AP.SF   0    13164
>> 610.1 M     1
>>
>> 2015-06-17 11:04:55.248 1970-01-01 05:30:00.000 2860448000.752 TCP
>>    192.168.50.2:5001  ->     192.168.50.1:43258 .A..SF   0    15663
>> 720504     1
>> .......
>>
>> 2016-01-02 07:16:04.432 1970-01-01 05:30:00.000 2843268131.568 TCP
>>    192.168.50.2:5001  ->     192.168.50.1:43268 .A..SF   0    18639
>> 857400     1
>>
>> 2016-01-02 07:16:04.421 1970-01-01 05:30:00.000 2843268131.579 TCP
>>    192.168.50.1:43269 ->     192.168.50.2:5001  .AP.SF   0    28301
>> 1.3 G     1
>>
>> 2016-01-02 07:16:04.421 1970-01-01 05:30:00.000 2843268131.579 TCP
>>    192.168.50.2:5001  ->     192.168.50.1:43269 .A..SF   0    34656
>> 1.6 M     1
>>
>> 2016-01-02 07:16:04.421 1970-01-01 05:30:00.000 2843268131.579 TCP
>>    192.168.50.1:43270 ->     192.168.50.2:5001  .AP.SF   0    29209
>> 1.3 G     1
>>
>>
>> ....
>>
>> Summary: total flows: 162, total bytes: 59.4 G, total packets: 2.6 M,
>> avg bps: 0, avg pps: 0, avg bpp: 0
>> Time window: 2014-08-28 09:09:31 - 2014-08-28 09:14:31
>> Total flows processed: 162, Blocks skipped: 0, Bytes read: 9832
>> Sys: 0.005s flows/second: 27009.0    Wall: 0.005s flows/second: 30291.7
>>
>>
>> I also used sec with ?A option but in that case also same problem
>> persist.  I attached tcpdump pcap file also. Pls find attachment.
>>
>> Can anybody know why it?s happening ?
>>
>>
>> Regards
>> Varun
>>
>>
>>
>> _______________________________________________
>> netflow-tools mailing list
>> netflow-tools at mindrot.org
>> https://lists.mindrot.org/mailman/listinfo/netflow-tools
>>
>

From vsdssd at gmail.com  Thu Sep  4 21:40:18 2014
From: vsdssd at gmail.com (Varun Sharma)
Date: Thu, 4 Sep 2014 17:10:18 +0530
Subject: [netflow-tools] softflowd dropping packets in high speed network
	traffic.
Message-ID: <CAFat2-j2gutM2nswwHqor62v_K59+z91kj0gpTbg99b0BPYiow@mail.gmail.com>

Hi,

I have two questions about softflowd.

1.  Is there any multithread supported version of softflowd?

2.  I have installed softflowd (Revision: 80aac3b2fec3). In case of
high speed network traffic, it starts dropping packets.
     How will it handle high speed network traffic?


I have tried it and the output is given below:

Command run:

 softflowd  -i eth3 -n 192.168.50.2:9995 -v 10 -d -t maxlife=300 -A milli


[root at varun]# softflowctl statistics

softflowd[5250]: Accumulated statistics since 2014-09-04T05:08:58 UTC:

Number of active flows: 65536
Packets processed: 181485841
Fragments: 0
Ignored packets: 38 (38 non-IP, 0 too short)
Flows expired: 65536 (0 forced)
Flows exported: 65536 in 16913 packets (0 failures)
Packets received by libpcap: 182171059
Packets dropped by libpcap: 6848371
Packets dropped by interface: 0
Expired flow statistics:  minimum       average       maximum
Flow bytes:               52368        117995        118422
Flow packets:              1218          2744          2754
Duration:                300.03s       300.21s       300.25s

Expired flow reasons:

tcp =         0   tcp.rst =         0   tcp.fin =         0
udp =         0      icmp =         0   general =         0
maxlife =     65536
over 2 GiB =         0
maxflows =         0
flushed =         0

Per-protocol statistics:     Octets      Packets   Avg Life    Max Life

            tcp (6):     7732952536    179836099     300.21s     300.25s


Regards
Varun

From raphaelruiz at gmail.com  Thu Sep  4 22:57:07 2014
From: raphaelruiz at gmail.com (Raphael Ruiz)
Date: Thu, 4 Sep 2014 09:57:07 -0300
Subject: [netflow-tools] softflowd dropping packets in high speed
 networktraffic.
In-Reply-To: <CAFat2-j2gutM2nswwHqor62v_K59+z91kj0gpTbg99b0BPYiow@mail.gmail.com>
References: <CAFat2-j2gutM2nswwHqor62v_K59+z91kj0gpTbg99b0BPYiow@mail.gmail.com>
Message-ID: <540861ca.c120340a.4907.ffffbebb@mx.google.com>

Hi Varun.

I belive that you need increase the max flow active.

By.

-----Mensagem Original-----
De: "Varun Sharma" <vsdssd at gmail.com>
Enviada em: ?04/?09/?2014 08:41
Para: "netflow-tools at mindrot.org" <netflow-tools at mindrot.org>
Assunto: [netflow-tools] softflowd dropping packets in high speed networktraffic.

Hi,

I have two questions about softflowd.

1.  Is there any multithread supported version of softflowd?

2.  I have installed softflowd (Revision: 80aac3b2fec3). In case of
high speed network traffic, it starts dropping packets.
     How will it handle high speed network traffic?


I have tried it and the output is given below:

Command run:

 softflowd  -i eth3 -n 192.168.50.2:9995 -v 10 -d -t maxlife=300 -A milli


[root at varun]# softflowctl statistics

softflowd[5250]: Accumulated statistics since 2014-09-04T05:08:58 UTC:

Number of active flows: 65536
Packets processed: 181485841
Fragments: 0
Ignored packets: 38 (38 non-IP, 0 too short)
Flows expired: 65536 (0 forced)
Flows exported: 65536 in 16913 packets (0 failures)
Packets received by libpcap: 182171059
Packets dropped by libpcap: 6848371
Packets dropped by interface: 0
Expired flow statistics:  minimum       average       maximum
Flow bytes:               52368        117995        118422
Flow packets:              1218          2744          2754
Duration:                300.03s       300.21s       300.25s

Expired flow reasons:

tcp =         0   tcp.rst =         0   tcp.fin =         0
udp =         0      icmp =         0   general =         0
maxlife =     65536
over 2 GiB =         0
maxflows =         0
flushed =         0

Per-protocol statistics:     Octets      Packets   Avg Life    Max Life

            tcp (6):     7732952536    179836099     300.21s     300.25s


Regards
Varun
_______________________________________________
netflow-tools mailing list
netflow-tools at mindrot.org
https://lists.mindrot.org/mailman/listinfo/netflow-tools
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mindrot.org/pipermail/netflow-tools/attachments/20140904/a73eb55b/attachment.html>

From irino at sfc.wide.ad.jp  Fri Sep  5 11:09:04 2014
From: irino at sfc.wide.ad.jp (Hitoshi Irino)
Date: Fri, 05 Sep 2014 10:09:04 +0900
Subject: [netflow-tools] Softflowd IPFIX date and time problem.
In-Reply-To: <CAFat2-jaaGbUbpxWGaZe+OkrvnTDP2GK1ZRfKO9mYmORT3Uf1g@mail.gmail.com>
References: <CAFat2-j_Af_o1b8ZukCGuFq2uXjXwU4=W-Sffa=+b06gs4n-_Q@mail.gmail.com>	<53FF26DD.3020106@sfc.wide.ad.jp>
 <CAFat2-jaaGbUbpxWGaZe+OkrvnTDP2GK1ZRfKO9mYmORT3Uf1g@mail.gmail.com>
Message-ID: <54090D30.6050704@sfc.wide.ad.jp>

Hello Varun,

Could you show me reuslt of this command:

grep ^# config.h

Regards,
Hitoshi

On 2014/09/01 13:14, Varun Sharma wrote:
> Hi Hitoshi,
>
> Test environment :
>
> Two 16 core machines are connected back to back using dual port 10G
> card. CentOS release 6.2 (Final) 64bit version install on both
> machines.
> $ uname -a
> Linux hwcentos 2.6.32-220.el6.x86_64 #1 SMP Tue Dec 6 19:48:22 GMT
> 2011 x86_64 x86_64 x86_64 GNU/Linux
>
> On one of machine I install Softflowd(Revision:80aac3b2fec3).
> Softflowd observed packet sent by iperf client.
>
>
> I go through the ipfix.c code. If I comment line number 409 and 426 in
> ipfix.c file.
>
> 409 : //#if defined (_BSD_SOURCE) && defined (HAVE_ENDIAN_H) ||
> defined (HAVE_HTOBE64) || defined (HAVE_HTONLL)
>
> 426 : //#endif
>
> Now Exported IPFIX flow records include accurate flow end time in
> milliseconds format and also read properly on collector(nfdump) side.
> I think  (HAVE_ENDIAN_H) is not defined  that?s why problem persist.
>
> How to resolve this problem ?
>
> Thanks in advance .
>
> Regards,
> Varun
>
> On Thu, Aug 28, 2014 at 6:25 PM, Hitoshi Irino <irino at sfc.wide.ad.jp> wrote:
>> Hello Varun,
>>
>> I tested on Ubuntu Linux 14.04.1 64bit version.
>> In my test environment, softflowd observed packets sent by nmap -sU(UDP port
>> scan). It works well. Exported IPFIX flow records include accurate flow end
>> time.
>>
>> Could you teach me your environment?
>>
>> Regards,
>> Hitoshi
>>
>>
>> On 2014/08/28 14:22, Varun Sharma wrote:
>>>
>>> Hi ,
>>>
>>> I am using Softflowd IPFIX supported version ( Revision : 80aac3b2fec3
>>> ) downloaded from google code. I export flows in IPFIX format to
>>> collector server ( NFDUMP 1.6.10 ) . I am seeing  issue  with date and
>>> time field when I am reading nfdump logs .
>>>
>>> Whereas In case of Netflow v5 and v9 it is working fine means proper
>>> date and time comes in nfdump logs.
>>>
>>> Command run :
>>>
>>> softflowd  -i eth3 -n 192.168.50.2:9995 -v 10 -d -t maxlife=30  -D -A
>>> milli
>>>
>>> nfdump log :
>>>
>>> $ nfdump -r nfcapd.201408280909
>>>
>>> Date first seen          Duration Proto      Src IP Addr:Port
>>> Dst IP Addr:Port   Packets    Bytes Flows
>>>
>>> 2005-04-02 04:35:37.968 1970-01-01 05:30:00.000 3182570558.032 TCP
>>>     192.168.50.1:43241 ->     192.168.50.2:5001  .AP.SF   0    17405
>>> 823.5 M     1
>>>
>>> 2005-04-02 04:35:37.968 1970-01-01 05:30:00.000 3182570558.032 TCP
>>>     192.168.50.2:5001  ->     192.168.50.1:43241 .A..SF   0    15470
>>> 711626     1
>>>
>>> 2005-04-02 04:35:37.968 1970-01-01 05:30:00.000 3182570558.032 TCP
>>>     192.168.50.1:43242 ->     192.168.50.2:5001  .AP.SF   0    20138
>>> 928.1 M     1
>>>
>>> 2005-04-02 04:35:37.968 1970-01-01 05:30:00.000 3182570558.032 TCP
>>>     192.168.50.2:5001  ->     192.168.50.1:43242 .A..SF   0    20814
>>> 957450     1
>>>
>>> 2005-04-02 04:35:37.967 1970-01-01 05:30:00.000 3182570558.033 TCP
>>>     192.168.50.1:43243 ->     192.168.50.2:5001  .AP.SF   0    20031
>>> 925.8 M     1
>>>
>>> .......
>>>
>>> 2015-06-17 11:04:55.259 1970-01-01 05:30:00.000 2860448000.741 TCP
>>>     192.168.50.1:43257 ->     192.168.50.2:5001  .AP.SF   0     7235
>>> 348.3 M     1
>>>
>>> 2015-06-17 11:04:55.259 1970-01-01 05:30:00.000 2860448000.741 TCP
>>>     192.168.50.2:5001  ->     192.168.50.1:43257 .A..SF   0    10138
>>> 466354     1
>>>
>>> 2015-06-17 11:04:55.248 1970-01-01 05:30:00.000 2860448000.752 TCP
>>>     192.168.50.1:43258 ->     192.168.50.2:5001  .AP.SF   0    13164
>>> 610.1 M     1
>>>
>>> 2015-06-17 11:04:55.248 1970-01-01 05:30:00.000 2860448000.752 TCP
>>>     192.168.50.2:5001  ->     192.168.50.1:43258 .A..SF   0    15663
>>> 720504     1
>>> .......
>>>
>>> 2016-01-02 07:16:04.432 1970-01-01 05:30:00.000 2843268131.568 TCP
>>>     192.168.50.2:5001  ->     192.168.50.1:43268 .A..SF   0    18639
>>> 857400     1
>>>
>>> 2016-01-02 07:16:04.421 1970-01-01 05:30:00.000 2843268131.579 TCP
>>>     192.168.50.1:43269 ->     192.168.50.2:5001  .AP.SF   0    28301
>>> 1.3 G     1
>>>
>>> 2016-01-02 07:16:04.421 1970-01-01 05:30:00.000 2843268131.579 TCP
>>>     192.168.50.2:5001  ->     192.168.50.1:43269 .A..SF   0    34656
>>> 1.6 M     1
>>>
>>> 2016-01-02 07:16:04.421 1970-01-01 05:30:00.000 2843268131.579 TCP
>>>     192.168.50.1:43270 ->     192.168.50.2:5001  .AP.SF   0    29209
>>> 1.3 G     1
>>>
>>>
>>> ....
>>>
>>> Summary: total flows: 162, total bytes: 59.4 G, total packets: 2.6 M,
>>> avg bps: 0, avg pps: 0, avg bpp: 0
>>> Time window: 2014-08-28 09:09:31 - 2014-08-28 09:14:31
>>> Total flows processed: 162, Blocks skipped: 0, Bytes read: 9832
>>> Sys: 0.005s flows/second: 27009.0    Wall: 0.005s flows/second: 30291.7
>>>
>>>
>>> I also used sec with ?A option but in that case also same problem
>>> persist.  I attached tcpdump pcap file also. Pls find attachment.
>>>
>>> Can anybody know why it?s happening ?
>>>
>>>
>>> Regards
>>> Varun
>>>
>>>
>>>
>>> _______________________________________________
>>> netflow-tools mailing list
>>> netflow-tools at mindrot.org
>>> https://lists.mindrot.org/mailman/listinfo/netflow-tools
>>>
>>
>

From vsdssd at gmail.com  Wed Sep 17 16:03:56 2014
From: vsdssd at gmail.com (Varun Sharma)
Date: Wed, 17 Sep 2014 11:33:56 +0530
Subject: [netflow-tools] Softflowd IPFIX date and time problem.
In-Reply-To: <54090D30.6050704@sfc.wide.ad.jp>
References: <CAFat2-j_Af_o1b8ZukCGuFq2uXjXwU4=W-Sffa=+b06gs4n-_Q@mail.gmail.com>
 <53FF26DD.3020106@sfc.wide.ad.jp>
 <CAFat2-jaaGbUbpxWGaZe+OkrvnTDP2GK1ZRfKO9mYmORT3Uf1g@mail.gmail.com>
 <54090D30.6050704@sfc.wide.ad.jp>
Message-ID: <CAFat2-gTFFEQOJX2L3zsbX_+D=_4FPQKn9sG+ouTWubDCZ6Tbw@mail.gmail.com>

Hi Hitoshi ,

I found that HAVE_ENDIAN_H not defined in configure script that?s why
problem persist. After define HAVE_ENDIAN_H problem is solved.

$ grep ^# config.h

#define HAVE_DAEMON 1

#define HAVE_INT16_T 1

#define HAVE_INT32_T 1

#define HAVE_INT64_T 1

#define HAVE_INT8_T 1

#define HAVE_INTTYPES_H 1

#define HAVE_LIBPCAP 1

#define HAVE_MEMORY_H 1

#define HAVE_PCAP_BPF_H 1

#define HAVE_PCAP_H 1

#define HAVE_SETGID 1

#define HAVE_SETRESGID 1

#define HAVE_SETRESUID 1

#define HAVE_SETREUID 1

#define HAVE_STDINT_H 1

#define HAVE_STDLIB_H 1

#define HAVE_STRINGS_H 1

#define HAVE_STRING_H 1

#define HAVE_STRUCT_IP6_EXT 1

#define HAVE_SYS_STAT_H 1

#define HAVE_SYS_TYPES_H 1

#define HAVE_UINT16_T 1

#define HAVE_UINT32_T 1

#define HAVE_UINT64_T 1

#define HAVE_UINT8_T 1

#define HAVE_UNISTD_H 1

#define HAVE_U_INT16_T 1

#define HAVE_U_INT32_T 1

#define HAVE_U_INT64_T 1

#define HAVE_U_INT8_T 1

#define OUR_CFG_INT16_T short int

#define OUR_CFG_INT32_T int

#define OUR_CFG_INT64_T long int

#define OUR_CFG_INT8_T signed char

#define OUR_CFG_U_INT16_T uint16_t

#define OUR_CFG_U_INT32_T uint32_t

#define OUR_CFG_U_INT64_T uint64_t

#define OUR_CFG_U_INT8_T uint8_t

#define PACKAGE_BUGREPORT ""

#define PACKAGE_NAME ""

#define PACKAGE_STRING ""

#define PACKAGE_TARNAME ""

#define PACKAGE_URL ""

#define PACKAGE_VERSION ""

#define SIZEOF_CHAR 1

#define SIZEOF_INT 4

#define SIZEOF_LONG_INT 8

#define SIZEOF_LONG_LONG_INT 8

#define SIZEOF_SHORT_INT 2

#define STDC_HEADERS 1



There is need to update configure script and config.h.in file. In
configure script add check for endian.h file.



for ac_header in endian.h

do :

  as_ac_Header=`$as_echo "ac_cv_header_$ac_header" | $as_tr_sh`

ac_fn_c_check_header_mongrel "$LINENO" "$ac_header" "$as_ac_Header"
"$ac_includes_default"

if eval test \"x\$"$as_ac_Header"\" = x"yes"; then :

  cat >>confdefs.h <<_ACEOF

#define `$as_echo "HAVE_$ac_header" | $as_tr_cpp` 1
_ACEOF

fi
done


In config.h.in add  #undef HAVE_ENDIAN_H line.


Regards,
Varun


On Fri, Sep 5, 2014 at 6:39 AM, Hitoshi Irino <irino at sfc.wide.ad.jp> wrote:
> Hello Varun,
>
> Could you show me reuslt of this command:
>
> grep ^# config.h
>
> Regards,
> Hitoshi
>
>
> On 2014/09/01 13:14, Varun Sharma wrote:
>>
>> Hi Hitoshi,
>>
>> Test environment :
>>
>> Two 16 core machines are connected back to back using dual port 10G
>> card. CentOS release 6.2 (Final) 64bit version install on both
>> machines.
>> $ uname -a
>> Linux hwcentos 2.6.32-220.el6.x86_64 #1 SMP Tue Dec 6 19:48:22 GMT
>> 2011 x86_64 x86_64 x86_64 GNU/Linux
>>
>> On one of machine I install Softflowd(Revision:80aac3b2fec3).
>> Softflowd observed packet sent by iperf client.
>>
>>
>> I go through the ipfix.c code. If I comment line number 409 and 426 in
>> ipfix.c file.
>>
>> 409 : //#if defined (_BSD_SOURCE) && defined (HAVE_ENDIAN_H) ||
>> defined (HAVE_HTOBE64) || defined (HAVE_HTONLL)
>>
>> 426 : //#endif
>>
>> Now Exported IPFIX flow records include accurate flow end time in
>> milliseconds format and also read properly on collector(nfdump) side.
>> I think  (HAVE_ENDIAN_H) is not defined  that?s why problem persist.
>>
>> How to resolve this problem ?
>>
>> Thanks in advance .
>>
>> Regards,
>> Varun
>>
>> On Thu, Aug 28, 2014 at 6:25 PM, Hitoshi Irino <irino at sfc.wide.ad.jp>
>> wrote:
>>>
>>> Hello Varun,
>>>
>>> I tested on Ubuntu Linux 14.04.1 64bit version.
>>> In my test environment, softflowd observed packets sent by nmap -sU(UDP
>>> port
>>> scan). It works well. Exported IPFIX flow records include accurate flow
>>> end
>>> time.
>>>
>>> Could you teach me your environment?
>>>
>>> Regards,
>>> Hitoshi
>>>
>>>
>>> On 2014/08/28 14:22, Varun Sharma wrote:
>>>>
>>>>
>>>> Hi ,
>>>>
>>>> I am using Softflowd IPFIX supported version ( Revision : 80aac3b2fec3
>>>> ) downloaded from google code. I export flows in IPFIX format to
>>>> collector server ( NFDUMP 1.6.10 ) . I am seeing  issue  with date and
>>>> time field when I am reading nfdump logs .
>>>>
>>>> Whereas In case of Netflow v5 and v9 it is working fine means proper
>>>> date and time comes in nfdump logs.
>>>>
>>>> Command run :
>>>>
>>>> softflowd  -i eth3 -n 192.168.50.2:9995 -v 10 -d -t maxlife=30  -D -A
>>>> milli
>>>>
>>>> nfdump log :
>>>>
>>>> $ nfdump -r nfcapd.201408280909
>>>>
>>>> Date first seen          Duration Proto      Src IP Addr:Port
>>>> Dst IP Addr:Port   Packets    Bytes Flows
>>>>
>>>> 2005-04-02 04:35:37.968 1970-01-01 05:30:00.000 3182570558.032 TCP
>>>>     192.168.50.1:43241 ->     192.168.50.2:5001  .AP.SF   0    17405
>>>> 823.5 M     1
>>>>
>>>> 2005-04-02 04:35:37.968 1970-01-01 05:30:00.000 3182570558.032 TCP
>>>>     192.168.50.2:5001  ->     192.168.50.1:43241 .A..SF   0    15470
>>>> 711626     1
>>>>
>>>> 2005-04-02 04:35:37.968 1970-01-01 05:30:00.000 3182570558.032 TCP
>>>>     192.168.50.1:43242 ->     192.168.50.2:5001  .AP.SF   0    20138
>>>> 928.1 M     1
>>>>
>>>> 2005-04-02 04:35:37.968 1970-01-01 05:30:00.000 3182570558.032 TCP
>>>>     192.168.50.2:5001  ->     192.168.50.1:43242 .A..SF   0    20814
>>>> 957450     1
>>>>
>>>> 2005-04-02 04:35:37.967 1970-01-01 05:30:00.000 3182570558.033 TCP
>>>>     192.168.50.1:43243 ->     192.168.50.2:5001  .AP.SF   0    20031
>>>> 925.8 M     1
>>>>
>>>> .......
>>>>
>>>> 2015-06-17 11:04:55.259 1970-01-01 05:30:00.000 2860448000.741 TCP
>>>>     192.168.50.1:43257 ->     192.168.50.2:5001  .AP.SF   0     7235
>>>> 348.3 M     1
>>>>
>>>> 2015-06-17 11:04:55.259 1970-01-01 05:30:00.000 2860448000.741 TCP
>>>>     192.168.50.2:5001  ->     192.168.50.1:43257 .A..SF   0    10138
>>>> 466354     1
>>>>
>>>> 2015-06-17 11:04:55.248 1970-01-01 05:30:00.000 2860448000.752 TCP
>>>>     192.168.50.1:43258 ->     192.168.50.2:5001  .AP.SF   0    13164
>>>> 610.1 M     1
>>>>
>>>> 2015-06-17 11:04:55.248 1970-01-01 05:30:00.000 2860448000.752 TCP
>>>>     192.168.50.2:5001  ->     192.168.50.1:43258 .A..SF   0    15663
>>>> 720504     1
>>>> .......
>>>>
>>>> 2016-01-02 07:16:04.432 1970-01-01 05:30:00.000 2843268131.568 TCP
>>>>     192.168.50.2:5001  ->     192.168.50.1:43268 .A..SF   0    18639
>>>> 857400     1
>>>>
>>>> 2016-01-02 07:16:04.421 1970-01-01 05:30:00.000 2843268131.579 TCP
>>>>     192.168.50.1:43269 ->     192.168.50.2:5001  .AP.SF   0    28301
>>>> 1.3 G     1
>>>>
>>>> 2016-01-02 07:16:04.421 1970-01-01 05:30:00.000 2843268131.579 TCP
>>>>     192.168.50.2:5001  ->     192.168.50.1:43269 .A..SF   0    34656
>>>> 1.6 M     1
>>>>
>>>> 2016-01-02 07:16:04.421 1970-01-01 05:30:00.000 2843268131.579 TCP
>>>>     192.168.50.1:43270 ->     192.168.50.2:5001  .AP.SF   0    29209
>>>> 1.3 G     1
>>>>
>>>>
>>>> ....
>>>>
>>>> Summary: total flows: 162, total bytes: 59.4 G, total packets: 2.6 M,
>>>> avg bps: 0, avg pps: 0, avg bpp: 0
>>>> Time window: 2014-08-28 09:09:31 - 2014-08-28 09:14:31
>>>> Total flows processed: 162, Blocks skipped: 0, Bytes read: 9832
>>>> Sys: 0.005s flows/second: 27009.0    Wall: 0.005s flows/second: 30291.7
>>>>
>>>>
>>>> I also used sec with ?A option but in that case also same problem
>>>> persist.  I attached tcpdump pcap file also. Pls find attachment.
>>>>
>>>> Can anybody know why it?s happening ?
>>>>
>>>>
>>>> Regards
>>>> Varun
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> netflow-tools mailing list
>>>> netflow-tools at mindrot.org
>>>> https://lists.mindrot.org/mailman/listinfo/netflow-tools
>>>>
>>>
>>
>

From vsdssd at gmail.com  Thu Sep 18 18:51:03 2014
From: vsdssd at gmail.com (Varun Sharma)
Date: Thu, 18 Sep 2014 14:21:03 +0530
Subject: [netflow-tools] ipfix sequence error problem.
Message-ID: <CAFat2-ifZiR2rnxyBjyUd12H0cEDJ2Ng422aSq=4cuRRrNt4Jw@mail.gmail.com>

Hi,

I am using Softflowd IPFIX supported version ( Revision
:80aac3b2fec3).I exported flows in IPFIX format to collector server (
NFDUMP 1.6.10 ) .
I've noticed sequence errors in nfcapd logs. Basically it means
somewhere flows are missing ( the number sequence errors ) for
whatever reason.
Whereas in case of netflow v5 or netflow v9 it?s work properly. Means
no sequence error in nfcapd logs.

After checking ipfix.c code I found that sequence number field of
ipfix header populates with wrong value.

According to RFC 5101 , IPFIX Sequence Number contains the total
number of IPFIX Data Records sent for the UDP Transport Session prior
to the receipt of this IPFIX Message, modulo 2^32.

But here sequence number field populates with total number of packet sent.

Do you have any patch that solve this problem ?

Thanks in advance.

Regards
Varun

From irino at sfc.wide.ad.jp  Sun Sep 28 21:46:48 2014
From: irino at sfc.wide.ad.jp (Hitoshi Irino)
Date: Sun, 28 Sep 2014 20:46:48 +0900
Subject: [netflow-tools] ipfix sequence error problem.
In-Reply-To: <CAFat2-ifZiR2rnxyBjyUd12H0cEDJ2Ng422aSq=4cuRRrNt4Jw@mail.gmail.com>
References: <CAFat2-ifZiR2rnxyBjyUd12H0cEDJ2Ng422aSq=4cuRRrNt4Jw@mail.gmail.com>
Message-ID: <5427F528.2030907@sfc.wide.ad.jp>

Hello

I commited new version (Revision: 33b6916b544d). It fixes sequence 
number in IPFIX header. And I fixed ifdef macro for htobe64 that does 
not need HAVE_ENDIAN_H.

Please do "autoreconf" and "configure" before you do "make".

Please try it.

Regards
Hitoshi Irino

On 2014?09?18? 17:51, Varun Sharma wrote:
> Hi,
>
> I am using Softflowd IPFIX supported version ( Revision
> :80aac3b2fec3).I exported flows in IPFIX format to collector server (
> NFDUMP 1.6.10 ) .
> I've noticed sequence errors in nfcapd logs. Basically it means
> somewhere flows are missing ( the number sequence errors ) for
> whatever reason.
> Whereas in case of netflow v5 or netflow v9 it?s work properly. Means
> no sequence error in nfcapd logs.
>
> After checking ipfix.c code I found that sequence number field of
> ipfix header populates with wrong value.
>
> According to RFC 5101 , IPFIX Sequence Number contains the total
> number of IPFIX Data Records sent for the UDP Transport Session prior
> to the receipt of this IPFIX Message, modulo 2^32.
>
> But here sequence number field populates with total number of packet sent.
>
> Do you have any patch that solve this problem ?
>
> Thanks in advance.
>
> Regards
> Varun
> _______________________________________________
> netflow-tools mailing list
> netflow-tools at mindrot.org
> https://lists.mindrot.org/mailman/listinfo/netflow-tools
>