TcpNoDelay socket option for QuickFIX

, ,
1

Imported from previous forum

2

Hi all,

This is more a QuickFIX implementation doubt than a FIX doubt. Anyone has experienced problems with disabling this QuickFIX socket option? I’m having a problem that the messages are delaying to be acknowledged. And I need to figure out where is the bottleneck.

Thoughts?

3

[ original email was from John Prewett - jprewett@lavatrading.com ]
> Hi all,

This is more a QuickFIX implementation doubt than a FIX doubt. Anyone
has experienced problems with disabling this QuickFIX socket option? I’m
having a problem that the messages are delaying to be acknowledged. And
I need to figure out where is the bottleneck.

Thoughts?

Under most “normal” scenarios, using setsockopt to enable TCP_NODELAY is a very good idea for FIX TCP sessions (no matter whether it is QuickFIX or not).
There is an implication of a double-negative in your question, so I’m not exactly sure which way you are going from/to.
Using a TCP FIX session without TCP_NODELAY enabled can cause occasional delays of up to 500mSec depending on the traffic patterns.
I would recommend that most TCP FIX sessions use TCP_NODELAY enabled.
The possible exception to this might be ultra high speed continuous message streaming such as market data where using TCP_NODELAY enabled could cause many “runt” packets which could conceivably make latency worse.
In general, use TCP_NODELAY enabled for best TCP FIX performance.

I hope this helps.

JohnP

4

Hi,
Can you clarify more on where do you see this delay? Is this delay in acknowledgement from your side or from your client side? We have been using this option for long time and have not seen any issue. You want to look at TCP dump and see if there are many packet resends due to exceeding network latency or anything like that.
Regards,
Ashish

Hi all,

This is more a QuickFIX implementation doubt than a FIX doubt. Anyone
has experienced problems with disabling this QuickFIX socket option?
I’m having a problem that the messages are delaying to be
acknowledged. And I need to figure out where is the bottleneck.

Thoughts?

Under most “normal” scenarios, using setsockopt to enable TCP_NODELAY is
a very good idea for FIX TCP sessions (no matter whether it is QuickFIX
or not). There is an implication of a double-negative in your question,
so I’m not exactly sure which way you are going from/to. Using a TCP FIX
session without TCP_NODELAY enabled can cause occasional delays of up to
500mSec depending on the traffic patterns. I would recommend that most
TCP FIX sessions use TCP_NODELAY enabled. The possible exception to this
might be ultra high speed continuous message streaming such as market
data where using TCP_NODELAY enabled could cause many “runt” packets
which could conceivably make latency worse. In general, use TCP_NODELAY
enabled for best TCP FIX performance.

I hope this helps.

JohnP

5

Hi, Can you clarify more on where do you see this delay? Is this delay
in acknowledgement from your side or from your client side? We have been
using this option for long time and have not seen any issue. You want to
look at TCP dump and see if there are many packet resends due to
exceeding network latency or anything like that. Regards, Ashish

Hi all,

This is more a QuickFIX implementation doubt than a FIX doubt.
Anyone has experienced problems with disabling this QuickFIX socket
option? I’m having a problem that the messages are delaying to be
acknowledged. And I need to figure out where is the bottleneck.

Thoughts?

Under most “normal” scenarios, using setsockopt to enable TCP_NODELAY
is a very good idea for FIX TCP sessions (no matter whether it is
QuickFIX or not). There is an implication of a double-negative in your
question, so I’m not exactly sure which way you are going from/to.
Using a TCP FIX session without TCP_NODELAY enabled can cause
occasional delays of up to 500mSec depending on the traffic patterns.
I would recommend that most TCP FIX sessions use TCP_NODELAY enabled.
The possible exception to this might be ultra high speed continuous
message streaming such as market data where using TCP_NODELAY enabled
could cause many “runt” packets which could conceivably make latency
worse. In general, use TCP_NODELAY enabled for best TCP FIX
performance.

I hope this helps.

JohnP

The delay seems to be at my side, I am delaying on sending a TCP acknowledge for my connection partner (I am the acceptor).

6

[ original email was from John Prewett - jprewett@lavatrading.com ]
Let me just clarify one point:

Enabling TCP_NODELAY needs to be done on both ends of a TCP session in order to be effective. If only one side of a TCP session enables it, there will probably still be some nasty FIX latencies.

This can be problematic if you enable TCP_NODELAY on “your” end but have no control over the “other” end.
A “desperate times need desperate measures” approach to this issue if you cannot get the other end to enable TCP_NODELAY is to disable DelayedAck on “your” end NIC card. This will affect all TCP usage on that NIC card and isn’t settable on a per-TCP session basis (on any operating system I have seen). You will then have solved the latency issue by only touching one end, but there will be many TCP ack-only runt packets now appearing on your network, one for each TCP packet received by your NIC.
This approach is not recommended. It is not for the faint-of-heart. It can get you out of a problem, provided it doesn’t upset the other TCP sessions (or even the problematic one) using that NIC.

JohnP

7

Hi, Can you clarify more on where do you see this delay? Is this delay
in acknowledgement from your side or from your client side? We have
been using this option for long time and have not seen any issue. You
want to look at TCP dump and see if there are many packet resends due
to exceeding network latency or anything like that. Regards, Ashish

Hi all,

This is more a QuickFIX implementation doubt than a FIX doubt.
Anyone has experienced problems with disabling this QuickFIX
socket option? I’m having a problem that the messages are delaying
to be acknowledged. And I need to figure out where is the
bottleneck.

Thoughts?

Under most “normal” scenarios, using setsockopt to enable
TCP_NODELAY is a very good idea for FIX TCP sessions (no matter
whether it is QuickFIX or not). There is an implication of a double-
negative in your question, so I’m not exactly sure which way you are
going from/to. Using a TCP FIX session without TCP_NODELAY enabled
can cause occasional delays of up to 500mSec depending on the
traffic patterns. I would recommend that most TCP FIX sessions use
TCP_NODELAY enabled. The possible exception to this might be ultra
high speed continuous message streaming such as market data where
using TCP_NODELAY enabled could cause many “runt” packets which
could conceivably make latency worse. In general, use TCP_NODELAY
enabled for best TCP FIX performance.

I hope this helps.

JohnP

The delay seems to be at my side, I am delaying on sending a TCP
acknowledge for my connection partner (I am the acceptor).

In that case, you want to enable TCP_NODELAY and see if that solves the issue.

8

Hi all,

This is more a QuickFIX implementation doubt than a FIX doubt. Anyone
has experienced problems with disabling this QuickFIX socket option?
I’m having a problem that the messages are delaying to be
acknowledged. And I need to figure out where is the bottleneck.

Thoughts?

Under most “normal” scenarios, using setsockopt to enable TCP_NODELAY is
a very good idea for FIX TCP sessions (no matter whether it is QuickFIX
or not). There is an implication of a double-negative in your question,
so I’m not exactly sure which way you are going from/to. Using a TCP FIX
session without TCP_NODELAY enabled can cause occasional delays of up to
500mSec depending on the traffic patterns. I would recommend that most
TCP FIX sessions use TCP_NODELAY enabled. The possible exception to this
might be ultra high speed continuous message streaming such as market
data where using TCP_NODELAY enabled could cause many “runt” packets
which could conceivably make latency worse. In general, use TCP_NODELAY
enabled for best TCP FIX performance.

I hope this helps.

JohnP

In normal conditions of the network, does anyone statistic about the performance between “TCP_NODELAY enabled” versus “TCP_NODELAY not enabled”?

Thanks in advance

9

Hi all,

This is more a QuickFIX implementation doubt than a FIX doubt. Anyone
has experienced problems with disabling this QuickFIX socket option?
I’m having a problem that the messages are delaying to be
acknowledged. And I need to figure out where is the bottleneck.

Thoughts?

Under most “normal” scenarios, using setsockopt to enable TCP_NODELAY is
a very good idea for FIX TCP sessions (no matter whether it is QuickFIX
or not). There is an implication of a double-negative in your question,
so I’m not exactly sure which way you are going from/to. Using a TCP FIX
session without TCP_NODELAY enabled can cause occasional delays of up to
500mSec depending on the traffic patterns. I would recommend that most
TCP FIX sessions use TCP_NODELAY enabled. The possible exception to this
might be ultra high speed continuous message streaming such as market
data where using TCP_NODELAY enabled could cause many “runt” packets
which could conceivably make latency worse. In general, use TCP_NODELAY
enabled for best TCP FIX performance.

I hope this helps.

JohnP

Thank you very much JohnP, great help sir.