Imported from previous forum
If I understand correctly, only FIX on TCP is being considered for this whitepaper. We would NOT be looking at FIX engine architectures for other transports like MQ or WebServices as part of FIXT.1.1 performance improvements. Please correct me if I have misunderstood the scope.
Regards,
K. Mahesh
[ original email was from John Prewett - jprewett@lavatrading.com ]
> If I understand correctly, only FIX on TCP is being considered for this
whitepaper. We would NOT be looking at FIX engine architectures for
other transports like MQ or WebServices as part of FIXT.1.1 performance
improvements. Please correct me if I have misunderstood the scope.Regards,
K. Mahesh
I do not believe we are necessarily restricting ourselves to TCP-only implementations. While FIX is predominantly implemented over TCP doesn’t preclude some people from contributing ideas to reduce latency over other lower-level (from a FIX point-of-view) transport layers. All that would happen is that we would probably have a new chapter or section related to good usage for them. I would really like to see a chapter related to FIX over UDP and/or multicast.
If I understand correctly, only FIX on TCP is being considered for
this whitepaper. We would NOT be looking at FIX engine architectures
for other transports like MQ or WebServices as part of FIXT.1.1
performance improvements. Please correct me if I have misunderstood
the scope.Regards,
K. MaheshI do not believe we are necessarily restricting ourselves to TCP-only
implementations. While FIX is predominantly implemented over TCP doesn’t
preclude some people from contributing ideas to reduce latency over
other lower-level (from a FIX point-of-view) transport layers. All that
would happen is that we would probably have a new chapter or section
related to good usage for them. I would really like to see a chapter
related to FIX over UDP and/or multicast.
[start quote from FIXML discussion forum]
http://fixprotocol.org/discuss/read/37b9f6f3
In theory, any transport mechanism, reliable or unreliable, could be used for FIX. In practice, FIX’s recovery mechanisms aren’t robust enough to maintain business connectivity over an unreliable transport mechanism like UDP or a noisy modem connection with no error correction.
So in practice, almost all FIX is run over TCP, which is a reliable transport protocol.
[end quote from FIXML discussion forum]
Are there any production FIX over UDP etc?
[ original email was from Andrei Goldchleger - agoldchleger@bmf.com.br ]
> > > If I understand correctly, only FIX on TCP is being considered for
this whitepaper. We would NOT be looking at FIX engine architectures
for other transports like MQ or WebServices as part of FIXT.1.1
performance improvements. Please correct me if I have misunderstood
the scope.Regards,
K. MaheshI do not believe we are necessarily restricting ourselves to TCP-only
implementations. While FIX is predominantly implemented over TCP
doesn’t preclude some people from contributing ideas to reduce latency
over other lower-level (from a FIX point-of-view) transport layers.
All that would happen is that we would probably have a new chapter or
section related to good usage for them. I would really like to see a
chapter related to FIX over UDP and/or multicast.[start quote from FIXML discussion forum]
http://fixprotocol.org/discuss/read/37b9f6f3
In theory, any transport mechanism, reliable or unreliable, could be
used for FIX. In practice, FIX’s recovery mechanisms aren’t robust
enough to maintain business connectivity over an unreliable transport
mechanism like UDP or a noisy modem connection with no error correction.So in practice, almost all FIX is run over TCP, which is a reliable
transport protocol.[end quote from FIXML discussion forum]
Are there any production FIX over UDP etc?
I though that the FIX session layer existed to cover the limitations of using a unreliable transport, such as UDP. Of course, if you experience 50% packet drop in the network, business performance will suffer, but the information will eventually reach the counter-party.
I am not aware of UDP usage for FIX order entry, but then, I am only aware of a tiny fraction of the trading systems out there. In most cases, this could also be due to inertia: people tend to use what is most commonly used, thus, TCP is prevalent.
[ original email was from Ryan Pierce (FPL Technical Director) - ryan.pierce@fixprotocol.org ]
> I though that the FIX session layer existed to cover the limitations of
using a unreliable transport, such as UDP. Of course, if you experience
50% packet drop in the network, business performance will suffer, but
the information will eventually reach the counter-party.
In theory, yes, the FIX session layer should be able to do this, but in practice, the FIX session layer is not robust enough to handle delivery over an unreliable transport.
I’ve seen unreliable transport defined as:
- Packets can be dropped
- Packets can be corrupted
- Packets can be reordered
- Packets can be duplicated
FIX addresses case 1 adequately through resend requests, although in the case of lots of messages in flight at once, multiple resend requests can be issued, which will slow things down.
Case 2 is, in theory, addressed by FIX’s checksum. In practice, the checksum is not adequate protection. Reordered or transposed bytes generate the same checksum, yet 55=MSFT becoming 55=MFST would be catastrophic, and the FIX checksum test in the engine will not catch it. Further, the checksum has only 256 possible values, so the likelihood that line noise corrupting multiple bytes is going to generate, by random chance, a corrupt message that still matches the checksum, is 1/256. If line noise is a persistent problem, then, statistically, there will be messages a FIX engine may think are valid, when they have been corrupted. TCP, on the other hand, uses something far more robust (32 bit CRC, maybe?) that can detect corruption easily.
Case 3 can be addressed by FIX, but very inefficiently. It is entirely possible that there are redundant network paths between parties, and that round-robin delivery is used. TCP doesn’t have a problem with this. If packets arrive out of order, TCP will hold them until there is a consecutive group of them, and present them in order to the application. TCP also assumes this behavior will happen, so it doesn’t go nuts with the equivalent of resend requests should packets arrive out of order. FIX, on the other hand, will go nuts. Sending 1 2 3 4 and having it received as 1 3 2 4 will likely turn into:
1 processed
3 dropped and resend request 2-0 issued
2 processed
4 dropped and resend request 3-0 issued
Resent 2 dropped
Resent 3 processed
Resent 4 processed
Resent 3 dropped
Resent 4 dropped
If packet reordering is the norm, not the exception, there’s no way a FIX session would be able to stay up.
Case 4, duplicated packets, will obviously result in the session terminating.
Bottom line: In theory, FIX may claim to operate over unreliable transport. In practice, this isn’t possible if the transport is truly unreliable. You can’t switch from TCP to UDP and expect FIX to work. Or, it may work on a LAN, but going to a WAN may suddenly result in the FIX session failing. Something that guarantees reliable, ordered delivery, like TCP, is needed. Even modems with hardware error correction may work. But truly unreliable transport won’t work.
Now I have heard of FIX for market data multicast over UDP. In this case, the FIX session is heavily modified, and ordered messaging rules are relaxed, often with an out-of-band resend mechanism employed. This addresses cases 1, 3 and 4. And Ethernet itself carries a CRC of sorts, so message corruption isn’t likely, which addresses case 2. [Note: I have seen a case of a bad router that corrupted data but then regenerated valid Ethernet CRCs on the corrupted data, so the receiver’s NIC thought the packets were valid, but TCP didn’t like them one bit.]
FIX over other transports and comparision of different transports should be part of the work of “GTC Transport Subcommittee” at
http://fixprotocol.org/committees/transports
since FIX over other transports is not yet mature enough to be compared and optimized. There arent any “production” FIX over other transports, I have been FIXperimenting with alternate “Transport Independence Framework(s)” at
http://fixprotocol.org/discuss/49
(Most posts in the above forum are by me 
Regards,
K. Mahesh