Imported from previous forum
[ original email was from Bob Lamoureux - blamoureux@bridge.com ]
I noticed that the security working group discussion briefly touched on UDP support. I would like to point out that one of the first design considerations of the IFX protocol was that it be transport independant. That is why encryption is done at the application level. Believe it or not, I have several secure FIX installations around the globe that do NOT use TCP as their transport. I even have a serial-line FIX system in production. So although one FIX security solution may be to implement at the lower layers of the comm stack, it is imperative that there be a solution provided that remains transport independant.
[ original email was from Ryan Pierce - rpierce@taltrade.com ]
> I noticed that the security working group discussion briefly touched on UDP support. I would like to point out that one of the first design considerations of the IFX protocol was that it be transport independant. That is why encryption is done at the application level. Believe it or not, I have several secure FIX installations around the globe that do NOT use TCP as their transport. I even have a serial-line FIX system in production. So although one FIX security solution may be to implement at the lower layers of the comm stack, it is imperative that there be a solution provided that remains transport independant.
>
Striving for transport independence is certainly a good goal. But I would argue that the FIX protocol, as currently designed, becomes unreliable if used over unreliable transports, unless changes are made which violate the FIX specification. Now I’m not saying FIX should require TCP; that’s absurd, as TCP is one of many reliable transports. I’d still consider FIX transport independent among the set of reliable transports.
The application-level crypto which you cited currently depends heavily on the reliability of the transport layer, and the app note defining PGP-DES-MD5 even states so:
"Both data encryption and data integrity checking depend on reliable data transmission between the two parties. If encryption is enabled, then lost or duplicate data will force the session to be restarted. This implies that the FIX data should be sent over a "stream oriented" transport layer, like TCP or X.25. Data transmitted by either party must be delivered in the correct sequence and without any gaps."
I can demonstrate the implicit requirement by using UDP as an example, which has the same four unreliable characteristics as IP, each of which can cause FIX failures.
Any IP datagram may be lost. FIX handles this very well with resend requests and gap fill operations in most cases. Logon / Logout still remain particularly vulnerable to message loss, since they are not protected by heartbeats triggering resends, and they are not allowed to be resent should one party feel the other isn’t responding.
Any IP datagram may be corrupted. FIX’s capability of handling this is less than ideal. The checksum is not robust enough. Any swapping of bytes, or any even number of bit errors in a message occuring in the same bit of their respective bytes will produce a valid checksum. Something like a CRC would be needed to handle reliable detection of transport-level errors. Given that a corrupted message has a 1/256 chance of being mistaken for a valid message, and “55=MSFT” has the same checksum as “55=MFST” and “55=NSFS” the risk of being wrong is too great to trust the FIX checksum alone without the assurance of other underlying guarantees.
IP datagrams may be reordered. FIX can fail quite terribly here if reordering becomes the norm. (In some network architectures, especially with poorly implemented load-balancing, this is the case.) If one side receives 10, 12, 11, that’s fine. That side drops 12, sends a resend request for 11-999999, gets 11, gets 11 poss dupe and 12 poss dupe from the resend request, and all is well. But if this happens regularly, especially if more messages are being sent and reordered while this is happening, the amount of bandwidth and retransmitted messages increases to the point of making the connection unusable. Were FIX to handle reordered messages in stride, it would have to keep 12 around and wait for 11 (much like how TCP handles out-of-order delivery), but that would violate the spec.
Reordering causes even more serious harm to the current PGP-DES-MD5. There, a perfectly transmitted, but out of order, message has a valid FIX checksum and MD5 signature. But the initialization vector for CBC mode DES will be wrong, and the first 8 bytes of the message will decrypt to garbage while the rest looks just fine. And since the sequence number is encrypted, the engine can’t even try to put things back in order before decrypting them.
IP datagrams may be duplicated. This one kills FIX. If you get a two copies of a message, you have no choice but terminate the session. I have seen first-hand packet duplication caused by a malfunctioning router and frame relay PVCs; since the FIX connection used TCP, I was fine, but had I used UDP I would have been dead in the water.
Worse, these kinds of errors don’t happen often with UDP on well-behaved LANs, but once you involve WANs or the Internet, then they do happen, and they are often hard to track down and reproduce. Saying FIX, and especially PGP-DES-MD5 encryption, is currently transport-independent enough to operate reliably over unreliable transports is a myth. It isn’t, and those who do so are either tempting fate or are making necessary kludges which render their engine non-compliant.
Given that FIX itself cannot handle non-reliable transports, and you’d be hard-pressed to find a crypto solution that does (stream ciphers and block chaining are the norm for symmetric cryptography; ECB modes are highly discouraged), I don’t see a problem in pursuing something like SSL/TLS which requires a reliable transport.
This doesn’t mean the death of serial lines. They can be made reliable via several means:
-
Use PPP encapsulation and layer TCP over it
-
Use something like X.25 or one of many other reliable encapsulations
-
Use a protocol like MNP5 or V.42/V.42bis, which are implemented in hardware in almost all telephone modems (and could possibly be implemented in software).
Any of these should be sufficiently reliable for SSL or TLS, and would make unencrypted FIX connections considerably more reliable as well.
[ original email was from Danny Shobrook - dzshobrook@dial.pipex.com ]
Basic UDP:
UDP has a checksum just like TCP, so probs with corruption are not an issue here.
We always send the sequence number outside the encrypted data, so what happens when there is a duplicate packet sent with the same number but no resend flag is we just throw the second one away. Whilst slightly naughty, this does not break any engine on the other side, as ours only receives it doesn’t send. If you really wanted to double check then you could do a check against what the previous message and drop the line as per spec if they differ.
We do it this way because of resource issues for holding a large number of idle TCP connections, but there are other places where TCP is not a solution eg - exchanges possibly using multicasting of new orders added to the book and execution reports, which can only be done natively using UDP.
I think TLS should definitely be an option, its open, its free and it will mean greater long-term adoption of FIX over the current PGP with its export, licensing and support issues. But it clearly cannot be the ONLY solution.
I feel a choice of transport layer encryption, possibly TLS and SOCKS - which does support UDP and I guess a serial line - and application level signatures will be the way forward.