Imported from previous forum
[ original email was from george kledaras - george@javtech.com ]
The reason why FIX is so succesful is that it completely defines communication from application message formats to the session, even implying TCP/IP.
FIXML formatted messages are a great step forward, especially becuase it lets business analyst types to extend the protocol without regard to the session layer. But currently the FIXML initiative uses the FIX session to trade over a wide area network.
The FIX session has some missing functionality that should be addressed. Here are some FIXML session requirements I have put together. Overall it is meant to simplify and add functionality to FIXML:
-
easy to understand so it lowers costs and is readable from log files.
-
can be implemented by any programming language on any operating system.
-
no heartbeats.
-
no limit to message sequence numbers.
-
handles redundant communication lines to support high availability scenarios.
-
must be streams based; the first few bytes represent the length of a message.
-
must not be limited by any specific length to support batches of information and program trading.
-
efficient in terms of network speed; once basic information is sent, deltas (changes) are sent making it much easier to support high volumes of information.
-
easy to parse and check for basic information like data integrity, message format, routing, error recovery, and security before it is passed on to the application for processing.
-
allow sessions to be created from client to client, not just server to server. This may eliminate the need for a central FIX server as a bottleneck. It also will provide better security.
-
acknowledgement of each message to ensure delivery. Acknowledgement all the way to the client application.
-
have built in security that is easy to implement, does not rely on 3rd party tools, and is generally license friendly.
-
provide data security for many years, not just a few days or hours.
-
allow users to plug in their own proprietary security encryption and authentication schemes instead of using well known ones.
[ original email was from John Goeller - john.goeller@ssmb.com ]
We debated this issue quite a bit in the early days of FIXML. During our discussions, Avneet
Sawhney (Morgan Stanley at the time) even went so far as to create a session-level DTD. Here is an
XML example:
<?xml version=‘1.0’?>
<!DOCTYPE FIXML SYSTEM ‘fixmlmain.dtd’
>
<FIXML>
<FIXMLMessage>
<Header BodyLength=“100” MsgType=“A” MsgSeqNum=“1”>
<Sender>
<CompID>Some text</CompID>
</Sender>
<Target>
<CompID>Some text</CompID>
</Target>
</Header>
<AdminMessage>
<Logon EncryptMethod=“1” HeartBtInt=“10”>
<RawData>Some text</RawData>
</Logon>
</AdminMessage>
<Trailer>
<Signature >Some text</Signature>
<CheckSum>Some textSome text</CheckSum>
</Trailer>
</FIXMLMessage>
</FIXML>
His interest was to create a mechanism which would allow interested parties to use FIXML without the tight coupling between the application and session layers. He wanted to enable an environment which didn’t require a traditional FIX engine at both ends of the transmission. The key issue was to focus on the data and not worry about the pipe. FIXML over HTTP was one example. In other words, see how FIXML could be used without any session level dependencies.
We felt at the time that using XML in the Session layer was out of scope. We thought we had
our hands full with the "XMLization" of FIX business messages without trying to figure out how to use FIXML over alternative delivery channels. The FIX session guarantees a reliable, real-time connection that for most people works pretty well.
Our challenge in introducing FIXML has always been that there was already a FIX tag-value format. We always felt that FIXML should be an evolutionary not a revolutionary process. Our pilot focused on moving FIXML over the existing FIX session because we needed to provide a migration path for those folks interested in
using FIXML with their existing FIX applications.
That said, I think the FIX Technical Committee has always been willing to listen to valid proposals that will help to improve the protocol. I’d be willing to discuss these items with you further to clarify some of your points. I believe some of these issues are currently being addressed In the XML space through a number of initiatives like BizTalk and the EcoFramework.
>
> The reason why FIX is so succesful is that it completely defines communication from application message formats to the session, even implying TCP/IP.
>
> FIXML formatted messages are a great step forward, especially becuase it lets business analyst types to extend the protocol without regard to the session layer. But currently the FIXML initiative uses the FIX session to trade over a wide area network.
>
> The FIX session has some missing functionality that should be addressed. Here are some FIXML session requirements I have put together. Overall it is meant to simplify and add functionality to FIXML:
>
> - easy to understand so it lowers costs and is readable from log files.
>
> - can be implemented by any programming language on any operating system.
>
> - no heartbeats.
>
> - no limit to message sequence numbers.
>
> - handles redundant communication lines to support high availability scenarios.
>
> - must be streams based; the first few bytes represent the length of a message.
>
> - must not be limited by any specific length to support batches of information and program trading.
>
> - efficient in terms of network speed; once basic information is sent, deltas (changes) are sent making it much easier to support high volumes of information.
>
> - easy to parse and check for basic information like data integrity, message format, routing, error recovery, and security before it is passed on to the application for processing.
>
> - allow sessions to be created from client to client, not just server to server. This may eliminate the need for a central FIX server as a bottleneck. It also will provide better security.
>
> - acknowledgement of each message to ensure delivery. Acknowledgement all the way to the client application.
>
> - have built in security that is easy to implement, does not rely on 3rd party tools, and is generally license friendly.
>
> - provide data security for many years, not just a few days or hours.
>
> - allow users to plug in their own proprietary security encryption and authentication schemes instead of using well known ones.
>
>
[ original email was from Ryan Pierce - rpierce@taltrade.com ]
> - no heartbeats.
I don’t see how this is a feature.
I don’t believe that heartbeats should be defined as a part of the FIX application layer, and they aren’t. Someone using FIXML over an alternate session layer, such as one of many messaging middleware products, is not sending heartbeats.
However, any underlying reliable session protocol really needs to support heartbeats of some kind. I’ve done work with several financial protocols other than FIX, and they almost always have some form of heartbeat. Knowing whether the other side of a connection is alive or dead is extremely important, and just waiting for a TCP socket to close is not reliable.
> - no limit to message sequence numbers.
I believe that these limits have been removed from FIX 4.2, and Resend Requests from X to 999999 will instead be represented as being from X to -1
> - efficient in terms of network speed; once basic information is sent, deltas (changes) are sent making it much easier to support high volumes of information.
Are you talking about sending deltas at an application layer, or are you talking about using a compression algorithm at the session layer?
The issue of deltas at an application layer, i.e. only sending fields that change from message to message, is rather complex. Some messages, like the Book draft which has become the Market Data draft, make efficient use of deltas. But I would imagine any firm’s compliance department would have a heart attack at the thought of accepting an order message without a symbol, with the implication that the order was for the last symbol the customer traded.
Now I don’t see why XML messages can’t be passed through one of many general data compression algorithms. FIX messages have a good deal of repetition, and FIXML in particular has quite a bit more; I would imagine that FIXML would compress rather nicely.
> - easy to parse and check for basic information like data integrity, message format, routing, error recovery, and security before it is passed on to the application for processing.
This is problematic because FIXML uses DTDs instead of schemas, and DTDs have much less flexibility in terms of data validation. Of course, as was stated, FIXML can’t really use schemas at this time because no one schema standard has emerged and parser support is limited.
> - have built in security that is easy to implement, does not rely on 3rd party tools, and is generally license friendly.
>
> - provide data security for many years, not just a few days or hours.
>
> - allow users to plug in their own proprietary security encryption and authentication schemes instead of using well known ones.
The general thought of the Encryption working group, at its last meeting (which I think was over a year ago) had been to wrap the FIX session in SSLv3 or TLS (the IETF standardized version of SSL.) This seems to meet the requirements. It is easy to implement because it fits in rather transparently below the session-layer (i.e. replacing your socket read, write, and connect calls with SSL calls), and unlike PGP-DES-MD5, it doesn’t require the application to parse the message and them pass part on to an encryption engine. The security is strong (128-bit). The standard is published as an RFC, so anyone can code an implementation, buy an implementation, or use a free implementation (I think at www.openssl.org). Note that patent issues may apply to some crypto algorithms. TLS also allows users to define their own cipher suites, and both SSL and TLS make use of server and client certificats, and PKI.