Imported from previous forum
[ original email was from Sreedhar Tulluri - stulluri@paxclearing.com ]
Many of the exchanges are either extending FIX protocol ( by adding custom tags) or changing the interpretation of messages or overloading messages to create a FIX Interface to their proprietary trading engines.
This approach is causing lot of work and confusion everytime an exchange comes with another version of their trading platform or upgrading to newer FIX Protocol. Is there an entity in FIX protocol Org to monitor FIX Protocol implementations of different FIX servers across many ECN’s/Exchanges ?
[ original email was from Ryan Pierce - rpierce@taltrade.com ]
> Many of the exchanges are either extending FIX protocol ( by adding custom tags) or changing the interpretation of messages or overloading messages to create a FIX Interface to their proprietary trading engines.
> This approach is causing lot of work and confusion everytime an exchange comes with another version of their trading platform or upgrading to newer FIX Protocol. Is there an entity in FIX protocol Org to monitor FIX Protocol implementations of different FIX servers across many ECN’s/Exchanges ?
You’re talking about an issue inherent to our industry, not one specific to the FIX protocol. Exchanges, ECNs, ATSs, etc. are all competing fiercely, and among other things, one way they can distinguish themselves is through offering more features and more control over order execution.
If FIX were not extensible, then nobody would use it. Protocols exist to serve business, and not the other way around. If an exchange believes adding a new order type will increase market share and differentiate itself from its competitors, it’s going to do so no matter what. Either the FIX protocol is extended to support the new feature, or the exchange will have no choice but drop FIX for a proprietary protocol, which would do far more damage in fragmenting the industry.
That said, there are good ways and bad ways to extend the FIX protocol.
The best thing to do is work directly with FPL to see that the new functionality is incorporated in the next FIX spec. In general, FPL has been very open towards this, provided that the feature is something that can be expressed in general terms and is not something specific to one market only.
Some firms will pull features from future FIX versions into previous versions. This is something of a gray area. It technically isn’t FIX compliant. However, if given a choice between supporting a proprietary field in FIX versions <= X and a standard field in FIX versions > X, vs. using the same field in all versions, I’d gladly do the latter, since it greatly simplifies the upgrade path.
The next best way of extending FIX is through the use of proprietary fields. This shouldn’t be a first approach if the feature is something that could be implemented across several exchanges / ECNs / ATSs. (For example, discretionary pricing, introduced in FIX 4.2).
The worst thing to do is overriding existing well-defined functionality with something entirely different.
[ original email was from Sreedhar Tulluri - stulluri@paxclearing.com ]
- Your are absolutely right about extensibility of FIX protocol and implementing exchange specific features. I think FIX is lacking the concept of ‘Generic’ message definition (abstract messages) that could help firms define concrete messages without having to add them to FIX protocol.
A classic example is BOX (Boston options exchange), which defined new messages completely
out of FIXionary (PIP,Direct Orders etc.) resulting in upgrades to all FIX servers
supporting BOX.
- Extensibility is often misused by the virtue of bad implementation.
Many FIX servers override the protocol and implements proprietary functions.
This pattern can be seen often when exchanges tries to create a facade of their
trading platforms with FIX protocol. Instead of implementing adapters between
FIX & proprietary SDK’s, they override FIX thus exposing internal workings of their
trading engine. A typical example is changing message sequence of cancel/replace in CBOE FIX 4.2
implementation or resend request messages in CME iLink2.0- resulting in FIX-look alikes not FIX.
[ original email was from Sam Johnson - sam.johnson@transacttools.net ]
This is a topic that’s near and dear to me so I thought I’d offer a couple comments:
The success of FIX can be directly attributed to its flexibility and extensibility. As Ryan pointed out, firms compete for marketshare by offering differentiated services; therefore, a homogeneous least-common-denominator messaging interface won’t cut it.
FPL does a very good job of catching the spec up to with common market practices each new release. This model works. Somebody innovates by extending the standard, their extensions get tweaked and refined with ongoing testing and usage, as they mature they are included in the standard specification and the standard’s coverage grows. The spec doesn’t hold anyone back.
There is not a thing wrong with this. I laugh every time someone complains that non-standard messages or fields “break” a fix engine. Support for this has been a basic industry requirement for quite a while now, if your vendor isn’t providing you with a simple way to accomodate this (that doesn’t require an upgrade) then you need to find a new vendor.
The FIX session layer is mature. It changes minutely when it changes at all from version to version. This is the piece that shouldn’t get altered. For reliable delivery of transactions, it works well.
There is some really good news on the extensibility side, but it’s a ways off. The work that the FIXML Schema Group (Jim Northey) has done recently is very exciting. Once finalized, and then adopted by the market, it will make extensions and customizations much easier for systems to recognize, validate, and process. There are other benefits to xml and the schema approach that these guys have taken too. But adoption will take some time, so for now we’re going to have to rely on fix engines that are smart enough to support increasing – and increasingly creative – customization.
[ original email was from Ryan Pierce - rpierce@taltrade.com ]
While I agree with almost all of what Sam says, I do need to comment on one part:
> The FIX session layer is mature. It changes minutely when it changes at all from version to version. This is the piece that shouldn’t get altered. For reliable delivery of transactions, it works well.
I hope you’re not saying that FPL should not make improvements to the session layer. I agree that it is very mature, and that haphazard changes should not be made, but I have a hard time believing that it is perfect.
For example, one noticeable change that I believe needs to be made is the handling of binary data field encoding. Having to keep all FIX engines up to date on the pair of tags for length and data is, in my view, a danger. Therefore, I’ve suggested during FIX 4.3’s development that the length fields should be dropped, and we use something like:
TAG:LENGTH=VALUE[SOH]
(Note that ONLY binary fields would be encoded this way.)
This tells an engine immediately that a field is binary, without the engine having to keep a special list. Otherwise, an out of date list or a user-defined binary field results in the engine choking when it first encounters an embedded [SOH].
Unfortunately, when FIX 4.4 rolled around, the consensus I heard was that it was still a good idea, but nobody talked about it between 4.3 and 4.4, so it couldn’t be added to 4.4. Therefore, in hopes of seeing this in FIX 4.5, I am subjecting everyone to this random non-sequitur at every opportunity. 
[ original email was from Jim Northey - jnorthey@lasalletech.com ]
Was enjoying reading the discussion - so didn’t want to weigh in right away. Good stuff was captured in this thread.
One of the agenda items proposed for the global derivatives committee for next year is to start applying "gentle persuasion" and assistance and education to derivative market places regarding more standardization in terms of FIX. Our derivatives area is one of the most fragmented in terms of implementation approaches across all the asset classes. Much of the work that Dan Doscas, Dean Kauffman, Lisa T, Sam, and the other folks on the Fixed Income committee are doing is to prevent the market place fragmentation that has occurred in other asset classes, especially listed derivatives.
A great deal of this fragmentation can be attributed to what Ryan described as the fierce competition and market place innovation and differentiation that is taking place. What an exciting and difficult time to be involved in the derivatives markets. Some of it also can be attributed to having to adapt FIX to legacy systems and market practices. I was directly involved in CBOE’s decision to digress from standard FIX semantics. I supported the digression. The reason was that the existing systems and the existing market place owners and existing market place users were used to and wanted the existing CMS style cancel replace semantics. We would have had to have retrained literally thousands of people on how to think and react to cxl/re requests. We would have also had to spend a large some of money making modifications to existing systems. Instead, we worked very hard to make sure the inbound messages were FIX compatible, but the resulting responses and the loss of order history after a cancel/replace differs from FIX. In 1999 after announcing FIX 4.2 Ryan nd I sketched out a simple solution for maintaining FIX style and CMS style order handling and state side by side. We chose not to implement this “emulation” of FIX semantics in order to prevent the potential chaos and lack of communication between how the order was handled at the exchange and how the user viewed the order externally. Regarding support for the differences - we document each order state transition matrix that differed from the FIX standards to show exactly the differences in behavior to help users contend with the variance from the standard order handling semantics. (For another perspective - someone also mentioned to me recently that CMS style order handling has been a de facto industry standard for 30+ years - thus trumping FIX’s standardization legitimacy 
).
Since the time we chose to differ from the FIX order handling adoption has spread. We are hearing from many folks that having to accomodate non-FIX cxl/re behavior creates a burden on CBOE users. Because of intensive competition - market places have to listen to these concerns - and believe me they have been heard and are being taken very seriously.
Back to reasons for derivatives market fragmentation: Some of the fragmentation results from differing market making models. This is going to probably continue for awhile. The FPL organization, working in conjunction with our partners such as the FIA Standards Working Group, need to provide encouragement, guidance, and leadership in proactively engaging the exchanges in dialogues in terms of common implementation. I believe that market innovation and differentiation can still occur behind the interface - what we need to do is to help standardize the interface to lower barriers to entry to markets - this will in turn help increase market place’s ability to compete.
Some of the fragmentation has occured due to pushing the limits of the FIX protocol for high volume derivatives market making applications. For example, CME added an enhancement for quicker convergence in the event of resends. While not officially part of the standard, it is an application note and is an optional usage feature we reviewed at the technical committee level. True, engine vendors and custom FIX implementations do have to account for this feature - but the value of the feature outweighs the cost - especially as message rates start to move into the 5,000 to 10,000 aggregate message per second at the larger derivatives exchanges.
At the CME, they also have modified order handling semantics because they use orders to create markets (futures market style as opposed to US options model). We have been encouraging them to look at the quoting and mass quoting models in 4.3 and 4.4 - as being superior - even if those messages result in orders being generated within their trading systems. We think this is now a proven approach that would permit a more standard implementation.
So, while recognizing that much of the success of FIX has been based upon its extensibility and the cultural bias to adopt new innovations in a timely manner, we still have work we can do as a community to encourage further standardization and commonality at the interface to derivatives markets. It is a journey not a destination…
Jim
> This is a topic that’s near and dear to me so I thought I’d offer a couple comments:
>
> The success of FIX can be directly attributed to its flexibility and extensibility. As Ryan pointed out, firms compete for marketshare by offering differentiated services; therefore, a homogeneous least-common-denominator messaging interface won’t cut it.
>
> FPL does a very good job of catching the spec up to with common market practices each new release. This model works. Somebody innovates by extending the standard, their extensions get tweaked and refined with ongoing testing and usage, as they mature they are included in the standard specification and the standard’s coverage grows. The spec doesn’t hold anyone back.
>
> There is not a thing wrong with this. I laugh every time someone complains that non-standard messages or fields “break” a fix engine. Support for this has been a basic industry requirement for quite a while now, if your vendor isn’t providing you with a simple way to accomodate this (that doesn’t require an upgrade) then you need to find a new vendor.
>
> The FIX session layer is mature. It changes minutely when it changes at all from version to version. This is the piece that shouldn’t get altered. For reliable delivery of transactions, it works well.
>
> There is some really good news on the extensibility side, but it’s a ways off. The work that the FIXML Schema Group (Jim Northey) has done recently is very exciting. Once finalized, and then adopted by the market, it will make extensions and customizations much easier for systems to recognize, validate, and process. There are other benefits to xml and the schema approach that these guys have taken too. But adoption will take some time, so for now we’re going to have to rely on fix engines that are smart enough to support increasing – and increasingly creative – customization.
>
>