Imported from previous forum
Fix has become a great success. My analyze result is: because it is easy, simple and efficient. At least this accounts for the fix version up to 4.4. We can plug in and out markets and counterparties at a fraction of the cost and time needed compare implementing unique protocols.
There is however still missing a real “Fast Fix”. That still is as simple and efficient as fix 4.4, but simply faster. Most FIX engines designed was design for trade reporting between brokers, and are simply not fast because no one ever tried to make them fast. I would like to replace all this silly unique protocols to each market with FIX, but latency is the thing that always pops up as the argument against this.
There is also FIX versions greater than 4.4. 5.0 etc. But, is anyone really using it? Our vendors claims, and I understand them, that there is a great deal of work moving from 4.4 to 5.x, and no one can motivate why this should be done. Simply adding some custom fields or implement a few fix 5.x messages in FIX 4.X seam more efficient.
It seems that FIX is making the same journey as HTML. From simple text, a little tags here and there, to complicated XML:s with schemas etc.
I don’t say this is the wrong journey, but I simply don’t understand it from a business point of view. As in last project, we have to move from FIX 4.4 to FIX 5.X to get “settlement date” in execution reports from our counterparty. Must be someone that freaked out somewhere? It can’t be an efficient decision, compare adding an existing field to the old solution.
Looking at the FIX Functionality Matrix, it seems there are only Parties- and Market structure data and some compliance functions that are non existing in 4.4 but Exists in 5.x Is that the only reason from moving from 4.4 to 5.X?
Maybe there is a clear line, a border, at FIX 4.4, where the idea of general and easy stops, and unique and specialized starts?
/ Jan
Jan,
it is hard to give a short answer to this. Thank you for your valuable comments which point to an issue that FPL is well aware of and has started to tackle recently.
The apparent perception of FIX being slow per se and the transition from FIX 4.x to FIX 5.x being a significant change is something that is being addressed by the FIX Global Technical Committee.
There are a number of aspects that impact the performance of a FIX interface, some of which should not be attributed to the FIX protocol.
Encoding
FIX currently offers tag=value (ASCII), FIXML (ASCII) and FAST (binary). Binary is the encoding to chose for high performance and FAST has only been used (very successfully) for market data so far. The extension of FAST to transactions is pretty straightforward and something being investigated. FIX 5 already allows to use a different encoding on the basis of transport independence. However, other than FAST, FIX has not come up with a standard binary encoding yet.
Verbosity
FIX is know to be fairly explicit in its responses to requests, e.g. requiring to echo a lot of information on Execution Report messages. High performance applications typically only need to know if their request succeeded or failed. Additional, unsolicited messages inform about fills or unsolicited changes. This area is more about guidelines than actual changes to FIX. Very few fields are formally required in a message. Verbosity also applies to the number of messages required and FIX has provided some guidelines with FIX 5 to cut down in this area, e.g. a single Execution Report instead of 3 to report back on an IOC order that was partially filled upon entry. This has more to do with usage and not with the message layouts, i.e. FIX already has everything needed for high performance, we just need to write down the rules more precisely.
Semantics
By far the biggest impact on performance is the area of semantics/concepts/paradigms where FIX offers standards for many key areas such as entity identification (orders, quotes, trades), message flows and error handling. The bad news is that electronic marketplaces have not designed their systems to follow the semantics established by FIX. The consequence is that FIX gateways are almost always only offered as an add-on to the proprietary interface which means that it can never be faster to begin with. The difference in transaction models (how many FIX vs proprietary messages for a given business transaction) between FIX and the core system often demand a stateful implementation of the FIX interface. This means big databases, lots of lookups, complex recovery/reconciliation mechanisms and lots of time wasted to align one’s proprietary concepts with the standard concepts of FIX. These are the real issues leading to user-defined messages, fields and valid values in order to tweak FIX towards one’s own concepts instead of giving the FIX concepts top priority. Here is where the many FIX dialects come in that cause people headaches. Some of it can be attributed to a lack of knowledge of the FIX concepts. This forum is one of the places where people can get clarification and more people should come forward during the design of their interfaces to get it right or to seek an extension to the standard where there are actual gaps. Often enough it is just a difference in terminology and FIX cannot be the superset of everybody’s terminology, the underlying business requirements are very comparable, people just do not always recognize this.
Your second issue was about the transition to FIX 5. I would really like to see a paper describing the key issues that people have encountered. There seems to be no issue with market data where FIX 5 is being used more often than not. Is it about the session layer? It cannot be about the functional concepts of FIX as there were more changes between FIX 4.2 and FIX 4.3.
As said before, FPL is well aware of this issue and we need to find a way to remove the negative perception which I believe is not justified. This is firmly on the agenda. The functional richness of FIX 5.0 and its many, many Extension Packs (116 in total up to now) has significantly increased the level of functional coverage with respect to trading on exchanges and other electronic marketplaces, including OTC trading. Areas such as reference data have minor coverage with FIX 4.x compared to today. FIX 4.2 had a total of 426 fields. We are now approaching 1700, i.e. a fourfold increase. It is still almost twofold when compared with FIX 4.4. This should tell you how much FIX has developed since FIX 4.4 which is already over seven years old.
In conclusion, I would like to reiterate my request for information on the key pain points preventing the transition to FIX 5. Is it merely a chicken and egg problem or is FIX 5 technically inferior? FIX interface development used to be a highly manual process. Since a few years ago, there is a FIX Repository that supports automated development and should make transitions a lot easier.
I look forward to more feedback on this topic to help FPL move into the right direction in supporting the community of FIX users.
Regards,
Hanno.
Fix has become a great success. My analyze result is: because it is easy, simple and efficient. At least this accounts for the fix version up to 4.4. We can plug in and out markets and counterparties at a fraction of the cost and time needed compare implementing unique protocols.
There is however still missing a real “Fast Fix”. That still is as simple and efficient as fix 4.4, but simply faster. Most FIX engines designed was design for trade reporting between brokers, and are simply not fast because no one ever tried to make them fast. I would like to replace all this silly unique protocols to each market with FIX, but latency is the thing that always pops up as the argument against this.
There is also FIX versions greater than 4.4. 5.0 etc. But, is anyone really using it? Our vendors claims, and I understand them, that there is a great deal of work moving from 4.4 to 5.x, and no one can motivate why this should be done. Simply adding some custom fields or implement a few fix 5.x messages in FIX 4.X seam more efficient.
It seems that FIX is making the same journey as HTML. From simple text, a little tags here and there, to complicated XML:s with schemas etc.
I don’t say this is the wrong journey, but I simply don’t understand it from a business point of view. As in last project, we have to move from FIX 4.4 to FIX 5.X to get “settlement date” in execution reports from our counterparty. Must be someone that freaked out somewhere? It can’t be an efficient decision, compare adding an existing field to the old solution.
Looking at the FIX Functionality Matrix, it seems there are only Parties- and Market structure data and some compliance functions that are non existing in 4.4 but Exists in 5.x Is that the only reason from moving from 4.4 to 5.X?
Maybe there is a clear line, a border, at FIX 4.4, where the idea of general and easy stops, and unique and specialized starts?
/ Jan
Thanks,
I noticed that http://www.quickfixengine.org/ has updated their engine to support FIX 5.0. Since we use it for all internal applications (basically only fix-drop-copy for receiving execution reports), I guess our problems migrating are small compare to all-in-house development. However the problem with vendors not upgrading is harder, when functions needed is only supported in some versions and upgrade on the other vendor is not easily forced. Since fragmentation and all this markets and connections everywhere, we must be close to a point there connectivity will become a commodity there competition is only on price and maybe performance. As you said, terminology may differ, but I doubt there is any missing functionality in FIX.
I think (hope?) we will see development in these faces
-
Some smart vendor start offering gates to all markets that converts the unique protocol of each market to a standard protocol, such as FIX, with a binary fast low latency version available. Tricky part is most likely to offer a good solution for structure data (instruments, list, coding such as RIC, ISIN, however this can be solved with advance solutions like daily file download via FTP)
-
More vendors start to offer this, and we have competition, bringing the price down. With a price for a gate today varying from 2000-9000 euro’s a month, there is for sure some lock-in by vendor in process, and some lack of competition (maybe lack of standard is the main cause?)
-
Market them self starting to offer this standard, bypassing the 3:d party, the vendor. Now we are close to market efficiency!
What I am talking about is that we are really still competing on a very low level. Like if we should use 110 or 220V power supply.
Some people will argue about latency, but, only once can be the fastest. And if markets only offer standard protocol, they can still try to implement this standard protocol and trim their computers, algos etc and try to be the fastest. However, if markets offer a special protocol, slightly faster than standard protocols, all prop-trades are doomed and have to implement the special unique protocol.
I don’t think there is some technical reason why not a standard protocol can be fast enough.
For brokers with real volume (not prop), it is more important to time there orders to hit different market at the same time (the pegs won’t have time to act)
I think we will see dramatically changes the next 1-3 years . I just hope the standard will come soon, only technical providers and consults benefits for lack of standard
/ Jan
Jan,
it is hard to give a short answer to this. Thank you for your valuable comments which point to an issue that FPL is well aware of and has started to tackle recently.
The apparent perception of FIX being slow per se and the transition from FIX 4.x to FIX 5.x being a significant change is something that is being addressed by the FIX Global Technical Committee.
There are a number of aspects that impact the performance of a FIX interface, some of which should not be attributed to the FIX protocol.
Encoding
FIX currently offers tag=value (ASCII), FIXML (ASCII) and FAST (binary). Binary is the encoding to chose for high performance and FAST has only been used (very successfully) for market data so far. The extension of FAST to transactions is pretty straightforward and something being investigated. FIX 5 already allows to use a different encoding on the basis of transport independence. However, other than FAST, FIX has not come up with a standard binary encoding yet.
Verbosity
FIX is know to be fairly explicit in its responses to requests, e.g. requiring to echo a lot of information on Execution Report messages. High performance applications typically only need to know if their request succeeded or failed. Additional, unsolicited messages inform about fills or unsolicited changes. This area is more about guidelines than actual changes to FIX. Very few fields are formally required in a message. Verbosity also applies to the number of messages required and FIX has provided some guidelines with FIX 5 to cut down in this area, e.g. a single Execution Report instead of 3 to report back on an IOC order that was partially filled upon entry. This has more to do with usage and not with the message layouts, i.e. FIX already has everything needed for high performance, we just need to write down the rules more precisely.
Semantics
By far the biggest impact on performance is the area of semantics/concepts/paradigms where FIX offers standards for many key areas such as entity identification (orders, quotes, trades), message flows and error handling. The bad news is that electronic marketplaces have not designed their systems to follow the semantics established by FIX. The consequence is that FIX gateways are almost always only offered as an add-on to the proprietary interface which means that it can never be faster to begin with. The difference in transaction models (how many FIX vs proprietary messages for a given business transaction) between FIX and the core system often demand a stateful implementation of the FIX interface. This means big databases, lots of lookups, complex recovery/reconciliation mechanisms and lots of time wasted to align one’s proprietary concepts with the standard concepts of FIX. These are the real issues leading to user-defined messages, fields and valid values in order to tweak FIX towards one’s own concepts instead of giving the FIX concepts top priority. Here is where the many FIX dialects come in that cause people headaches. Some of it can be attributed to a lack of knowledge of the FIX concepts. This forum is one of the places where people can get clarification and more people should come forward during the design of their interfaces to get it right or to seek an extension to the standard where there are actual gaps. Often enough it is just a difference in terminology and FIX cannot be the superset of everybody’s terminology, the underlying business requirements are very comparable, people just do not always recognize this.
Your second issue was about the transition to FIX 5. I would really like to see a paper describing the key issues that people have encountered. There seems to be no issue with market data where FIX 5 is being used more often than not. Is it about the session layer? It cannot be about the functional concepts of FIX as there were more changes between FIX 4.2 and FIX 4.3.
As said before, FPL is well aware of this issue and we need to find a way to remove the negative perception which I believe is not justified. This is firmly on the agenda. The functional richness of FIX 5.0 and its many, many Extension Packs (116 in total up to now) has significantly increased the level of functional coverage with respect to trading on exchanges and other electronic marketplaces, including OTC trading. Areas such as reference data have minor coverage with FIX 4.x compared to today. FIX 4.2 had a total of 426 fields. We are now approaching 1700, i.e. a fourfold increase. It is still almost twofold when compared with FIX 4.4. This should tell you how much FIX has developed since FIX 4.4 which is already over seven years old.
In conclusion, I would like to reiterate my request for information on the key pain points preventing the transition to FIX 5. Is it merely a chicken and egg problem or is FIX 5 technically inferior? FIX interface development used to be a highly manual process. Since a few years ago, there is a FIX Repository that supports automated development and should make transitions a lot easier.
I look forward to more feedback on this topic to help FPL move into the right direction in supporting the community of FIX users.
Regards,
Hanno.Fix has become a great success. My analyze result is: because it is easy, simple and efficient. At least this accounts for the fix version up to 4.4. We can plug in and out markets and counterparties at a fraction of the cost and time needed compare implementing unique protocols.
There is however still missing a real “Fast Fix”. That still is as simple and efficient as fix 4.4, but simply faster. Most FIX engines designed was design for trade reporting between brokers, and are simply not fast because no one ever tried to make them fast. I would like to replace all this silly unique protocols to each market with FIX, but latency is the thing that always pops up as the argument against this.
There is also FIX versions greater than 4.4. 5.0 etc. But, is anyone really using it? Our vendors claims, and I understand them, that there is a great deal of work moving from 4.4 to 5.x, and no one can motivate why this should be done. Simply adding some custom fields or implement a few fix 5.x messages in FIX 4.X seam more efficient.
It seems that FIX is making the same journey as HTML. From simple text, a little tags here and there, to complicated XML:s with schemas etc.
I don’t say this is the wrong journey, but I simply don’t understand it from a business point of view. As in last project, we have to move from FIX 4.4 to FIX 5.X to get “settlement date” in execution reports from our counterparty. Must be someone that freaked out somewhere? It can’t be an efficient decision, compare adding an existing field to the old solution.
Looking at the FIX Functionality Matrix, it seems there are only Parties- and Market structure data and some compliance functions that are non existing in 4.4 but Exists in 5.x Is that the only reason from moving from 4.4 to 5.X?
Maybe there is a clear line, a border, at FIX 4.4, where the idea of general and easy stops, and unique and specialized starts?
/ Jan
I’d like to elaborate further on what was said regarding the difficulty of FIX 5 migration.
Personally, I think this has more to do with the psychology of the numbers than anything else. People see 4.4 and 5.0 SP2 and think that the process has to be difficult and burdensome. I have heard plenty of people say migration to FIX 5.0 SP2 is hard, but I’ve never heard any detailed technical argument to defend this position.
Yes, there was a change at the session layer to FIXT 1.1. This allows support for multiple application versions of FIX over one session. But if you are only using one application version on a given session, then the only significant differences are:
- You send 8=FIXT.1.1 instead of 8=FIX.4.4
- You send and receive DefaultApplVerID (1137) in the Logon
That’s it. Two field changes are needed. It is my understanding that virtually every FIX engine out there already has support for FIXT.1.1, at least in the single application version per session mode.
There is one small complication, in that FIX 4.x includes the application version in each message (via the BeginString), while FIXT.1.1 doesn’t. This means that if you have tools to analyze FIX logfiles, and those tools behave differently based on FIX version, you may need to start telling them which application version is being used, since this can’t directly be discerned from a given message taken out of context.
At an application layer, a lot of new functionality was added. But in my experience, it is rare that FPL makes changes that break existing functionality. They do happen, e.g. ExecType introduced in FIX 4.1, some tweaks to order state changes in 4.3, migration to ISO standards like MIC codes in 4.3, etc. But 4.4 is fairly mature and similar to FIX 5.0 SP2 in most respects.
So if the choice is between pulling 5.0 SP2 features into 4.4, or implementing 5.0 SP2, it seems logical to do the latter. The same work is needed to implement the new application level features in either case. And, as I’ve said, the session layer changes are trivial.
One suggestion I have for FPL which might help with 5.0 SP2 adoption: Put together a checklist of any existing 4.4 functionality that changed in a way that is not backwards compatible. Then, a firm can simply look at the list, compare it against their implementation, and see what changes to existing functions (if any) would be needed to support SP2. I think this would combat the notion that FIX 5.0 is difficult to implement; I imagine many firms running 4.4 could simply bump up the version number without other changes.
I would like to add to this discussion a “Problem loading FIXT.1.1 + FIX.5.0 repositories together” @ http://fixprotocol.org/discuss/read/6716e28d
When loading MsgType.xml my engine was expecting MsgType to be unique, so after loading FIXT.1.1 repository, when it starts loading the FIX.5.0 repository, it complains about duplicate MsgType, throws an exception and exits. I handled it by manually editing the FIX.5.0 repository and removing all the Session level MsgTypes which were already present in FIXT.1.1 repository. This worked, but I found it a dirty solution and didn’t like it. I want to be able to deploy the FIX repository for a new version without any manual edits.
[Start Reply from Jim Northey / The LaSalle Technology Group jimn@lasalletech.com
26 Apr 2010 2:46PM ET]
The approach we have come up with is to create configuration rules that define how to handle fields that overlap. For instance MsgType=D should be unknown to FIXT.1.1 But MsgType=D is known for FIXT.1.1+FIX.[45]*
There are other fields. Phil and Ryan did separate analyses on this. We are close to having both: 1) Configuration rules for dynamically binding and 2) producing static, pre-configured repositories for use by member firms.
[End Reply]
Regards,
K. Mahesh