Imported from previous forum
Ability to transport multiple HFT application objects in a single HFT Protocol transmission message.
HFT Application Objects represent IOIs, Orders, Order Cancels, Order Cancel-Replaces, Executions, DKTs, Allocs etc. i.e. they are Application / non-Session objects exchanged between a buy side and a sell side. HFT Message - the basic unit of information exchange between counterparties is a sequence of bytes (the wire representation) used to facilitate exchange of application objects between these applications.
Permitting a HFT Engine to package multiple HFT application objects (destined to the same Target) into a single HFT message would improve performance by saving bandwidth by deduplicating common header information. Taking a hypothetical HFT message layout as
HFTMessage = MsgLen HdrLen Header Body ;
MsgLen = 4 byte Unsigned int > 0;
HdrLen = 4 byte Unsigned int > 0;
Header = MsgSeqnNum SendingTime CompIds OtherFields;
Body = NumHFTObjects HFTObjects;
NumHFTObjects = 1 byte Unsigned int > 0;
HFTObjects = HFTObject HFTObjects;
HFTObject = HFTObjectLen HFTObjectDefn;
HFTObjectLen = 2 byte Unsigned int > 0;
HFTObjectDefn = HFTObjectType HFTObjectFields;
HFTObjectType = 2 ASCII chars;
//represents the equivalent of FIX Tag=Value^ tag 35 message type
HFTObjectFields = HFTField HFTObjectFields;
Some scenarios where this feature could be useful :-
-
A buy side HFT Engine has a DKT for an incorrect execution on the first order, a cancel replace of the second order and a third new order single, all three of these are to be sent to the same sell side. Instead of sending three seperate messages which almost identical message headers and different bodies, the proposed layout would permit one HFT message with these three application objects.
-
A sell side has one Order Cancel reject and many execution reports for different orders, all of these are destined to the same buy side. This layout will permit the cancel reject and all these execution reports to be sent as a single HFT message instead of multiple seperate messages. (Presently multiple execution reports of the same order only can be packaged into a single execution report - the proposed layout would permit execution reports belonging to different orders to be packaged into a single physical message).
Redundant case :- When there is only one application object to be sent, then NumHFTObjects value would be set to 1 and the single application object would be sent in a new message alone.
Regards,
K. Mahesh
Mahesh,
To make your proposal more complete, what are your definitions of:
- MsgSeqNum
- SendingTime
- CompIds
- OtherFields
- HFTField
I assume that you propose that MsgSeqNum is for the first HFTObject and that the following HFTObjects are assigned the following values of MsgSeqNum.
What’s your view on versioning?
Should there be a version field in the message header?
/Rolf
Rolf,
My ideas are as below :-
- MsgSeqNum - Its at the HFT Message level. Its similar to FIX Tag=Value^ MsgSeqNum and uniquely identifies a Message in the specified direction within a given Session. Useful for recovering missed messages.
Since its NOT a Object seq num, HFT Objects themselves would NOT have any seperate MsgSeqNum, their uniqueness is defined by other fields like Orders have ClOrdId, Execs have ExecID, Allocs have AllocID, etc. Once a HFT engine receives a message with MsgSeqNum matching the “next inbound seq num”, the primary purpose of MsgSeqNum is over.
- SendingTime - Its the time at which the Message was prepared by the HFT Engine just before transmission.
Each HFT Object has / can have its own TransactTime (FIX Tag 60) which is expected to be no later than Sending Time (FIX Tag 52).
-
CompIds - All the HFT objects within a HFT message share all attributes of the CompIds heirarchy. But smart order routers etc. may use other attributes of HFT objects which is outside the scope of this consideration.
-
OtherFields - Its other fields of the HFT message header like SecureDataLen + SecureData, PossDupFlag, PossResend, OrigSendingTime, XMLDataLen + XMLData, MessageEncoding, LastMsgSeqNumProcessed, HopGrp, version fields, etc. (This list is based on header fields presently used in FIX Tag=Value^, more / less header fields could exist in the new protocol’s message header).
-
HFTField - This is a HFT field which would be a tag and an associated value, the encoding / wire representation of which would be decided later. The ability to transport multiple application objects in a single Protocol transmission message is independent of how fields would be encoded on the wire. The benefits would apply equally to any tag+value representation like present FIX Tag=Value^ encoding, Compact Binary format / Opcode Binary Format, etc. unless packing many application objects into a single protocol message causes message fragmentation across multiple transport level data packets.
-
HFT version - Message header should have field(s) to specify the version, version extention, custom version extension etc. to retain versioning flexibility.
Regards,
Mahesh
Mahesh,
To make your proposal more complete, what are your definitions of:
- MsgSeqNum
- SendingTime
- CompIds
- OtherFields
- HFTField
I assume that you propose that MsgSeqNum is for the first HFTObject and that the following HFTObjects are assigned the following values of MsgSeqNum.
What’s your view on versioning?
Should there be a version field in the message header?/Rolf
[ original email was from John Harris - john.harris@bondmart.com ]
Mahesh,
I can imagine that the ability to batch like or related objects, e.g., orders, in a single message may be valuable, but I am having trouble understanding the business case for batching seemingly unrelated objects.
Consider your sell-side case below, in which you posit the aggregation of an order-cancel-reject object with multiple, execution-report objects.
I would presume that the order-cancel-reject event that gives rise to the associated order-cancel-reject object is independent of the execution events that give rise to their associated execution-report objects.
If so, under your scheme, what would the sending application do? Do you presume that it will batch messages and send them on a certain schedule, e…g., every 500 milliseconds? If so, I’m not sure I would like that as a user. I would like to learn of an order-cancel reject as soon as possible, irrespective of what other events may be taking place in which I have an interest.
Sorry if I am misunderstanding…
Best,
John
HFT Application Objects represent IOIs, Orders, Order Cancels, Order Cancel-Replaces, Executions, DKTs, Allocs etc. i.e. they are Application / non-Session objects exchanged between a buy side and a sell side. HFT Message - the basic unit of information exchange between counterparties is a sequence of bytes (the wire representation) used to facilitate exchange of application objects between these applications.
Permitting a HFT Engine to package multiple HFT application objects (destined to the same Target) into a single HFT message would improve performance by saving bandwidth by deduplicating common header information. Taking a hypothetical HFT message layout as
HFTMessage = MsgLen HdrLen Header Body ;
MsgLen = 4 byte Unsigned int > 0;
HdrLen = 4 byte Unsigned int > 0;
Header = MsgSeqnNum SendingTime CompIds OtherFields;
Body = NumHFTObjects HFTObjects;
NumHFTObjects = 1 byte Unsigned int > 0;
HFTObjects = HFTObject HFTObjects;
HFTObject = HFTObjectLen HFTObjectDefn;
HFTObjectLen = 2 byte Unsigned int > 0;
HFTObjectDefn = HFTObjectType HFTObjectFields;
HFTObjectType = 2 ASCII chars;
//represents the equivalent of FIX Tag=Value^ tag 35 message typeHFTObjectFields = HFTField HFTObjectFields;
Some scenarios where this feature could be useful :-
A buy side HFT Engine has a DKT for an incorrect execution on the first order, a cancel replace of the second order and a third new order single, all three of these are to be sent to the same sell side. Instead of sending three seperate messages which almost identical message headers and different bodies, the proposed layout would permit one HFT message with these three application objects.
A sell side has one Order Cancel reject and many execution reports for different orders, all of these are destined to the same buy side. This layout will permit the cancel reject and all these execution reports to be sent as a single HFT message instead of multiple seperate messages. (Presently multiple execution reports of the same order only can be packaged into a single execution report - the proposed layout would permit execution reports belonging to different orders to be packaged into a single physical message).
Redundant case :- When there is only one application object to be sent, then NumHFTObjects value would be set to 1 and the single application object would be sent in a new message alone.
Regards,
K. Mahesh
John, you make a good point.
Batching has its drawbacks in that it prevents you from streaming updates as they occur, or you will not make use of the efficiencies that batching was supposed to bring.
Ideally you would like to be able to stream updates and leverage whatever data affinities that happen to occur in the message stream.
This is how FAST field encoding works and that is the reason it is so effective.
/Rolf
Mahesh,
I can imagine that the ability to batch like or related objects, e.g., orders, in a single message may be valuable, but I am having trouble understanding the business case for batching seemingly unrelated objects.
Consider your sell-side case below, in which you posit the aggregation of an order-cancel-reject object with multiple, execution-report objects.
I would presume that the order-cancel-reject event that gives rise to the associated order-cancel-reject object is independent of the execution events that give rise to their associated execution-report objects.
If so, under your scheme, what would the sending application do? Do you presume that it will batch messages and send them on a certain schedule, e…g., every 500 milliseconds? If so, I’m not sure I would like that as a user. I would like to learn of an order-cancel reject as soon as possible, irrespective of what other events may be taking place in which I have an interest.
Sorry if I am misunderstanding…
Best,
John
I agree and would like to add that batching only makes sense in the context of single business events which are typically reflected in FIX with a single message layout containing repeating groups, e.g. . If a new mass quote entering the book leads to updates of multiple price levels in multiple series, this single event can be efficiently broadcast in a single physical message (containing multiple logical messages) with FAST. This does not apply to a collection of independent business events, e.g. a mass entry of orders for different instruments that are sent to different matching engines and may or may not get filled. The engines run in parallel for performance reasons and send their results back to the submitter via a gateway holding the connection to the submitter. Waiting on the gateway for results from multiple sources to allow batching most likely defeats the purpose of high performance if you introduce delays for the sake of batching.
John, you make a good point.
Batching has its drawbacks in that it prevents you from streaming updates as they occur, or you will not make use of the efficiencies that batching was supposed to bring.
Ideally you would like to be able to stream updates and leverage whatever data affinities that happen to occur in the message stream.
This is how FAST field encoding works and that is the reason it is so effective.
/Rolf
Mahesh,
I can imagine that the ability to batch like or related objects, e.g., orders, in a single message may be valuable, but I am having trouble understanding the business case for batching seemingly unrelated objects.
Consider your sell-side case below, in which you posit the aggregation of an order-cancel-reject object with multiple, execution-report objects.
I would presume that the order-cancel-reject event that gives rise to the associated order-cancel-reject object is independent of the execution events that give rise to their associated execution-report objects.
If so, under your scheme, what would the sending application do? Do you presume that it will batch messages and send them on a certain schedule, e…g., every 500 milliseconds? If so, I’m not sure I would like that as a user. I would like to learn of an order-cancel reject as soon as possible, irrespective of what other events may be taking place in which I have an interest.
Sorry if I am misunderstanding…
Best,
John
I agree and would like to add that batching only makes sense in the context of single business events which are typically reflected in FIX with a single message layout containing repeating groups, e.g. …
Yes, Hanno, but I want to make the case that multiple orders and order modifications for the same product, triggered by the same market event, encompasses important use cases for high frequency trading (or whatever we’re calling it). In fact, these use cases are a large part of what distinguishes high frequency trading from other forms. I expect that any protocol this working group devises should cover these use cases efficiently.
Traders commonly run multiple instances of an algorithm concurrently or ladder prices at several points, perhaps on both sides of the market. When the market moves, algos react by entering several orders or modifying multiple orders at once for the same product. Essentially, a batch of orders is the result of a single trading decision.
It may be more efficient to deliver a batch of orders or modifications from the trading application to a match engine, routed through various intermediate nodes, as opposed to sending several individual messages. (Applications may optimize by preparing the batch in advance and just firing the transmit operation when the market meets an anticipated condition.)
Agreed that for individual messages, the one that gets to the engine first, would have lowest latency. However, to the get the whole set of operations performed, overall latency could be lower for a batch compared to the sum of many individual transations.
I believe we are on the same page. Your example was about “batched” order entry. With such a use case in mind for quotes, FIX has provided the MassQuote message and would provide a MassOrder if needed. There are always some common business elements, in your case e.g. “the same product” which get mapped to root level fields. Batching is something else for me as it is a technical term. The “batching device” has no business context, it just takes multiple independent messages and puts them physically together, possibly applying efficiencies related to repeating elements. The FIXML attribute does that (without efficiencies) and the FAST protocol or zlib do that (with efficiencies).
Therefore I am not convinced we need a generic batching mechanism for arbitrary FIX message types (going into or coming out of an exchange) to achieve high performance. A MassQuote message is not an efficient concatenation of Quote messages, it is a message type in its own right. Even NewOrderList is not a mere concatenation of NewOrderSingle messages even if it comes awfully close. Batching that appears useful to the submitter’s algo (e.g. putting a NewOrderSingle for A, OrderCancelReplace for B and OrderCancel for C into a single message) might be a pain for the receiver (e.g. A, B and C are handled by a single matching engine initially, and, as market share picks after 3 months, C is moved to its own separate engine). MassQuote is an example for a functional solution to a technical problem. It does add/modify/delete implicitly and all in a single message type. Matching engines are prepared to handle such messages, split the series into independent threads etc. I think this is much better than an efficient batching of Quote messages.
I agree and would like to add that batching only makes sense in the context of single business events which are typically reflected in FIX with a single message layout containing repeating groups, e.g. …
Yes, Hanno, but I want to make the case that multiple orders and order modifications for the same product, triggered by the same market event, encompasses important use cases for high frequency trading (or whatever we’re calling it). In fact, these use cases are a large part of what distinguishes high frequency trading from other forms. I expect that any protocol this working group devises should cover these use cases efficiently.
Traders commonly run multiple instances of an algorithm concurrently or ladder prices at several points, perhaps on both sides of the market. When the market moves, algos react by entering several orders or modifying multiple orders at once for the same product. Essentially, a batch of orders is the result of a single trading decision.
It may be more efficient to deliver a batch of orders or modifications from the trading application to a match engine, routed through various intermediate nodes, as opposed to sending several individual messages. (Applications may optimize by preparing the batch in advance and just firing the transmit operation when the market meets an anticipated condition.)
Agreed that for individual messages, the one that gets to the engine first, would have lowest latency. However, to the get the whole set of operations performed, overall latency could be lower for a batch compared to the sum of many individual transations.
The application objects being batched together are related to each other in the business sense in that they originate from the same counterparty X and are destined to the same counterparty Y and hence they share the same routing information at HFT engine messaging level, so batching them into a single protocol transmission unit is correct in the “business sense”.
I am not batching them based on a time interval gap, messaging would be kept realtime. The gateway does not wait for results to batch. Lets say many matching engines are running, one per security. Within a millisecond difference, let say three executions of three different orders from the same counterparty (for three different securities) occurs which are delivered to the gateway. Now the gateway can batch them and send a single message instead of three seperate messages.
Only when a HFT engine has more than one application object to be sent across to the same counterparty, it batches them together, otherwise each application object would be sent across as they reach the HFT engine from the application behind the engine.
[Start Quote] However, to the get the whole set of operations performed, overall latency could be lower for a batch compared to the sum of many individual transations. [End quote]
This line of thinking had led me to the idea of introducing a facility to “permit batching of application objects which cannot be batched together as per present FIXProtocol” - Think outside the box or bun or can or … 
Take a non trade messaging example of transport - the postal envolope - there are restrictrictions only on the type of contents that can be packaged in it “Nothing fragile, liquid, perishable, combustile, explosive, prohibited, …”, there is no restriction on the number of contents. Postal service is only concerned with delivery address, transport route, cost to sender, proof of delivery, etc., they are not bothered about what is being sent or how many are being sent long as as long as they comply with applicable regulations and fit into one envolope. Is there any postal regulation which says “A personal DVD cannot be sent in the same package as a hand written letter” or something more generic like “plastic and paper items should not be sent in the same package” these statements backed by “violaters will be prosecuted to the fullest extent permissible by law”? Sounds funny. Similarly why should there be a restriction in a new trading protocol which prevents batching an order-cancel-reject along with some execution reports for other orders? or batching a DKT for an incorrect execution on the first order, a cancel replace of the second order and a third new order single together if the gateway HFT engine on my destination conterparty side can correctly “open the package, seperate the objects and hand over the contents to the respective back end systems”. Postal regulation says “Each package can and should have one and only one valid destination address”, similarly the HFT engine should only bother about the one destination to which a message is intended rather than look into the business relationships between the payload contents of the message which it is transporting.
Providing a message structure / type to permit batching would let users find out if its useful. Users who do not find it useful would not use it - as simple as that. Since we are in the stage of HFT requirements gathering, lets presently just add a requirement to support batching (a new message type) and find out from testing how much of an efficiency gain we can achieve using reallife historical data.
Regards,
K. Mahesh
I disagree. Just because messages come from the same counterparty does not constitute a business context. It is a technical context. FIX application level message types are primarily business functions. A new message type for batching blurs the line between application and transport. I am assuming you would want a repeating group of messages starting with the FIX field MsgType and not something along the line of the session layer message XMLnonFIX.
The working group cannot afford to design various elements and then let the users decide whether they are useful or not. You would need to prototype and test whether, in a high load situation, it is more efficient to pack and unpack unrelated messages compared to sending them individually. Savings can only occur if you get away with a single header for the entire packed message. FIX Standard Header fields are (unfortunately) not all purely session related, e.g. PossResend needs to be evaluated on the application level to determine if the order (not the message = PossDupFlag) has been sent before. All I am saying is that it is not trivial and needs proof. Which of the proprietary protocols out there uses batched messages to improve performance and what are the details? I cannot believe that we are the first ones having thought about such an option.
I think there is more to gain if we focus on the semantic verbosity of FIX (e.g. why do we need to echo input fields in a response?) and design an efficient encoding of messages, fields and valid values.
Regards,
Hanno.
The application objects being batched together are related to each other in the business sense in that they originate from the same counterparty X and are destined to the same counterparty Y and hence they share the same routing information at HFT engine messaging level, so batching them into a single protocol transmission unit is correct in the “business sense”.
I am not batching them based on a time interval gap, messaging would be kept realtime. The gateway does not wait for results to batch. Lets say many matching engines are running, one per security. Within a millisecond difference, let say three executions of three different orders from the same counterparty (for three different securities) occurs which are delivered to the gateway. Now the gateway can batch them and send a single message instead of three seperate messages.
Only when a HFT engine has more than one application object to be sent across to the same counterparty, it batches them together, otherwise each application object would be sent across as they reach the HFT engine from the application behind the engine.
[Start Quote] However, to the get the whole set of operations performed, overall latency could be lower for a batch compared to the sum of many individual transations. [End quote]
This line of thinking had led me to the idea of introducing a facility to “permit batching of application objects which cannot be batched together as per present FIXProtocol” - Think outside the box or bun or can or …
Take a non trade messaging example of transport - the postal envolope - there are restrictrictions only on the type of contents that can be packaged in it “Nothing fragile, liquid, perishable, combustile, explosive, prohibited, …”, there is no restriction on the number of contents. Postal service is only concerned with delivery address, transport route, cost to sender, proof of delivery, etc., they are not bothered about what is being sent or how many are being sent long as as long as they comply with applicable regulations and fit into one envolope. Is there any postal regulation which says “A personal DVD cannot be sent in the same package as a hand written letter” or something more generic like “plastic and paper items should not be sent in the same package” these statements backed by “violaters will be prosecuted to the fullest extent permissible by law”? Sounds funny. Similarly why should there be a restriction in a new trading protocol which prevents batching an order-cancel-reject along with some execution reports for other orders? or batching a DKT for an incorrect execution on the first order, a cancel replace of the second order and a third new order single together if the gateway HFT engine on my destination conterparty side can correctly “open the package, seperate the objects and hand over the contents to the respective back end systems”. Postal regulation says “Each package can and should have one and only one valid destination address”, similarly the HFT engine should only bother about the one destination to which a message is intended rather than look into the business relationships between the payload contents of the message which it is transporting.
Providing a message structure / type to permit batching would let users find out if its useful. Users who do not find it useful would not use it - as simple as that. Since we are in the stage of HFT requirements gathering, lets presently just add a requirement to support batching (a new message type) and find out from testing how much of an efficiency gain we can achieve using reallife historical data.
Regards,
K. Mahesh
Hanno,
Below are my thoughts for each of the points you mentioned.
I disagree.
We agree to disagree.
FIX application level message types are primarily business functions. A new message type for batching blurs the line between application and transport.
I was not thinking along lines of FIX Tag=Value^ Session and application layer seperations.
As I understand, session messages exist only to help communicate application objects. Its the applications behind the HFT engines who have objects of business value to communicate to each other, the HFT engines communicate with each other to help the applications behind them talk to each other, the HFT engines themselves do not have any messages of business value. No use in two HFT engines connected to each other exchanging only heartbeat messages ;-( Since we are talking about a new **Trading protocol, do we really need to maintain the older session versus application philosophy :-?
I am assuming you would want a repeating group of messages starting with the FIX field MsgType and not something along the line of the session layer message XMLnonFIX.
U R right, in the context of a FIX Tag=Value^ encoding, I am looking at a repeating group of messages, i.e. multiple business object payloads within a single physical message.
The working group cannot afford to design various elements and then let the users decide whether they are useful or not.
Thats the way the world works - users take what they like and discard what does not “appeal” to them. An example from my experience - I came across one OMS where the trader can select multiple open orders on the blotter, right-click, select “Send to FIX”, select a broker from drop down list, verify quantities and click send. All these orders change status to released and all these orders get bundled under a tree view header row with as the description. I checked the FIX engine log and noted that a bunch of NewOrderSingles 35=D^ had gone out instead of a single NewOrderList 35=E^. I reported it as a bug. The reply I got was “its not a bug, we like it this way” without any meaningful reason 
You would need to prototype and test whether, in a high load situation, it is more efficient to pack and unpack unrelated messages compared to sending them individually.
I don’t think I have the infrastructure / hardware to run a test of a size which can impress HFTWG / GTC. If there are others in this WG who are planning such a test, I can help them / participate in the test.
Savings can only occur if you get away with a single header for the entire packed message.
Every HFT message has only one header and that header information is applicable to all HFT application objects in that message. Its this header commonality that the batched message would take advantage of to give better performance. Much of the header information is meant for the messaging engine and not for the application.
FIX Standard Header fields are (unfortunately) not all purely session related, e.g. PossResend needs to be evaluated on the application level to determine if the order (not the message = PossDupFlag) has been sent before. All I am saying is that it is not trivial and needs proof.
This gives us good reason to reconsider the FIX Tag=Value^ seperation between header and body fields / session and application related functionality. PossResend could be moved to HFT application object level and PossDup flag should be retained at HFT message level.
Which of the proprietary protocols out there uses batched messages to improve performance and what are the details?
I have not made any survey of existing proprietary trading protocols to have an answer to this question. If proprietary protocols are so much better than FIX, then FIX would not have gained industrywide acceptance. Amex discontinued CMS to move over completely to FIX
http://www.fixprotocol.org/documents/3465/NW%20to%20Members%206%2028%2007.pdf
If there are other examples of FIXProtocol replacing proprietary trading protocols, please share details. I do not see much gain in taking proprietary protocols as our starting point.
We should aim for something new as John Harris put it “Let’s accomplish something insanely useful through our new protocol efforts. We have no excuse for settling for anything less.”
I cannot believe that we are the first ones having thought about such an option.
In anything / everything somebody + sometime has to be first.
In the Transport Independence framework I had posted “6. This may also pave the way to permit multiple FIX message bodies to be carried in a “single” FIX message - repeating group of messages.” at http://fixprotocol.org/discuss/read/94232f30 on 16 Oct 2007 1:34AM ET. Since there was nobody else to discuss the advantages / disadvantages of “repeating group of messages”, I never got an opportunity to explain its advantages in that thread of posting.
We are not solving a (chicken/egg) ? which came first problem. There is spicy Chicken rice with boiled eggs curry - it does not matter which was cooked first - chicken rice / egg curry.
I think there is more to gain if we focus on the semantic verbosity of FIX (e.g. why do we need to echo input fields in a response?) and design an efficient encoding of messages, fields and valid values.
As I had already stated in my previous posts in this forum, optimizing FIX semantics, encoding and transport level optimization are multiple fronts on which we have to fight the low-latency & high frequency battle.
Yesterday, I observed a grocer at point of sale packing customer purchases into carry bags. Each bag is packed to full in weight or volume - whichever occurs first / also subject to seperation based on type - dry food items, fresh produce, precooked food, detergents and cleaners, school and office supplies, garden supplies, health & body care, etc. Applying this grocery bag philosophy - I would try to batch as many application objects into a single transmission message so that the message fills the TCP data packet’s payload as much as possible. TCP header and HFT message header are present whether only one application object or multiple application objects (is) are packaged into one HFT message, by packing multiple application objects into a single message which fits into a single TCP data packet, the per application object share (cost) of TCP + HFT header is reduced.
I have observed the traffic rules of overcrowded zones in cities - there are special lanes / priorities for high occupancy vehicles and all the “one person alone in one car” are diverted to remote parking from which a shuttle is provided - these decrease traffic congestation.
Regards,
K. Mahesh
Regards,
Hanno.I am not batching them based on a time interval gap, messaging would be kept realtime. The gateway does not wait for results to batch. Lets say many matching engines are running, one per security. Within a millisecond difference, let say three executions of three different orders from the same counterparty (for three different securities) occurs which are delivered to the gateway. Now the gateway can batch them and send a single message instead of three seperate messages.
Only when a HFT engine has more than one application object to be sent across to the same counterparty, it batches them together, otherwise each application object would be sent across as they reach the HFT engine from the application behind the engine.
[Start Quote] However, to the get the whole set of operations performed, overall latency could be lower for a batch compared to the sum of many individual transations. [End quote]
This line of thinking had led me to the idea of introducing a facility to “permit batching of application objects which cannot be batched together as per present FIXProtocol” - Think outside the box or bun or can or …
Providing a message structure / type to permit batching would let users find out if its useful. Users who do not find it useful would not use it - as simple as that. Since we are in the stage of HFT requirements gathering, lets presently just add a requirement to support batching (a new message type) and find out from testing how much of an efficiency gain we can achieve using reallife historical data.
Regards,
K. Mahesh
Mahesh,
as I commented earlier; it is unclear to me what benefits we can get from using a message structure like the one we have discussed here.
I’m with Hanno on this, it needs to be empirically tested before we can seriously consider such a construct. I suggest that we come up with one or more scenarios to be tested. In my experience from mdowg and other projects, it’s defining the test cases and compiling valid test data that takes the most resources. In this case I guess the people involved will want to see some diversity in the test data to make sure that we are not trying to optimize for some specific use case.
This points us back to an earlier discussion on the forum, defining and compiling relevant test traces. If I recall correctly it was brought up early on by Philip and a few others. It’s time to focus on test data so we can start testing our hypotheses. I’m not going to vote for any suggestions that haven’t been or can be validated.
Best,
Rolf
As per trading and/or market practices / FIXProtocol specifications what all could be considered as related in business context ? It would be really helpful if someone could elaborate more on this or point me to relevant documents at any website which could make this distinction clearer to me.
Just because messages come from the same counterparty does not constitute a business context. It is a technical context.
The application objects being batched together are related to each other in the business sense in that they originate from the same counterparty X and are destined to the same counterparty Y and hence they share the same routing information at HFT engine messaging level, so batching them into a single protocol transmission unit is correct in the “business sense”.
[ original email was from John Harris - john.harris@bondmart.com ]
Mahesh,
Please see my comments (preceded with *** for clarity) in-line. I suspect that I may be misunderstanding your proposal, because you are arguing so passionately for it and I do not see the benefit at all. So, I welcome better understanding and to be proven wrong. Sorry to be slow in commenting.
The application objects being batched together are related to each other in the business sense in that they originate from the same counterparty X and are destined to the same counterparty Y and hence they share the same routing information at HFT engine messaging level, so batching them into a single protocol transmission unit is correct in the “business sense”.
*** The conclusion does not follow from the premise. Just because objects share an origin and destination does not mean, ipso facto, batching them for transmission is correct.
I am not batching them based on a time interval gap, messaging would be kept realtime. The gateway does not wait for results to batch. Lets say many matching engines are running, one per security. Within a millisecond difference, let say three executions of three different orders from the same counterparty (for three different securities) occurs which are delivered to the gateway. Now the gateway can batch them and send a single message instead of three seperate messages.
*** All batching implies a time interval gap, whether based on discrete passage of time or the occurrence of an event. The processing upstream from the gateway is sequential at each matching engine and uncoordinated across matching engines. That the gateway might sometimes have messages to send destined for the same address has to be coincidental, not purposeful, from a business perspective. Whether it makes sense to code the gateway to batch such messages that just happen to have the same address some of the time into a single envelope is doubtful, but there is no harm in building a prototype and testing the hypothesis. But you would have to account for and demonstrate the efficiencies on an end-to-end basis in order for market-center operators to adopt it.
Only when a HFT engine has more than one application object to be sent across to the same counterparty, it batches them together, otherwise each application object would be sent across as they reach the HFT engine from the application behind the engine.
*** This is self-contradictory, if I am understanding it correctly. You cannot simultaneously send each object as it reaches the gateway while also pausing such sending to determine whether two or more objects having a common destination address can avail themselves of a batching procedure.
[Start Quote] However, to the get the whole set of operations performed, overall latency could be lower for a batch compared to the sum of many individual transations. [End quote]
*** This is correct, and what it means is that buffering messages and processing them on a cycle or in response to an event is sensible - but not merely as a function of address. This concept applies to inbound and outbound messages. Further, it is sensible to offer multiple service tiers for data outbound from an exchange, e.g., market data updates every 100 milliseconds, every 500 milliseconds, every second, every ten seconds, and so forth. And sure, if it can be done efficiently on both ends and in a way that makes business sense (which has to be demonstrated), there may be a case for combining messages intended for the same destination in a single envelope under such a scenario.
This line of thinking had led me to the idea of introducing a facility to “permit batching of application objects which cannot be batched together as per present FIXProtocol” - Think outside the box or bun or can or …
*** Sure, you can send in one postal envelope disparate goods. But how often to people find it expedient to do so? People could save postage by paying their utility bills every other month, but other considerations trump the postal savings. We could buffer and batch love letters to our sweethearts across days, but doing so tends to defeat the purpose of the missives. And the postal service is a batch processor, but we still rarely exploit our right as patrons to combine otherwise unrelated objects in the same envelope.
Providing a message structure / type to permit batching would let users find out if its useful. Users who do not find it useful would not use it - as simple as that. Since we are in the stage of HFT requirements gathering, lets presently just add a requirement to support batching (a new message type) and find out from testing how much of an efficiency gain we can achieve using reallife historical data.
*** By all means, build a prototype and test this scheme. But were we business partners, I would respectfully discourage you from dealing with this except as a low priority. I understand that the concept is technically feasible. I just don’t think it makes business sense. I am nonetheless happy to learn a better way of doing anything
Best,
John
Hi John,
Please see my comments inline,
are my initial comment lines,
*** are your replies / comments and
++++ are my latest comments.
*** I suspect that I may be misunderstanding your proposal, because you are arguing so passionately for it and I do not see the benefit at all. So, I welcome better understanding and to be proven wrong. Sorry to be slow in commenting.
++++ Your comments are very thought provoking and make me rethink my assumptions about this proposal. I am not saying that batching will definetly solve all of the speed complaints that users of FIX Tag=Value^ have compared to proprietary trading protocols, problems which FIX over FAST could not solve for them, the reason why a HFTWG was created by FPL.
++++ Batching is just one more tool available in the proposed **Trading protocol, it nothing more or less than that. We have not tried batching previously as part of FIX Tag=Value^ neither have any other proprietrary protocols tried it.
The application objects being batched together are related to each other in the business sense in that they originate from the same counterparty X and are destined to the same counterparty Y and hence they share the same routing information at HFT engine messaging level, so batching them into a single protocol transmission unit is correct in the “business sense”.
*** The conclusion does not follow from the premise. Just because objects share an origin and destination does not mean, ipso facto, batching them for transmission is correct.
++++ Pardon my lack of knowledge, I am unsure of this. I have some scenario questions regarding how ownership of stocks is treated as per securities trading laws and regulations, I shall post them to the International forum @ http://fixprotocol.org/discuss/14 and would be really happy to get your inputs there because I do not want to pollute this discussion forum with stuff of what all constitutes a business relationship / how do securities regulations compliance see businesses as entities etc.
I am not batching them based on a time interval gap, messaging would be kept realtime. The gateway does not wait for results to batch. Lets say many matching engines are running, one per security. Within a millisecond difference, let say three executions of three different orders from the same counterparty (for three different securities) occurs which are delivered to the gateway. Now the gateway can batch them and send a single message instead of three seperate messages.
*** All batching implies a time interval gap, whether based on discrete passage of time or the occurrence of an event. The processing upstream from the gateway is sequential at each matching engine and uncoordinated across matching engines. That the gateway might sometimes have messages to send destined for the same address has to be coincidental, not purposeful, from a business perspective. Whether it makes sense to code the gateway to batch such messages that just happen to have the same address some of the time into a single envelope is doubtful, but there is no harm in building a prototype and testing the hypothesis. But you would have to account for and demonstrate the efficiencies on an end-to-end basis in order for market-center operators to adopt it.
++++ My design is based on the occurence of discrete event(s).
++++ I agree that its coincidental for multiple messages in a queue to share desination address / route, I do not have any historical data to say how much % this coincidence occurs. The higher the statistical occurence of this coicidence, the more **T protocol would benefit by providing a mechanism for batching.
Only when a HFT engine has more than one application object to be sent across to the same counterparty, it batches them together, otherwise each application object would be sent across as they reach the HFT engine from the application behind the engine.
*** This is self-contradictory, if I am understanding it correctly. You cannot simultaneously send each object as it reaches the gateway while also pausing such sending to determine whether two or more objects having a common destination address can avail themselves of a batching procedure.
++++ The pausing is not intentional - its the time taken by a Socket writer to write one message into its output stream (which the counterparty would receive thru its input stream), increment the last used seqnum of session and move to the next message. I agree that a single threaded / sequential process cannot simultaneously send each object as it reaches the gateway while also pausing such sending to determine whether two or more objects having a common destination address can avail themselves of a batching procedure. Lets take the case of multiple Threads in a **T protocol gateway doing the jobs of
- receiving business objects from applications behind the engine,
- preparing **T messages
- sending the messages out to counterparty.
Threads are
Batcher - Receives business objects which come from the application(s) behind the engine / gateway, batches them based on routing and destination (CompIDs), hands over batched business objects sets to Message factory. More that one worker threads could scan the inbound queue and the object which encapsulates the underlying queue would be required to ensure appropriate synchronization / locking to make sure two copies of the same application object do NOT end up in two different batches.
When the inbound queue is empty, there is nothing to send, so wait.
When there is only one application object in the inbound queue, create a batch with only this one object, hand over created batch to Message factory.
When inbound queue size is more than one, apply batching rule(s) - routing & destination address matching, expected size of final **T protocol transmission is not too long i.e. fits into one data payload of transmission data packet and hand over created batch to Message factory.
Since this is a high level look at how batching would look, a for / while / if-else code level discussion would be too low level at this stage of our **Trading protocol design.
Message factory - Takes a batch of business objects and creates an “Almost Complete Message” (ACM) and queues it for Socket writer to complete and send.
Socket writer - picks up an ACM, gets last outbound sequence number from session, populates MesgSeqNum after adding one to the temp value retrieved from session, gets current time and populates SendingTime, calculates and populates body length, calculates & populates Checksum (if the protocol requires each message to have one), converts the entire message to String and writes it to network Socket, flushes the outstream, increments the last outbound seqn number of session by 1 and looks for next ACM to complete and send.
I think that FIX Engines ++ application architecures currently optimized for one application object per protocol transmission message would not perform optimally, but by permitting such a structure, the HFT (or whatever **T) protocol community gets the opportunity to try such an optimization and my expectation is that programmers will definetly design architectues which deliver better thruput than the present one application object per message design as per :-
[Start Quote] However, to the get the whole set of operations performed, overall latency could be lower for a batch compared to the sum of many individual transations. [End quote]
*** This is correct, and what it means is that buffering messages and processing them on a cycle or in response to an event is sensible - but not merely as a function of address. This concept applies to inbound and outbound messages. Further, it is sensible to offer multiple service tiers for data outbound from an exchange, e.g., market data updates every 100 milliseconds, every 500 milliseconds, every second, every ten seconds, and so forth. And sure, if it can be done efficiently on both ends and in a way that makes business sense (which has to be demonstrated), there may be a case for combining messages intended for the same destination in a single envelope under such a scenario.
++++ I am NOT bufferring messages / processing them in a cycle / delaying them for a threshold volume to be reached before batching them and sending them. My present proposal is batching them only as a function of routing and destination as representation of CompIds.
++++ I do not have a comment on the timeinterval based delivery of market data. Also FAST exists to deliver streaming market data efficiently.
This line of thinking had led me to the idea of introducing a facility to “permit batching of application objects which cannot be batched together as per present FIXProtocol” - Think outside the box or bun or can or …
*** Sure, you can send in one postal envelope disparate goods. But how often to people find it expedient to do so? People could save postage by paying their utility bills every other month, but other considerations trump the postal savings. We could buffer and batch love letters to our sweethearts across days, but doing so tends to defeat the purpose of the missives. And the postal service is a batch processor, but we still rarely exploit our right as patrons to combine otherwise unrelated objects in the same envelope.
++++ A word of caution on using the postal analogy - its applicable upto a point and meaningless beyond it, so we must be careful not to stretch the postal anology too far till breaking point. A postal worker once told me “with people moving more and more towards electronic communication, he is seeing gradual but sure reduction in paper based postal communication and this is forcing postal departments to downsize and work more cost effectively”.
++++ Utility bills which require a monthly payment on or before a specific date every month incur a late fee / penalty if payment is not made on time, so saving postage by delaying payment of bills is out of question. One utility company for which we had a fixed monthly payment due, gave us the option of making quarterly payments & accepting quarterly statements and get a discount of 1 U$D in each 40 U$D of monthly bill.
++++ The latest production release of VALOMESS (Valentine’s Love Messaging) protocol is 1.0 based on thousands of years old design given jointly by Cupid (from the EMEA angels committee) and Kama (from the APAC devas WG) under the leadership of God, based on “1 couple = 1 man + 1 woman” rule is basically Unicast in nature and hence cannot deal with plurals like ours, misses, sweethearts etc… FYI, my CurrentMaritalStatus=HappilyMarried prohibits me from doing further research on any multicast love messaging (too late in my life, refer Girlfriend vs Wife). I can always cuntinue my research on optimizing bidirection asynchronous secure session oriented connectionless authenticated UNICAST love messaging protocol by using the excuse of “Darling I am making things between us better ;-)” Anything else - I will be Gratefully Dead. Regarding applicability of batching in the Unicast VALOMESS 1.0 protocol, it has been used to very positive end results - a new dress, a jewelery, an Audio CD containing sweet love songs, a birthday card and a handwritten feelings letter packaged into a birthday gift package delivered as a single postal package has better appeal than many seperate gift packages. FYI VALOMESS 1.0 supports NewOrderList for use by girlfriend / wife for efficiently communicating multiple orders which have to be immediately executed by the boyfriend / husband failing which the boyfriend / husband would be executed by the originator of list order. Used incorrectly VALOMESS can make a mess of the user’s life.
Happy Valentine’s day
Providing a message structure / type to permit batching would let users find out if its useful. Users who do not find it useful would not use it - as simple as that. Since we are in the stage of HFT requirements gathering, lets presently just add a requirement to support batching (a new message type) and find out from testing how much of an efficiency gain we can achieve using reallife historical data.
*** By all means, build a prototype and test this scheme. But were we business partners, I would respectfully discourage you from dealing with this except as a low priority. I understand that the concept is technically feasible. I just don’t think it makes business sense. I am nonetheless happy to learn a better way of doing anything
++++ As I have already stated in my previous post “I don’t think I have the infrastructure / hardware to run a test of a size which can impress HFTWG / GTC”, what you have mentioned as “demonstrate the efficiencies on an end-to-end basis in order for market-center operators to adopt it” in your post. This requires some really big hardware and networks which I do NOT have. I am only capable of writing code. I have one HP laptop with intel Core i5 (4 cores in 1 chip), a netbook (toy) for travel browsing use and a old DELL laptop (purchased 2006) whose builtin screen does not work and can be used by connecting an external monitor and its harddisk may fail anytime - so what real life end to end trade messaging test can be done with these hardware ? Assume I run such a test, would our WG / GTC be impressed with the “end to end test results” run on these machines ?
++++ I understand your business perspective of dealing with this esoteric idea as a low priority, but assuming folks out there who have that kind of hardware and interested to FIXperiment could employ me and we both could benefit from these FIXperiments - my employer gets good quality code ahead of emergence of new market standard(s) and I get paid to do what I like
Regards,
Mahesh
[ original email was from John Harris - john.harris@bondmart.com ]
Mahesh,
VALOMESS will be hard to top. Fortunately, my wife does not read this forum. Otherwise, she will know that other wives have a much better deal
Thank you for your thoughtful reply.
We’ve hashed this out pretty thoroughly at the discussion stage. Hopefully your concept can be tested and will prove helpful. I remain doubtful on a business level, but that is not to discourage the effort and certainly not to disrespect the creativity and effort that you have invested in the proposal.
Happy Valentines Day!
Best,
John
Hi John,
Please see my comments inline,
are my initial comment lines,
*** are your replies / comments and
++++ are my latest comments.*** I suspect that I may be misunderstanding your proposal, because you are arguing so passionately for it and I do not see the benefit at all. So, I welcome better understanding and to be proven wrong. Sorry to be slow in commenting.
++++ Your comments are very thought provoking and make me rethink my assumptions about this proposal. I am not saying that batching will definetly solve all of the speed complaints that users of FIX Tag=Value^ have compared to proprietary trading protocols, problems which FIX over FAST could not solve for them, the reason why a HFTWG was created by FPL.
++++ Batching is just one more tool available in the proposed **Trading protocol, it nothing more or less than that. We have not tried batching previously as part of FIX Tag=Value^ neither have any other proprietrary protocols tried it.The application objects being batched together are related to each other in the business sense in that they originate from the same counterparty X and are destined to the same counterparty Y and hence they share the same routing information at HFT engine messaging level, so batching them into a single protocol transmission unit is correct in the “business sense”.
*** The conclusion does not follow from the premise. Just because objects share an origin and destination does not mean, ipso facto, batching them for transmission is correct.
++++ Pardon my lack of knowledge, I am unsure of this. I have some scenario questions regarding how ownership of stocks is treated as per securities trading laws and regulations, I shall post them to the International forum @ http://fixprotocol.org/discuss/14 and would be really happy to get your inputs there because I do not want to pollute this discussion forum with stuff of what all constitutes a business relationship / how do securities regulations compliance see businesses as entities etc.
I am not batching them based on a time interval gap, messaging would be kept realtime. The gateway does not wait for results to batch. Lets say many matching engines are running, one per security. Within a millisecond difference, let say three executions of three different orders from the same counterparty (for three different securities) occurs which are delivered to the gateway. Now the gateway can batch them and send a single message instead of three seperate messages.
*** All batching implies a time interval gap, whether based on discrete passage of time or the occurrence of an event. The processing upstream from the gateway is sequential at each matching engine and uncoordinated across matching engines. That the gateway might sometimes have messages to send destined for the same address has to be coincidental, not purposeful, from a business perspective. Whether it makes sense to code the gateway to batch such messages that just happen to have the same address some of the time into a single envelope is doubtful, but there is no harm in building a prototype and testing the hypothesis. But you would have to account for and demonstrate the efficiencies on an end-to-end basis in order for market-center operators to adopt it.
++++ My design is based on the occurence of discrete event(s).
++++ I agree that its coincidental for multiple messages in a queue to share desination address / route, I do not have any historical data to say how much % this coincidence occurs. The higher the statistical occurence of this coicidence, the more **T protocol would benefit by providing a mechanism for batching.Only when a HFT engine has more than one application object to be sent across to the same counterparty, it batches them together, otherwise each application object would be sent across as they reach the HFT engine from the application behind the engine.
*** This is self-contradictory, if I am understanding it correctly. You cannot simultaneously send each object as it reaches the gateway while also pausing such sending to determine whether two or more objects having a common destination address can avail themselves of a batching procedure.
++++ The pausing is not intentional - its the time taken by a Socket writer to write one message into its output stream (which the counterparty would receive thru its input stream), increment the last used seqnum of session and move to the next message. I agree that a single threaded / sequential process cannot simultaneously send each object as it reaches the gateway while also pausing such sending to determine whether two or more objects having a common destination address can avail themselves of a batching procedure. Lets take the case of multiple Threads in a **T protocol gateway doing the jobs of
- receiving business objects from applications behind the engine,
- preparing **T messages
- sending the messages out to counterparty.
Threads are
Batcher - Receives business objects which come from the application(s) behind the engine / gateway, batches them based on routing and destination (CompIDs), hands over batched business objects sets to Message factory. More that one worker threads could scan the inbound queue and the object which encapsulates the underlying queue would be required to ensure appropriate synchronization / locking to make sure two copies of the same application object do NOT end up in two different batches.
When the inbound queue is empty, there is nothing to send, so wait.
When there is only one application object in the inbound queue, create a batch with only this one object, hand over created batch to Message factory.
When inbound queue size is more than one, apply batching rule(s) - routing & destination address matching, expected size of final **T protocol transmission is not too long i.e. fits into one data payload of transmission data packet and hand over created batch to Message factory.
Since this is a high level look at how batching would look, a for / while / if-else code level discussion would be too low level at this stage of our **Trading protocol design.
Message factory - Takes a batch of business objects and creates an “Almost Complete Message” (ACM) and queues it for Socket writer to complete and send.
Socket writer - picks up an ACM, gets last outbound sequence number from session, populates MesgSeqNum after adding one to the temp value retrieved from session, gets current time and populates SendingTime, calculates and populates body length, calculates & populates Checksum (if the protocol requires each message to have one), converts the entire message to String and writes it to network Socket, flushes the outstream, increments the last outbound seqn number of session by 1 and looks for next ACM to complete and send.
I think that FIX Engines ++ application architecures currently optimized for one application object per protocol transmission message would not perform optimally, but by permitting such a structure, the HFT (or whatever **T) protocol community gets the opportunity to try such an optimization and my expectation is that programmers will definetly design architectues which deliver better thruput than the present one application object per message design as per :-
[Start Quote] However, to the get the whole set of operations performed, overall latency could be lower for a batch compared to the sum of many individual transations. [End quote]
*** This is correct, and what it means is that buffering messages and processing them on a cycle or in response to an event is sensible - but not merely as a function of address. This concept applies to inbound and outbound messages. Further, it is sensible to offer multiple service tiers for data outbound from an exchange, e.g., market data updates every 100 milliseconds, every 500 milliseconds, every second, every ten seconds, and so forth. And sure, if it can be done efficiently on both ends and in a way that makes business sense (which has to be demonstrated), there may be a case for combining messages intended for the same destination in a single envelope under such a scenario.
++++ I am NOT bufferring messages / processing them in a cycle / delaying them for a threshold volume to be reached before batching them and sending them. My present proposal is batching them only as a function of routing and destination as representation of CompIds.
++++ I do not have a comment on the timeinterval based delivery of market data. Also FAST exists to deliver streaming market data efficiently.
This line of thinking had led me to the idea of introducing a facility to “permit batching of application objects which cannot be batched together as per present FIXProtocol” - Think outside the box or bun or can or …
*** Sure, you can send in one postal envelope disparate goods. But how often to people find it expedient to do so? People could save postage by paying their utility bills every other month, but other considerations trump the postal savings. We could buffer and batch love letters to our sweethearts across days, but doing so tends to defeat the purpose of the missives. And the postal service is a batch processor, but we still rarely exploit our right as patrons to combine otherwise unrelated objects in the same envelope.
++++ A word of caution on using the postal analogy - its applicable upto a point and meaningless beyond it, so we must be careful not to stretch the postal anology too far till breaking point. A postal worker once told me “with people moving more and more towards electronic communication, he is seeing gradual but sure reduction in paper based postal communication and this is forcing postal departments to downsize and work more cost effectively”.
++++ Utility bills which require a monthly payment on or before a specific date every month incur a late fee / penalty if payment is not made on time, so saving postage by delaying payment of bills is out of question. One utility company for which we had a fixed monthly payment due, gave us the option of making quarterly payments & accepting quarterly statements and get a discount of 1 U$D in each 40 U$D of monthly bill.++++ The latest production release of VALOMESS (Valentine’s Love Messaging) protocol is 1.0 based on thousands of years old design given jointly by Cupid (from the EMEA angels committee) and Kama (from the APAC devas WG) under the leadership of God, based on “1 couple = 1 man + 1 woman” rule is basically Unicast in nature and hence cannot deal with plurals like ours, misses, sweethearts etc… FYI, my CurrentMaritalStatus=HappilyMarried prohibits me from doing further research on any multicast love messaging (too late in my life, refer Girlfriend vs Wife). I can always cuntinue my research on optimizing bidirection asynchronous secure session oriented connectionless authenticated UNICAST love messaging protocol by using the excuse of “Darling I am making things between us better ;-)” Anything else - I will be Gratefully Dead. Regarding applicability of batching in the Unicast VALOMESS 1.0 protocol, it has been used to very positive end results - a new dress, a jewelery, an Audio CD containing sweet love songs, a birthday card and a handwritten feelings letter packaged into a birthday gift package delivered as a single postal package has better appeal than many seperate gift packages. FYI VALOMESS 1.0 supports NewOrderList for use by girlfriend / wife for efficiently communicating multiple orders which have to be immediately executed by the boyfriend / husband failing which the boyfriend / husband would be executed by the originator of list order. Used incorrectly VALOMESS can make a mess of the user’s life.
Providing a message structure / type to permit batching would let users find out if its useful. Users who do not find it useful would not use it - as simple as that. Since we are in the stage of HFT requirements gathering, lets presently just add a requirement to support batching (a new message type) and find out from testing how much of an efficiency gain we can achieve using reallife historical data.
*** By all means, build a prototype and test this scheme. But were we business partners, I would respectfully discourage you from dealing with this except as a low priority. I understand that the concept is technically feasible. I just don’t think it makes business sense. I am nonetheless happy to learn a better way of doing anything
++++ As I have already stated in my previous post “I don’t think I have the infrastructure / hardware to run a test of a size which can impress HFTWG / GTC”, what you have mentioned as “demonstrate the efficiencies on an end-to-end basis in order for market-center operators to adopt it” in your post. This requires some really big hardware and networks which I do NOT have. I am only capable of writing code. I have one HP laptop with intel Core i5 (4 cores in 1 chip), a netbook (toy) for travel browsing use and a old DELL laptop (purchased 2006) whose builtin screen does not work and can be used by connecting an external monitor and its harddisk may fail anytime - so what real life end to end trade messaging test can be done with these hardware ? Assume I run such a test, would our WG / GTC be impressed with the “end to end test results” run on these machines ?
++++ I understand your business perspective of dealing with this esoteric idea as a low priority, but assuming folks out there who have that kind of hardware and interested to FIXperiment could employ me and we both could benefit from these FIXperiments - my employer gets good quality code ahead of emergence of new market standard(s) and I get paid to do what I likeRegards,
Mahesh
Thanks John. Its a please discussing these topics with you. Hope to have more fun as we work on FIX Protocol’s different initiatives.
Regards,
K. Mahesh
Hi All,
Presently executions for different orders belonging to the same list cannot be sent in a single protocol transmission message. Permitting batching would permit these executions to be batched together. Could these executions be considered related in the business context ?
Regards,
K. Mahesh
Extensible Messaging and Presence Protocol (XMPP): Instant Messaging and Presence :-
http://www.xmpp.org/rfcs/rfc3921.html
The above referenced webpage contains Love messaging in XML format between
romeo@example.net/orchard
and
juliet@example.com/balcony
Reminded me of VALOMESS.1.0 (Valentine's Love Messaging protocol)
Regards,
K. Mahesh