Imported from previous forum
When a FIX Engine / application is recovering from a crash / failure, it restores its internal state using persistent data to the last known best state after the last persistence record was written and before the failure happened. After having restored itself, if its wants to validate its state using data from the counterparty(ies), it can do a resend request 0 to infinity and reconstruct the state of Application objects using the data contained in the responses to resend requests (not a network friendly solution but works ;-). Resend request processing is synchronization at the FIX Engine level, these new set of messages to aid connected counterparty applications synchronize themselves operates at Application level. In resend request processing logic, there is no way for an application to ask its counterparty what messages it sent to counterparty. This sounds confusing - this is like me asking you in the middle of our conversation “sorry, What was I telling you ?” or “friend, what were we talking about ?”. I propose that the following new message types be created in FIXProtocol / HFT to help communicate Application Objects (Non-Session objects).
-
Application Objects request - FIX message requesting counterparty application to provide list of application objects they share. This message can ask for :-
1.1. All objects, only active objects, only Expired / terminal state objects.
1.2. Both directions, only you to me, only me to you. Using hierarchy of CompIds, more complex Application object requests like “my down stream counterparty XYZ is asking for all its objects which were delivered to you thru me” can be constructed.
1.3. Specific Object types, for example, a buy side saying “give me list of all single orders that we share”
1.4. Higher level of granularity can be designed - “give me all the Objects we exchanged with each other which have ClOrdId=12345” to query for an Order and its Execution chain.
1.5. Can specify start time + end time / Session / sub session etc.
-
Application Objects response - Response to 1. Can be sent unsolicited to mean “This is what I think we have been discussing” or “This is what I think you have told me”. This message should be able to represent chains of objects to aid in faster reconstruction of application objects at the receiving end.
-
Application Objects request reject - Response to signal error(s) in request parameters of request. This is not a fatal error.
-
Application Objects response ack - Response to signal successfull processing of Objects response. Receipt of this Ack indicates Synchronization complete.
-
Application Objects response reject - Response to signal failure in processing of response. This is a fatal error.
To illustrate the use of above messages, let me take an example. A buy side BS and sell side SS connect on a FIX Session. Session is stable and the following business messages are exchanged
BS > SS : Order1 Buy 1000 shares of IBM LimitPx 80, TIF Day
BS > SS : Order2 Sell 2000 shares of MSFT LimitPx 40, TIF Day
SS > BS : Exec1 for Order1, 1000 shares at 79.99
SS > BS : Exec2 for Order2, 200 shares at 40.01
Now Buy side crashes and recovers. After recovery, on reconnect, BS sends an unsolicited Application objects response which says “I think I sent you Order1 (Buy 1000 shares of IBM LimitPx 80, TIF Day), Order2 (Sell 2000 shares of MSFT LimitPx 40, TIF Day) and you sent me Exec1 for Order1 (1000 shares at 79.99), Exec2 for Order2 (200 shares at 40.01)”. SS receives the unsolicited response and validates it. All Objects in response match objects in SS’s internal memory, so SS sends a response Ack. Buy side receives Ack and knows that Application Objects synchronization is complete and applications are ready to resume business.
Regards,
K. Mahesh
The information flow in a trading setting is typically assymetric where the traditional FIX session layer takes a symmetric view on the message flow.
A symmetric session layer design leads to an unnecessary processing and messaging overhead, at least in some settings (HFT).
Letting exactly-once semantics be implemented using sequencing at the session layer eliminates an optimization opportunity.
Letting the server request resends from the client exposes the client to resending stale HFT orders, forcing the client implementation to blend its application layer with the session layer to gap fill these stale orders that reside in the outgoing message journal.
In other words, the symmetric design adds an overhead and then makes you solve a problem that is not there to begin with in a more simplistic assymetric design.
How about lightweight unsequenced exactly-once semantics for order entry and a sequenced return flow?
Exactly once semantics as in at most once plus at least once.
At most once - server side filtering based on a client assigned order identifier.
At least once - client resends at will.
Unsequenced order entry enables concurrency (potentially better performance) and the use of an unreliable transport like UDP (potentially better performance).
A sequenced return flow provides an ordered, recoverable stream of resulting messages that has nice and simple semantics.
The exactly-once logic and the identifier required for that in the client to server flow belongs in the the application layer.
The sequencing concept in the server to client flow belongs in the session layer.
OUCH got these semantics right, at least for the subset of trading it supports. What’s in there is minimalistic but sufficient.
For a new protocol to be competitive from this perspective, it should be possible to apply the protocol in a way where there is no additional transfer (on the wire), state (data that consumes processor cache), and processing (instructions) overhead than the bare minimum required.
In FIX terms, this means there should be a mode where client messages leaves out MsgSeqNum/ApplSeqNum and relies on an application-level field to uniquely identify an order (like ClOrdID) or an operation on an order (OrderID plus a new ClOrdID or similar).
Regards,
Anders
When a FIX Engine / application is recovering from a crash / failure, it restores its internal state using persistent data to the last known best state after the last persistence record was written and before the failure happened. After having restored itself, if its wants to validate its state using data from the counterparty(ies), it can do a resend request 0 to infinity and reconstruct the state of Application objects using the data contained in the responses to resend requests (not a network friendly solution but works ;-). Resend request processing is synchronization at the FIX Engine level, these new set of messages to aid connected counterparty applications synchronize themselves operates at Application level. In resend request processing logic, there is no way for an application to ask its counterparty what messages it sent to counterparty. This sounds confusing - this is like me asking you in the middle of our conversation “sorry, What was I telling you ?” or “friend, what were we talking about ?”. I propose that the following new message types be created in FIXProtocol / HFT to help communicate Application Objects (Non-Session objects).
Application Objects request - FIX message requesting counterparty application to provide list of application objects they share. This message can ask for :-
1.1. All objects, only active objects, only Expired / terminal state objects.
1.2. Both directions, only you to me, only me to you. Using hierarchy of CompIds, more complex Application object requests like “my down stream counterparty XYZ is asking for all its objects which were delivered to you thru me” can be constructed.
1.3. Specific Object types, for example, a buy side saying “give me list of all single orders that we share”
1.4. Higher level of granularity can be designed - “give me all the Objects we exchanged with each other which have ClOrdId=12345” to query for an Order and its Execution chain.
1.5. Can specify start time + end time / Session / sub session etc.
Application Objects response - Response to 1. Can be sent unsolicited to mean “This is what I think we have been discussing” or “This is what I think you have told me”. This message should be able to represent chains of objects to aid in faster reconstruction of application objects at the receiving end.
Application Objects request reject - Response to signal error(s) in request parameters of request. This is not a fatal error.
Application Objects response ack - Response to signal successfull processing of Objects response. Receipt of this Ack indicates Synchronization complete.
Application Objects response reject - Response to signal failure in processing of response. This is a fatal error.
To illustrate the use of above messages, let me take an example. A buy side BS and sell side SS connect on a FIX Session. Session is stable and the following business messages are exchanged
BS > SS : Order1 Buy 1000 shares of IBM LimitPx 80, TIF Day
BS > SS : Order2 Sell 2000 shares of MSFT LimitPx 40, TIF Day
SS > BS : Exec1 for Order1, 1000 shares at 79.99
SS > BS : Exec2 for Order2, 200 shares at 40.01Now Buy side crashes and recovers. After recovery, on reconnect, BS sends an unsolicited Application objects response which says “I think I sent you Order1 (Buy 1000 shares of IBM LimitPx 80, TIF Day), Order2 (Sell 2000 shares of MSFT LimitPx 40, TIF Day) and you sent me Exec1 for Order1 (1000 shares at 79.99), Exec2 for Order2 (200 shares at 40.01)”. SS receives the unsolicited response and validates it. All Objects in response match objects in SS’s internal memory, so SS sends a response Ack. Buy side receives Ack and knows that Application Objects synchronization is complete and applications are ready to resume business.
Regards,
K. Mahesh
I concur. A rich re-send/recovery logic appears to be at cross-purposes with an efficient lightweight protocol design. Transport and /or session layer are responsible for reliable delivery, after that it’s an application’s responsibility to preserve data properly.
At the session layer, resends may be supported implicitly by allowing peer to specify last received sequence number when connection is initiated.