Imported from previous forum
FIX Tag=Value^ over TCP/IP Session uses optimistic messaging model “normal delivery of data is assumed (i.e. no acknowledgment of individual messages) with errors in delivery identified by message sequence number gaps.”
I propose a compact new HFT Session level “Application Message Receipt Acknowledgement” message (Tag 35 TBD) to let the receipent of a HFT Application message inform its sender “I got your message with sequence number nnnn (details…), I have not yet processed it.” This helps the sending HFT Session engine know a particular HFT message reached its destination HFT Session Engine completely.
Some characteristics of this Acknowledgement message are :-
-
It can contain acknowledgements to multiple messages received by receipent. For example, a Sell side receives two single orders and a DKT in succession, the single acknowledgement message should be able to contain acknowledgment for all three application messages.
-
Acknowledgement could be optional. If a HFT Session Engine receives a message for which a different immediate response is availabe, then a seperate ack for the received message is not needed because the other response would serve as acknowledgement. For example, if a NewOrderSingle message is received with a required field missing and a SLR is sent, then a seperate acknowledgement is not required.
-
Further responses to the acked messages remain as they are currently in FIXProtocol. For example, a CancelReplace message which has been acked, could be followed by present execution report workflows like pending cancel, too late to cancel, fills etc.
Another example:- A Sell side HFT Session engine receives a NewOrderSingle which is “fully correct at Session level”, the HFT Session engine sends ack to order submitter and forwards Order to Trading application. Trading application finds unknown security and sends a Execution report as Order Rejected.
TBD : what information about the “Message being Acknowledged” should be present in the Acknowledgment to make the acknowledgement more useful to the receipent of the Acknowledgment?
This acknowledgement permits a memory optimization in the HFT Session engine sending the original application message - Once an application message receipt has been acked by the receipent, the sending session engine no longer needs to keep a copy of the HFT Application message in its memory to answer a future “from one to infinity resend request”. Any resend requests asking for starting message sequence number less than / equal to last acked message sequence number can be rejected at Session level with Tag 58 text “You said you received them, so I discarded my copies. Sorry”. At this level, Trading Applications behind the HFT Session engines can synchronize themselves using the proposed Application Object Request / Response messages workflow refer:- http://www.fixprotocol.org/discuss/read/0c1ccfb6
Mahesh,
In my view, the proposed solution will have limited value for a number of error situations while adding message processing overhead.
Limited value
An error at the receiving side may render an earlier acknowledgement invalid. Sending a separate ack message should imply that the receiver can (to some extent) guarantee that it is able to retain the received message (or info about the message). This guarantee would generate extra overhead which is contrary to some of the goals of the HFT dev work.
Added processing overhead
The receiver must encode and send an extra message. This will use extra resources in the form of CPU, network bandwidth etc.
Piggy-backing ACKs
One alternative would be to send the acks via an extra field in some other outgoing message. This is similar to how it is done in TCP, which supports both piggy-backed and separate ACKs (although in TCP, the ACK header field is not an extra field).
I’m not convinced this is a feasible alternative and would rather see an asymmetric solution that has exactly-once semantics on client-to-server messages and sequencing on outgoing messages per an earlier post (http://fixprotocol.org/discuss/read/04cfcef5).
Outgoing (result) messages will contain transaction ids of the original requests and the client can use the id to correlate with sent requests. Each request that has generated a response is “acked”.
Side-note: There are obviously situations where the current sequenced “store-and-forward” model is applicable. The discussion above is focused on the HFT scenario.
Best,
Rolf
FIX Tag=Value^ over TCP/IP Session uses optimistic messaging model “normal delivery of data is assumed (i.e. no acknowledgment of individual messages) with errors in delivery identified by message sequence number gaps.”
I propose a compact new HFT Session level “Application Message Receipt Acknowledgement” message (Tag 35 TBD) to let the receipent of a HFT Application message inform its sender “I got your message with sequence number nnnn (details…), I have not yet processed it.” This helps the sending HFT Session engine know a particular HFT message reached its destination HFT Session Engine completely.
Some characteristics of this Acknowledgement message are :-
It can contain acknowledgements to multiple messages received by receipent. For example, a Sell side receives two single orders and a DKT in succession, the single acknowledgement message should be able to contain acknowledgment for all three application messages.
Acknowledgement could be optional. If a HFT Session Engine receives a message for which a different immediate response is availabe, then a seperate ack for the received message is not needed because the other response would serve as acknowledgement. For example, if a NewOrderSingle message is received with a required field missing and a SLR is sent, then a seperate acknowledgement is not required.
Further responses to the acked messages remain as they are currently in FIXProtocol. For example, a CancelReplace message which has been acked, could be followed by present execution report workflows like pending cancel, too late to cancel, fills etc.
Another example:- A Sell side HFT Session engine receives a NewOrderSingle which is “fully correct at Session level”, the HFT Session engine sends ack to order submitter and forwards Order to Trading application. Trading application finds unknown security and sends a Execution report as Order Rejected.TBD : what information about the “Message being Acknowledged” should be present in the Acknowledgment to make the acknowledgement more useful to the receipent of the Acknowledgment?
This acknowledgement permits a memory optimization in the HFT Session engine sending the original application message - Once an application message receipt has been acked by the receipent, the sending session engine no longer needs to keep a copy of the HFT Application message in its memory to answer a future “from one to infinity resend request”. Any resend requests asking for starting message sequence number less than / equal to last acked message sequence number can be rejected at Session level with Tag 58 text “You said you received them, so I discarded my copies. Sorry”. At this level, Trading Applications behind the HFT Session engines can synchronize themselves using the proposed Application Object Request / Response messages workflow refer:- http://www.fixprotocol.org/discuss/read/0c1ccfb6