Imported from previous forum
[ original email was from Valentin Melamed - valo@easetech.com ]
I am posting this here because there is almost no difference between the session protocols for 4.0 and 4.1.
The spec. says on pg.15: "Note: it is imperative that the receiving application process messages in sequence order, e.g. if message number 7 is missed and 8-9 received, the application should ignore 8 and 9 and ask for a resend of 7-9."
Suppose both parties are missing the top 3-4 messages. They both issue resend requests but since they have gap, they ignore the others resend request and keep sending their own resend requests.
Is there anything in the protocol which would prevent from such a dead-lock or the wording should be changed so that only business messages are to be ignored and admin messages (or only ResendRequest) should be processed regardless of any gaps.
Val
[ original email was from Benedict Zoe - bzoe@bloomberg.net ]
I "ignore" mismatch sequence numbers on ResendRequest for this very reason. At the worst, the sender could get multiple dup messages but there will not be any deadlocks.
The mismatch itself will cause my side to trigger off ResendRequests.
[ original email was from Valentin Melamed - valo@easetech.com ]
> I "ignore" mismatch sequence numbers on ResendRequest for this very reason. At the worst, the sender could get multiple dup messages but there will not be any deadlocks.
>
> The mismatch itself will cause my side to trigger off ResendRequests.
>
Yes, I guess this is what I am suggesting (and what I decided to do) as a change in the wording. Shouldn’t this spread over all admin messages though? I still have hard time picturing how ResendRequest-s would work with Logout-s. Seems to me, that Logon is a message which should never be ignored. Moreover for the sell side at logon time before processing the message itself the session layer should not bother with sequence numbers since it still hasn’t identified the session.
How about TestRequest?
The HeartBeat-s should at least reset the recieve timeout, which is equivalent to not ignoring it.
How about Reject?
My personal opinion for now is that the admin messages should not be ignored. Any thoughts?
Val
[ original email was from Ryan Pierce - rpierce@taltrade.com ]
> Yes, I guess this is what I am suggesting (and what I decided to do) as a change in the wording. Shouldn’t this spread over all admin messages though?
No. Doing so would actually introduce lots of problems.
As Scott Atwell pointed out, the spec clearly states which messages to process should they be a higher sequence number than expected, and which to ignore and just send a Resend Request. Note that if the sequence number is less than expected, and the message is not Poss Dupe, then you are required, regardless of the message type, to send a Logout with a descriptive text message and drop the connection.
>I still have hard time picturing how ResendRequest-s would work with Logout-s. Seems to me, that Logon is a message which should never be ignored.
You’re right. The spec states clearly that Logon should be processed anyway. You send your own Logon in response, and then a Resend Request for what you missed. (Preferably requesting from where they left off through 999999 to avoid a separate issue I’m about to post about later today.) The other side is required to process your Resend Request, even if they see a higher sequence number than they expect. They are required to first resend the messages, and then send their own Resend Request for what they missed. (Reversing the order can lead to infinite loops.)
> How about TestRequest?
The table does not list Test Request explicitly, so it is something which should not be processed. Drop it and send a Resend Request. In fact, I know of some FIX implementations which use Test Request as an explicit confirmation of receiving all messages up to that point, and responding to a Test Request that is out of sequence completely defeats the validity of that check.
> The HeartBeat-s should at least reset the recieve timeout, which is equivalent to not ignoring it.
Yes, it should reset the incoming count, but I wouldn’t consider it “not ignoring” the heartbeat. The spec states: “When either end of the connection has not received any data for (HeartBtInt + “some reasonable transmission time”) seconds, it will transmit a Test Request message.” Receiving out of sequence data is still data received, so your inactivity timer should be reset regardless of the sequence number of the messages. Technically, you should reset the inactivity timer on receipt of random line noise, although I think in practice most people reset theirs only after receipt of a FIX message with valid formatting.
> How about Reject?
It is not specifically listed, so it should not be processed out of order. Note that Reject is a somewhat special kind of admin message, as it can be retransmitted. So if you receive a Reject out of sequence, you should drop it, send a Resend Request, and the other side should send you the skipped messages as well as the Reject. The other side should not Gap Fill over the Reject but should retransmit it Poss Dupe.
> My personal opinion for now is that the admin messages should not be ignored. Any thoughts?
I disagree. FIX relies upon ordered processing of messages, and only those messages which absolutely must be processed out of order (Logon, Logout, Resend Request, and Sequence Reset that is not a Gap Fill) can be processed out of order.
In addition to the Test Request example I gave, another good example of a problem caused by processing admins out of order is a Sequence Reset - Gap Fill. If I am expecting 10 and receive 15 Gap Fill to 20, then should I process it and set my next expected incoming sequence number to 20, I’ve just lost messages 10-14.
Please refer to the table "Response by Message Type" within the "Message Recovery" section of the spec. There are certain messages, ResendRequest, included which should be "processed" before sending a ResendRequest.
> I am posting this here because there is almost no difference between the session protocols for 4.0 and 4.1.
>
> The spec. says on pg.15: "Note: it is imperative that the receiving application process messages in sequence order, e.g. if message number 7 is missed and 8-9 received, the application should ignore 8 and 9 and ask for a resend of 7-9."
>
> Suppose both parties are missing the top 3-4 messages. They both issue resend requests but since they have gap, they ignore the others resend request and keep sending their own resend requests.
>
> Is there anything in the protocol which would prevent from such a dead-lock or the wording should be changed so that only business messages are to be ignored and admin messages (or only ResendRequest) should be processed regardless of any gaps.
>
> Val
>