Imported from previous forum
I have two slightly related questions. Although I did some research, I am still not sure about the right answers.
The questions are:
-
Should a Resender process incoming messages during the resend? For example, should an incoming TestRequest be immediately replied to or can it be safely put on a queue until the resend is finished? Is it true that not all but only certain messages types have to be processed during a resend by the resender?
-
Suppose one party misses message 10 to 20, and asks for a resend 10-0. Can this resend-requester expect the resender to send a TestRequest with sequence number 21 in the middle of the resend? And is the resend-requester expected to reply to this TestRequest or can it be safely ignored?
Thanks in advance!
Raymond Hoofman.
[ original email was from Ryan Pierce - rpierce@taltrade.com ]
> 1) Should a Resender process incoming messages during the resend? For example, should an incoming TestRequest be immediately replied to or can it be safely put on a queue until the resend is finished? Is it true that not all but only certain messages types have to be processed during a resend by the resender?
Messages must be processed in order. The only exceptions are listed in a table in the spec explicitly (page 14 for 4.2), and Test Request is NOT one of those exceptions. Hence if you get an out of sequence Test Request, do NOT respond to it; send a Resend Request instead.
The fact that Test Request must be processed in sequence is used by some FIX implementations as a way to query whether their counterparty party is in sync. If one party sends a test request and receives a response, that party can then assume that their counterparty has received all messages up to and including that Test Request.
> 2) Suppose one party misses message 10 to 20, and asks for a resend 10-0. Can this resend-requester expect the resender to send a TestRequest with sequence number 21 in the middle of the resend? And is the resend-requester expected to reply to this TestRequest or can it be safely ignored?
In general, admin messages are not resent. Hence the test request isn’t likely to be resent. The other party should gap fill over it, and if so, it can be safely ignored. The other party should not penalize you for not responding specifically to a Test Request they gap fill over by dropping the connection. However, the other party may send you a new Test Request message, and if received in sequence, you are required to respond.
> > 1) Should a Resender process incoming messages during the resend? For example, should an incoming TestRequest be immediately replied to or can it be safely put on a queue until the resend is finished?
Since we have a difference of opinion with our FIX counterparty over exactly this issue we would like to have a further clarification.
Suppose we ask for a resend xxxx untill infinity. While the other party is resending, we send a Test Request. Should the other party send the reply AFTER processing the resend request, or can the reply be mixed with the resent messages.
Regards
Wilbert Kroon
Compaq professional Services
[ original email was from Ryan Pierce - rpierce@taltrade.com ]
> Suppose we ask for a resend xxxx untill infinity. While the other party is resending, we send a Test Request. Should the other party send the reply AFTER processing the resend request, or can the reply be mixed with the resent messages.
FIX requires ordered message processing.
So the other party in this case receives two messages:
Resend Request xxxx-0
Test Request
That party must FIRST honor the Resend Request completely, and THEN send the Heartbeat back in response to the Test Request.
Are you saying that if the other party is resending 50-60, that party is sending 50-55, then 61 Heartbeat, then 56-60? If so, that is likely a problem. The other side needs to send 50-60, and only then send 61 Heartbeat in response to your Test Request.
> Are you saying that if the other party is resending 50-60, that party is sending 50-55, then 61 Heartbeat, then 56-60? If so, that is likely a problem. The other side needs to send 50-60, and only then send 61 Heartbeat in response to your Test Request.
Yes that is exactly what I am saying. Our FIX engine can deal with this (by again asking for a resend when the 61 comes in!) but we already suspected that what our counterparty is doing is not correct FIX behaviour.
Thanks everybody (online and offline) for the explanations, with this info we will try to convince our counterparty to modify their FIX engine.
[ original email was from Matt Koehler - matt.koehler@transacttools.net ]
> Suppose we ask for a resend xxxx untill infinity. While the other party is resending, we send a Test Request. Should the other party send the reply AFTER processing the resend request, or can the reply be mixed with the resent messages.
The 4.2 spec says "It is also recommended that an engine should store out of sequence messages in a temporary queue and process them in order when the gap is closed. This prevents generating resend requests for n->m, n->m+1, n->m+2, … which can result in many resent PossDupFlag=Y messages." In your example, the reply would be out of sequence from the resend request, and ought to be sent after (according to the spec.)
BUT, I’ve dealt with FIX engines that disconnect if you don’t respond almost immediately to a specific test request, despite the fact that you might already be sending tons of other messages as part of a resend (implying a healthy connection). So it kinda depends on your counterparty, but if you’re asking what the spec says, check p. 12 (where i got that quote from.)
-Matt
[ original email was from Ryan Pierce - rpierce@taltrade.com ]
> The 4.2 documentation confusingly appears to me to recommend both forms of resend request over each other. Apart from the quote Matt mentioned, the "resend request" documentation discusses the two forms of resend request and has the following comment about the "to infinity" method : "This latter approach is strongly recommended to recover from out of sequence conditions as it …".
It is my understanding that the best way to handle an out of sequence condition is to send a Resend Request from X to 0.
However, if you take the approach of receiving Y when expecting X, and sending a Resend Request from X to Y-1, then it is best to keep Y in a queue, and process it after Y-1. Otherwise, if you discard Y, then you will receive Y+1 next, be expecting Y, and have to Resend Request Y again. And even then I do not think you are in sync if you’ve tossed out Y+1, becauce you are then expecting Y+1 and the next message the other party will send you is Y+2.
The explicit range requests are generally much mode difficult to handle to do right, and jn active trading are prone to race conditions that can make convergence take a long time. Requesting from X-0 is thus recommended.