Imported from previous forum
Hi all,
I’ve got a basic question on message queueing:
In the case of a higher message sequence number than expected, my FIX Engine currently sends a ResendRequest Message and queues all received messages with a higher sequence number than the expected one. After the resending process the messages in the queue will be handled.
This leads to some problems: If the resending was caused driectly after the logon (for example in case of a further crash), the Logon Message will also be queued. The last message of the resend will be a SequenceReset-Gap Fill with exactly the same sequence message as the queued logon, so it will be dropped.
The described scenario above is just one case where I get in trouble with my queue.
After all I come to the conclusion, that I don’t need any queue for incoming messages at all, right? If I use only ResendRequests with EndSeqNo = 0, I will always receive everything from that SeqNo on, so it’s allowed to ignore messages I can’t handle due to a too small sequence number, isn’t it?
What’s your experience with Message queueing? Is this useful for a FIX Client Session or can I get rid of my nasty queue with a clear conscience :-)?
Thanks for any help!
Kind regards,
Eva
[ original email was from John Prewett - jprewett@lavatrading.com ]
Hi Eva,
Your question is a good one.
As you have noticed, there are two totally different strategies that can be used when receiving a high sequence number. The “send thru infinity” and the “fill the gap” methods are your two choices.
Each strategy has different merits.
The “fill the gap” strategy is network friendly as you don’t request that various high sequenced messages that you have already received are retransmitted to you. Unfortunately, it is a significantly more complex problem. You must guarantee to store any high sequence messages while awaiting for a response to your resend request. This can become a technical challenge when sequence gaps occur in the responses to the resend request or other gaps in the messages you are receiving. Another challenge is where you might store these unprocessed high sequence messages as you could receive quite a few.
The “resend thru infinity” strategy is very simple. You discard all high sequence messages and simply wait for the correct sequence number. Unfortunately, all those high sequence messages you are discarding will have to be resent to you.
So the choice is a simple implementation (“resend thru infinity”) versus network friendly (“fill the gap”).
In my experience of “being assaulted” by FIX engines from many different vendors and customers, it is my preference that people use the simple “resend thru infinity” method. This gives rise to fewer problems. An incorrect implementation of the significantly more complex “fill the gap” method can leave the FIX session dead for the day when problems occur and an unrecoverable sequence number error occurs.
On whatever implementation you choose, ensure that your applications speaking FIX can have the sequence numbers reset while they are running. In this manner, a serious sequence number issue can be manually rectified and trading can continue, albeit with reconciliation required for any missing/unprocessed messages.
A common error when implementing the “resend thru infinity” method occurs when receiving multiple messages with a contiguous range of high sequence numbers in a burst. I have seen several FIX engines issue a resend request for each high sequenced message instead of just a single one. This is extremely network unfriendly. You should only issue one resend request per configurable time interval, which is reset upon receipt of the expected sequence number.
I hope this helps.
JohnP
[ original email was from Strat Efstratiou - sefstratiou@bear.com ]
Eva,
As always John is correct in his advice.
Your conclusion is correct, you don’t need any queue for incoming messages. Using ResendRequests (as outlined by John in pervious message) for out of sequence scenarios (sequence number greater then expected) will allow you to ignore those messages.
In my experience message queueing is only useful for FIX Client Sessions that are very sensitive to latency issues. ResendRequests can slowdown message processing. It is your call is the added speed is neccessary or worth the added headaches.
I hope this helps,
Strat…
Hello,
thanks a lot for your help!!
All the best!
Eva