Imported from previous forum
[ original email was from Steve Wilkinson - steve.wilkinson@solutionforge.com ]
Consider the following scenario (counterparty seq nums omitted for clarity):
We connect to counterparty after broken TCP session.
We send: Logon, MsgSeqNum=10
Counterparty sends: Logon
Counterparty sends: Test Request
We send: Heartbeat (in response to Test Req), MsgSeqNum=11
Counterparty sends: Resend Request, BeginSeqNo=7, EndSeqNo=10
We send: Business Msg, MsgSeqNum=7, PossDup=Y
We send: Business Msg, MsgSeqNum=8, PossDup=Y
We send: Sequence Reset, GapFillFlag=Y, MsgSeqNum=9,
NewSeqNo=11, PossDup=Y
We send: Heartbeat, MsgSeqNum=12
Our understanding is that the gap fill only fills over the messages requested in the resend request, so we’re doing the right thing.
Unfortunately, our counterparty drops the session because they believe we’ve lowered the next expected sequence number in our Sequence Reset message. This is because the FIX 4.1 spec says of the Sequence Reset message: “The message in all situations specifies NewSeqNo to reset as the value of the next sequence number to be transmitted.”
In the example, their interpretation is that we should set the NewSeqNo of the Sequence Reset message to 12, but that means that if message 11 had been a business message, it would have the potential to get lost.
(Obviously the situation would have been different if they’d asked for 7 to 999999, in which case we would have safely gap-filled up to 12.)
Can anyone advise/comment?
Thanks - Steve.
[ original email was from Himanshu Sharma - himanshu_sharma@rsco.com ]
> Consider the following scenario (counterparty seq nums omitted for clarity):
>
> We connect to counterparty after broken TCP session.
>
> We send: Logon, MsgSeqNum=10
>
> Counterparty sends: Logon
> Counterparty sends: Test Request
>
> We send: Heartbeat (in response to Test Req), MsgSeqNum=11
>
> Counterparty sends: Resend Request, BeginSeqNo=7, EndSeqNo=10
>
> We send: Business Msg, MsgSeqNum=7, PossDup=Y
> We send: Business Msg, MsgSeqNum=8, PossDup=Y
> We send: Sequence Reset, GapFillFlag=Y, MsgSeqNum=9,
> NewSeqNo=11, PossDup=Y
>
> We send: Heartbeat, MsgSeqNum=12
>
> Our understanding is that the gap fill only fills over the messages requested in the resend request, so we’re doing the right thing.
>
> Unfortunately, our counterparty drops the session because they believe we’ve lowered the next expected sequence number in our Sequence Reset message. This is because the FIX 4.1 spec says of the Sequence Reset message: “The message in all situations specifies NewSeqNo to reset as the value of the next sequence number to be transmitted.”
>
> In the example, their interpretation is that we should set the NewSeqNo of the Sequence Reset message to 12, but that means that if message 11 had been a business message, it would have the potential to get lost.
>
> (Obviously the situation would have been different if they’d asked for 7 to 999999, in which case we would have safely gap-filled up to 12.)
>
> Can anyone advise/comment?
>
> Thanks - Steve.
>
>
Really tricky! FIX protocol should be more precise at these points.
I would suggest the following change in your message:
We send: Sequence Reset, GapFillFlag=N, MsgSeqNum=9,
NewSeqNo=11.
In FIX protocol (FIX4.0 Sequence Reset (Gap Fill)) it has been specifically mentioned that in the event of application faipure we should set GapFillFlag=N so that counterparty ignores the MsgSeqNum and NewSeqNum becomes our next outgoingSeqNum.
[ original email was from scott gietler - scott@transacttools.net ]
You are stumbling on some of the ambiguities of the FIX spec… basically, there two ways someone can implement what happens when they receive a seq. number too high (like seq. #11). One, they can queue the message, and ask for 7-10, Two, they can throw it away, and ask for 7-infinity. A third way would be to ask for 7-10 without queuing the message, but this takes longer to stabilize.
It seems like your counterparty is doing the first case.
So in your case, what you are doing is technically correct.
Maybe your counterparty can’t handle gap fills. Your counterparty must process messages in order, which means they must process 7-10 before 11, and 11 before 12. So after they receive your gap fill, the next seq. number they process has to be 11, not 12. They may have a bug.
A second option is for your engine to have the ability, on a counterparty basis, to treat all resend requests as ending in infinity (this would be configured on a per connection basis.) This means you would treat a resend request for 7-10 like it was 7-infinity.
Scott Gietler
TransactTools
> Consider the following scenario (counterparty seq nums omitted for clarity):
>
> We connect to counterparty after broken TCP session.
>
> We send: Logon, MsgSeqNum=10
>
> Counterparty sends: Logon
> Counterparty sends: Test Request
>
> We send: Heartbeat (in response to Test Req), MsgSeqNum=11
>
> Counterparty sends: Resend Request, BeginSeqNo=7, EndSeqNo=10
>
> We send: Business Msg, MsgSeqNum=7, PossDup=Y
> We send: Business Msg, MsgSeqNum=8, PossDup=Y
> We send: Sequence Reset, GapFillFlag=Y, MsgSeqNum=9,
> NewSeqNo=11, PossDup=Y
>
> We send: Heartbeat, MsgSeqNum=12
>
> Our understanding is that the gap fill only fills over the messages requested in the resend request, so we’re doing the right thing.
>
> Unfortunately, our counterparty drops the session because they believe we’ve lowered the next expected sequence number in our Sequence Reset message. This is because the FIX 4.1 spec says of the Sequence Reset message: “The message in all situations specifies NewSeqNo to reset as the value of the next sequence number to be transmitted.”
>
> In the example, their interpretation is that we should set the NewSeqNo of the Sequence Reset message to 12, but that means that if message 11 had been a business message, it would have the potential to get lost.
>
> (Obviously the situation would have been different if they’d asked for 7 to 999999, in which case we would have safely gap-filled up to 12.)
>
> Can anyone advise/comment?
>
> Thanks - Steve.
>
>
Looks like you did it right. Clearly, you should be able to perform sequence resets with the newseqno < your real OSN. Consider the following
you send:
heartbeat with seqno 5
execution with seqno 6
heartbeat with seqno 7
logon with seqno 8
other party sends:
logon with seqno 5
resend with seqno 6, start = 5 end = 8
your heartbeat with sequence number 5 must be gap filled. If the new sequence number in your gap fill is 9 instead of 6, then the other party will never process you resend of the execution report.
I know the above is not the same scenario you described, but it should make it clear that quote
from the spec about the newseqno should not be interpretted as your counterparty is interpretting it.
> Consider the following scenario (counterparty seq nums omitted for clarity):
>
> We connect to counterparty after broken TCP session.
>
> We send: Logon, MsgSeqNum=10
>
> Counterparty sends: Logon
> Counterparty sends: Test Request
>
> We send: Heartbeat (in response to Test Req), MsgSeqNum=11
>
> Counterparty sends: Resend Request, BeginSeqNo=7, EndSeqNo=10
>
> We send: Business Msg, MsgSeqNum=7, PossDup=Y
> We send: Business Msg, MsgSeqNum=8, PossDup=Y
> We send: Sequence Reset, GapFillFlag=Y, MsgSeqNum=9,
> NewSeqNo=11, PossDup=Y
>
> We send: Heartbeat, MsgSeqNum=12
>
> Our understanding is that the gap fill only fills over the messages requested in the resend request, so we’re doing the right thing.
>
> Unfortunately, our counterparty drops the session because they believe we’ve lowered the next expected sequence number in our Sequence Reset message. This is because the FIX 4.1 spec says of the Sequence Reset message: “The message in all situations specifies NewSeqNo to reset as the value of the next sequence number to be transmitted.”
>
> In the example, their interpretation is that we should set the NewSeqNo of the Sequence Reset message to 12, but that means that if message 11 had been a business message, it would have the potential to get lost.
>
> (Obviously the situation would have been different if they’d asked for 7 to 999999, in which case we would have safely gap-filled up to 12.)
>
> Can anyone advise/comment?
>
> Thanks - Steve.
>
>
I agree with the others that you are responding properly and your counterparty should not interpret the Gap Fill as lowering the seq num. The counterparty can queue the "too high" message but cannot consider the seq num consumed or set to that level.
Also, I think your counterparty should have sent their ResendRequest immediately following their Logon response (vs. the Test Request as documented in your example) as they should have detected the gap off of your Logon.
> Consider the following scenario (counterparty seq nums omitted for clarity):
>
> We connect to counterparty after broken TCP session.
>
> We send: Logon, MsgSeqNum=10
>
> Counterparty sends: Logon
> Counterparty sends: Test Request
>
> We send: Heartbeat (in response to Test Req), MsgSeqNum=11
>
> Counterparty sends: Resend Request, BeginSeqNo=7, EndSeqNo=10
>
> We send: Business Msg, MsgSeqNum=7, PossDup=Y
> We send: Business Msg, MsgSeqNum=8, PossDup=Y
> We send: Sequence Reset, GapFillFlag=Y, MsgSeqNum=9,
> NewSeqNo=11, PossDup=Y
>
> We send: Heartbeat, MsgSeqNum=12
>
> Our understanding is that the gap fill only fills over the messages requested in the resend request, so we’re doing the right thing.
>
> Unfortunately, our counterparty drops the session because they believe we’ve lowered the next expected sequence number in our Sequence Reset message. This is because the FIX 4.1 spec says of the Sequence Reset message: “The message in all situations specifies NewSeqNo to reset as the value of the next sequence number to be transmitted.”
>
> In the example, their interpretation is that we should set the NewSeqNo of the Sequence Reset message to 12, but that means that if message 11 had been a business message, it would have the potential to get lost.
>
> (Obviously the situation would have been different if they’d asked for 7 to 999999, in which case we would have safely gap-filled up to 12.)
>
> Can anyone advise/comment?
>
> Thanks - Steve.
>
>