Imported from previous forum
Hello all,
If the incoming sequence number is greater than expected,it indicates that messages are missed. We need to use Gap fill mechanism o retieve the missed sequences.
The way I understood this is this situation occurs only if the administrative messages (like HeartBeat) comes in between the business messages. So, while constructing the Fix message we can give some special number ( for eg: “-1”) as the sequence number for the administrative messages AND at the recieving end we check for the type of the message. If it is administrative message then discard that messsage and expect the same number as it was before.
Please correct me, if I am wrongly interpreting the mechanism.
Thanks & Regards
Bhasker Reddy A.
No,
if you see there are missing messages you use the resend request to ask for those missing message.
Say you are missing messages 6,7,8,9,10.
if 8 and 9 are admin message (eg heartbeat) then these will not be sent to you, instead 6,7 will be sent, then a gapfill message for 8 and 9, and then message 10 will be sent to you.
Hello all, If the incoming sequence number is greater than expected,it
indicates that messages are missed. We need to use Gap fill mechanism o
retieve the missed sequences. The way I understood this is this
situation occurs only if the administrative messages (like HeartBeat)
comes in between the business messages. So, while constructing the Fix
message we can give some special number ( for eg: “-1”) as the sequence
number for the administrative messages AND at the recieving end we check
for the type of the message. If it is administrative message then
discard that messsage and expect the same number as it was before.Please correct me, if I am wrongly interpreting the mechanism.
Thanks & Regards Bhasker Reddy A.
My apologies for not mentioning this before …
Actually I was talking about the way we can avoid using gap fill mechanism. The mechanism is as follows …
while constructing the Fix message we can give some special number ( for eg: “-1”) as the sequence number for the administrative messages AND at the recieving end, we check for the type of the message. If it is administrative message then discard that messsage and expect the same number as it was before.
Correct me, If I am wrong.
No,
if you see there are missing messages you use the resend request to ask
for those missing message.Say you are missing messages 6,7,8,9,10.
if 8 and 9 are admin message (eg heartbeat) then these will not be sent
to you, instead 6,7 will be sent, then a gapfill message for 8 and 9,
and then message 10 will be sent to you.
Hello all, If the incoming sequence number is greater than expected,it
indicates that messages are missed. We need to use Gap fill mechanism
o retieve the missed sequences. The way I understood this is this
situation occurs only if the administrative messages (like HeartBeat)
comes in between the business messages. So, while constructing the Fix
message we can give some special number ( for eg: “-1”) as the
sequence number for the administrative messages AND at the recieving
end we check for the type of the message. If it is administrative
message then discard that messsage and expect the same number as it
was before.Please correct me, if I am wrongly interpreting the mechanism.
Thanks & Regards Bhasker Reddy A.
Sure you could do that, but its not part of the FIX protocol and if you connect to any other FIx engines then it wont work?
It also completely breaks the reliability of the messaging system.
for example : if you receive a business message of 8 and then a logoff with -1, how do you know if you have not missed business message 9, 10 etc… ?
My apologies for not mentioning this before …
Actually I was talking about the way we can avoid using gap fill
mechanism. The mechanism is as follows …while constructing the Fix message we can give some special number ( for
eg: “-1”) as the sequence number for the administrative messages AND at
the recieving end, we check for the type of the message. If it is
administrative message then discard that messsage and expect the same
number as it was before.Correct me, If I am wrong.
No,
if you see there are missing messages you use the resend request to
ask for those missing message.Say you are missing messages 6,7,8,9,10.
if 8 and 9 are admin message (eg heartbeat) then these will not be
sent to you, instead 6,7 will be sent, then a gapfill message for 8
and 9, and then message 10 will be sent to you.
Hello all, If the incoming sequence number is greater than
expected,it indicates that messages are missed. We need to use Gap
fill mechanism o retieve the missed sequences. The way I understood
this is this situation occurs only if the administrative messages
(like HeartBeat) comes in between the business messages. So, while
constructing the Fix message we can give some special number ( for
eg: “-1”) as the sequence number for the administrative messages AND
at the recieving end we check for the type of the message. If it is
administrative message then discard that messsage and expect the
same number as it was before.Please correct me, if I am wrongly interpreting the mechanism.
Thanks & Regards Bhasker Reddy A.
My apologies for my partial knowledge of FIX.
Here I don’t understand how will it breaks the reliability of messaging system?
As per your example, if it logsoff after recieving 8 then it wont recieve 9 & 10. Since the sequence number is persistent on both sides, We can transmit from 9 onwards when we logon next time.
Correct me, If I am wrong
Bhasker Reddy
Sure you could do that, but its not part of the FIX protocol and if you
connect to any other FIx engines then it wont work?It also completely breaks the reliability of the messaging system.
for example : if you receive a business message of 8 and then a logoff
with -1, how do you know if you have not missed business message 9, 10
etc… ?My apologies for not mentioning this before …
Actually I was talking about the way we can avoid using gap fill
mechanism. The mechanism is as follows …while constructing the Fix message we can give some special number (
for eg: “-1”) as the sequence number for the administrative messages
AND at the recieving end, we check for the type of the message. If it
is administrative message then discard that messsage and expect the
same number as it was before.Correct me, If I am wrong.
No,
if you see there are missing messages you use the resend request to
ask for those missing message.Say you are missing messages 6,7,8,9,10.
if 8 and 9 are admin message (eg heartbeat) then these will not be
sent to you, instead 6,7 will be sent, then a gapfill message for 8
and 9, and then message 10 will be sent to you.
Hello all, If the incoming sequence number is greater than
expected,it indicates that messages are missed. We need to use Gap
fill mechanism o retieve the missed sequences. The way I
understood this is this situation occurs only if the
administrative messages (like HeartBeat) comes in between the
business messages. So, while constructing the Fix message we can
give some special number ( for eg: “-1”) as the sequence number
for the administrative messages AND at the recieving end we check
for the type of the message. If it is administrative message then
discard that messsage and expect the same number as it was before.Please correct me, if I am wrongly interpreting the mechanism.
Thanks & Regards Bhasker Reddy A.
If you have received message 8, and then logged off, but the counterparty has sent you messages 9 & 10, then the next message you will receive will be number 11. This means that you will need to request the missing messages.
FIX is not constrained to use TCP so we cannot necessarily guarantee the order and delivery of messages.
As for gap fill - it is not used only for administrative messages, but it can be used for any messages you like (e.g. stale quotes or orders).
Regards,
James
My apologies for my partial knowledge of FIX.
Here I don’t understand how will it breaks the reliability of
messaging system?As per your example, if it logsoff after recieving 8 then it wont
recieve 9 & 10. Since the sequence number is persistent on both sides,
We can transmit from 9 onwards when we logon next time.Correct me, If I am wrong
Bhasker Reddy
Sure you could do that, but its not part of the FIX protocol and if
you connect to any other FIx engines then it wont work?It also completely breaks the reliability of the messaging system.
for example : if you receive a business message of 8 and then a logoff
with -1, how do you know if you have not missed business message 9, 10
etc… ?My apologies for not mentioning this before …
Actually I was talking about the way we can avoid using gap fill
mechanism. The mechanism is as follows …while constructing the Fix message we can give some special number (
for eg: “-1”) as the sequence number for the administrative messages
AND at the recieving end, we check for the type of the message. If
it is administrative message then discard that messsage and expect
the same number as it was before.Correct me, If I am wrong.
No,
if you see there are missing messages you use the resend request
to ask for those missing message.Say you are missing messages 6,7,8,9,10.
if 8 and 9 are admin message (eg heartbeat) then these will not be
sent to you, instead 6,7 will be sent, then a gapfill message for
8 and 9, and then message 10 will be sent to you.
Hello all, If the incoming sequence number is greater than
expected,it indicates that messages are missed. We need to use
Gap fill mechanism o retieve the missed sequences. The way I
understood this is this situation occurs only if the
administrative messages (like HeartBeat) comes in between the
business messages. So, while constructing the Fix message we can
give some special number ( for eg: “-1”) as the sequence number
for the administrative messages AND at the recieving end we
check for the type of the message. If it is administrative
message then discard that messsage and expect the same number as
it was before.Please correct me, if I am wrongly interpreting the mechanism.
Thanks & Regards Bhasker Reddy A.
Although it would be against FIX protocol, Bhasker’s method should work in the case that he wants to avoid using Gap Fills and he has implmented both the client and server side, however, as the previous poster suggested, his engine would not work with other engines as a seq. number of -1 would most likely be dk’d.
In the example, if he receives a 11, he would send a resend from 9, which would be fine, because assuming 9 and 10 are business messages and all of the admin messages are sequenced with -1, only the business messages would be resent (assuming this is what he wants), however, as previously suggested, Gap Fills can be used for any sort of message that you would like to ignore, but at the same time, you could just sequence these with a -1. Regardless, your engine wouldn’t work properly with someone else’s.
If you have received message 8, and then logged off, but the
counterparty has sent you messages 9 & 10, then the next message you
will receive will be number 11. This means that you will need to request
the missing messages.FIX is not constrained to use TCP so we cannot necessarily guarantee the
order and delivery of messages.As for gap fill - it is not used only for administrative messages, but
it can be used for any messages you like (e.g. stale quotes or orders).Regards,
James
My apologies for my partial knowledge of FIX.
Here I don’t understand how will it breaks the reliability of
messaging system?As per your example, if it logsoff after recieving 8 then it wont
recieve 9 & 10. Since the sequence number is persistent on both sides,
We can transmit from 9 onwards when we logon next time.Correct me, If I am wrong
Bhasker Reddy
Sure you could do that, but its not part of the FIX protocol and if
you connect to any other FIx engines then it wont work?It also completely breaks the reliability of the messaging system.
for example : if you receive a business message of 8 and then a
logoff with -1, how do you know if you have not missed business
message 9, 10 etc… ?My apologies for not mentioning this before …
Actually I was talking about the way we can avoid using gap fill
mechanism. The mechanism is as follows …while constructing the Fix message we can give some special number
( for eg: “-1”) as the sequence number for the administrative
messages AND at the recieving end, we check for the type of the
message. If it is administrative message then discard that
messsage and expect the same number as it was before.Correct me, If I am wrong.
No,
if you see there are missing messages you use the resend request
to ask for those missing message.Say you are missing messages 6,7,8,9,10.
if 8 and 9 are admin message (eg heartbeat) then these will not
be sent to you, instead 6,7 will be sent, then a gapfill message
for 8 and 9, and then message 10 will be sent to you.
Hello all, If the incoming sequence number is greater than
expected,it indicates that messages are missed. We need to use
Gap fill mechanism o retieve the missed sequences. The way I
understood this is this situation occurs only if the
administrative messages (like HeartBeat) comes in between the
business messages. So, while constructing the Fix message we
can give some special number ( for eg: “-1”) as the sequence
number for the administrative messages AND at the recieving
end we check for the type of the message. If it is
administrative message then discard that messsage and expect
the same number as it was before.Please correct me, if I am wrongly interpreting the
mechanism.Thanks & Regards Bhasker Reddy A.
Perhaps I am missing something but it seems to me that this system would have a few undesirable side effects:
If all admin messages are sent with sequence number -1 then lets say that a client sends three orders and receives 3 opens. Then the client disconnects, while disconnected the three orders are filled. When the client reconnects he will have no idea that any messages were missed because the Logon messages (test request and so forth) all have the sequence number of -1 in this case, and thus do not cause a resend request. The client will only learn of the fills after he sends his first “business message”.
I don’t think that many traders would be happy hearing you say “don’t worry you will find out what happened to those orders just as soon as you send another!”.
Hello all, If the incoming sequence number is greater than expected,it
indicates that messages are missed. We need to use Gap fill mechanism o
retieve the missed sequences. The way I understood this is this
situation occurs only if the administrative messages (like HeartBeat)
comes in between the business messages. So, while constructing the Fix
message we can give some special number ( for eg: “-1”) as the sequence
number for the administrative messages AND at the recieving end we check
for the type of the message. If it is administrative message then
discard that messsage and expect the same number as it was before.Please correct me, if I am wrongly interpreting the mechanism.
Thanks & Regards Bhasker Reddy A.
Perhaps I am missing something but it seems to me that this system would
have a few undesirable side effects:If all admin messages are sent with sequence number -1 then lets say
that a client sends three orders and receives 3 opens. Then the client
disconnects, while disconnected the three orders are filled. When the
client reconnects he will have no idea that any messages were missed
because the Logon messages (test request and so forth) all have the
sequence number of -1 in this case, and thus do not cause a resend
request. The client will only learn of the fills after he sends his
first “business message”.
Since sequence number is persistent on both client & server side. On client side if it is a administrative message then we don’t update that sequence number in database( persistence ). Even if client disconnects, when reconnects next time it synchronises with the server side sequence number.
I don’t think that many traders would be happy hearing you say “don’t
worry you will find out what happened to those orders just as soon as
you send another!”.Hello all, If the incoming sequence number is greater than expected,it
indicates that messages are missed. We need to use Gap fill mechanism
o retieve the missed sequences. The way I understood this is this
situation occurs only if the administrative messages (like HeartBeat)
comes in between the business messages. So, while constructing the Fix
message we can give some special number ( for eg: “-1”) as the
sequence number for the administrative messages AND at the recieving
end we check for the type of the message. If it is administrative
message then discard that messsage and expect the same number as it
was before.Please correct me, if I am wrongly interpreting the mechanism.
Thanks & Regards Bhasker Reddy A.
Perhaps I am missing something but it seems to me that this system
would have a few undesirable side effects:If all admin messages are sent with sequence number -1 then lets say
that a client sends three orders and receives 3 opens. Then the client
disconnects, while disconnected the three orders are filled. When the
client reconnects he will have no idea that any messages were missed
because the Logon messages (test request and so forth) all have the
sequence number of -1 in this case, and thus do not cause a resend
request. The client will only learn of the fills after he sends his
first “business message”.Since sequence number is persistent on both client & server side. On
client side if it is a administrative message then we don’t update
that sequence number in database( persistence ). Even if client
disconnects, when reconnects next time it synchronises with the server
side sequence number.
So do administrative messages have “real” sequence numbers when they are first sent and then -1 in response to a resend request?
If you do not allow administrative messages (logons, heartbeats, testrequests etc …) to increment the sequence then after a disconnect you will have to have special logic on both sides to synchronize the server and client.
Tell me where I am wrong-
–The day starts fix engines find each other–
…logon heartbeat etc all 34=-1
–First business message of the day–
client -sends order (35=D) 34=1
server -ack’s order (35=8) 39=0 34=1
–client disconnects with no logout–
server -sends order filled(39=2) 34=2
–client comes back–
…logon hearbeat etc… 34=-1
The server may not know that the client did not receive the fill, the sockets could even have been connected but the client code was in the middle of a core dump or whatever. When the client comes back and logs on and receives the logon ack he cannot use the value of 34 (sequence number) to establish the fact that he did indeed miss messages (since all admin messages have the sequence number of -1). So what you would now have to do is either wait for another business message to be sent by either party which would trigger a resend request or build in other handshake logic to make sure that no messages were dropped in the interum.
So what mechanism do you use to synchronize the server and the client?
And why go to all this trouble just to “avoid” a sequence reset message?
I don’t think that many traders would be happy hearing you say “don’t
worry you will find out what happened to those orders just as soon as
you send another!”.Hello all, If the incoming sequence number is greater than
expected,it indicates that messages are missed. We need to use Gap
fill mechanism o retieve the missed sequences. The way I understood
this is this situation occurs only if the administrative messages
(like HeartBeat) comes in between the business messages. So, while
constructing the Fix message we can give some special number ( for
eg: “-1”) as the sequence number for the administrative messages AND
at the recieving end we check for the type of the message. If it is
administrative message then discard that messsage and expect the
same number as it was before.Please correct me, if I am wrongly interpreting the mechanism.
Thanks & Regards Bhasker Reddy A.
Unless the logon messages are assigned “real” seq. numbers, so that you can figure out that you’re out of synch right at logon, or there will be a delay as suggested until the next “real” seq number/business message shows up, but then you will still need to gap fill these logon messages.
Also, you wold not be able to connect to any other FIX provider or have another provider connect to you.
Do you want to avoid using the seq. reset message? or just the Gap Fill portion of it? Seq. reset is quite essential.
Perhaps I am missing something but it seems to me that this system
would have a few undesirable side effects:If all admin messages are sent with sequence number -1 then lets say
that a client sends three orders and receives 3 opens. Then the
client disconnects, while disconnected the three orders are filled.
When the client reconnects he will have no idea that any messages
were missed because the Logon messages (test request and so forth)
all have the sequence number of -1 in this case, and thus do not
cause a resend request. The client will only learn of the fills
after he sends his first “business message”.Since sequence number is persistent on both client & server side. On
client side if it is a administrative message then we don’t update
that sequence number in database( persistence ). Even if client
disconnects, when reconnects next time it synchronises with the server
side sequence number.So do administrative messages have “real” sequence numbers when they are
first sent and then -1 in response to a resend request?If you do not allow administrative messages (logons, heartbeats,
testrequests etc …) to increment the sequence then after a disconnect
you will have to have special logic on both sides to synchronize the
server and client.Tell me where I am wrong- --The day starts fix engines find each other–
…logon heartbeat etc all 34=-1 --First business message of the day–client -sends order (35=D) 34=1 server -ack’s order (35=8) 39=0 34=1
–client disconnects with no logout-- server -sends order filled(39=2)
34=2 --client comes back-- …logon hearbeat etc… 34=-1The server may not know that the client did not receive the fill, the
sockets could even have been connected but the client code was in the
middle of a core dump or whatever. When the client comes back and
logs on and receives the logon ack he cannot use the value of 34
(sequence number) to establish the fact that he did indeed miss
messages (since all admin messages have the sequence number of -1).
So what you would now have to do is either wait for another business
message to be sent by either party which would trigger a resend
request or build in other handshake logic to make sure that no
messages were dropped in the interum.So what mechanism do you use to synchronize the server and the client?
And why go to all this trouble just to “avoid” a sequence reset message?
I don’t think that many traders would be happy hearing you say
“don’t worry you will find out what happened to those orders just as
soon as you send another!”.Hello all, If the incoming sequence number is greater than
expected,it indicates that messages are missed. We need to use Gap
fill mechanism o retieve the missed sequences. The way I
understood this is this situation occurs only if the
administrative messages (like HeartBeat) comes in between the
business messages. So, while constructing the Fix message we can
give some special number ( for eg: “-1”) as the sequence number
for the administrative messages AND at the recieving end we check
for the type of the message. If it is administrative message then
discard that messsage and expect the same number as it was before.Please correct me, if I am wrongly interpreting the mechanism.
Thanks & Regards Bhasker Reddy A.
I want to avoid the gap fill portion of it but not sequence reset.
Here sequence reset happens on client side only if the sequence number on client (In client’s database)is greater than server’s expected sequence number.
Anyways, This was just a thought. I am not implementing this
Unless the logon messages are assigned “real” seq. numbers, so that you
can figure out that you’re out of synch right at logon, or there will be
a delay as suggested until the next “real” seq number/business message
shows up, but then you will still need to gap fill these logon messages.Also, you wold not be able to connect to any other FIX provider or have
another provider connect to you.Do you want to avoid using the seq. reset message? or just the Gap Fill
portion of it? Seq. reset is quite essential.Perhaps I am missing something but it seems to me that this system
would have a few undesirable side effects:If all admin messages are sent with sequence number -1 then lets
say that a client sends three orders and receives 3 opens. Then
the client disconnects, while disconnected the three orders are
filled. When the client reconnects he will have no idea that any
messages were missed because the Logon messages (test request and
so forth) all have the sequence number of -1 in this case, and
thus do not cause a resend request. The client will only learn of
the fills after he sends his first “business message”.Since sequence number is persistent on both client & server side. On
client side if it is a administrative message then we don’t update
that sequence number in database( persistence ). Even if client
disconnects, when reconnects next time it synchronises with the
server side sequence number.So do administrative messages have “real” sequence numbers when they
are first sent and then -1 in response to a resend request?If you do not allow administrative messages (logons, heartbeats,
testrequests etc …) to increment the sequence then after a
disconnect you will have to have special logic on both sides to
synchronize the server and client.Tell me where I am wrong- --The day starts fix engines find each other–
…logon heartbeat etc all 34=-1 --First business message of the day–client -sends order (35=D) 34=1 server -ack’s order (35=8) 39=0 34=1
–client disconnects with no logout-- server -sends order filled(39=2)
34=2 --client comes back-- …logon hearbeat etc… 34=-1The server may not know that the client did not receive the fill, the
sockets could even have been connected but the client code was in the
middle of a core dump or whatever. When the client comes back and logs
on and receives the logon ack he cannot use the value of 34 (sequence
number) to establish the fact that he did indeed miss messages (since
all admin messages have the sequence number of -1). So what you would
now have to do is either wait for another business message to be sent
by either party which would trigger a resend request or build in other
handshake logic to make sure that no messages were dropped in the
interum.So what mechanism do you use to synchronize the server and the client?
And why go to all this trouble just to “avoid” a sequence reset
message?I don’t think that many traders would be happy hearing you say
“don’t worry you will find out what happened to those orders just
as soon as you send another!”.Hello all, If the incoming sequence number is greater than
expected,it indicates that messages are missed. We need to use
Gap fill mechanism o retieve the missed sequences. The way I
understood this is this situation occurs only if the
administrative messages (like HeartBeat) comes in between the
business messages. So, while constructing the Fix message we can
give some special number ( for eg: “-1”) as the sequence number
for the administrative messages AND at the recieving end we
check for the type of the message. If it is administrative
message then discard that messsage and expect the same number as
it was before.Please correct me, if I am wrongly interpreting the mechanism.
Thanks & Regards Bhasker Reddy A.
Perhaps I am missing something but it seems to me that this system
would have a few undesirable side effects:If all admin messages are sent with sequence number -1 then lets say
that a client sends three orders and receives 3 opens. Then the
client disconnects, while disconnected the three orders are filled.
When the client reconnects he will have no idea that any messages
were missed because the Logon messages (test request and so forth)
all have the sequence number of -1 in this case, and thus do not
cause a resend request. The client will only learn of the fills
after he sends his first “business message”.Since sequence number is persistent on both client & server side. On
client side if it is a administrative message then we don’t update
that sequence number in database( persistence ). Even if client
disconnects, when reconnects next time it synchronises with the server
side sequence number.So do administrative messages have “real” sequence numbers when they are
first sent and then -1 in response to a resend request?If you do not allow administrative messages (logons, heartbeats,
testrequests etc …) to increment the sequence then after a disconnect
you will have to have special logic on both sides to synchronize the
server and client.Tell me where I am wrong- --The day starts fix engines find each other–
…logon heartbeat etc all 34=-1 --First business message of the day–client -sends order (35=D) 34=1 server -ack’s order (35=8) 39=0 34=1
–client disconnects with no logout-- server -sends order filled(39=2)
34=2 --client comes back-- …logon hearbeat etc… 34=-1The server may not know that the client did not receive the fill, the
sockets could even have been connected but the client code was in the
middle of a core dump or whatever. When the client comes back and
logs on and receives the logon ack he cannot use the value of 34
(sequence number) to establish the fact that he did indeed miss
messages (since all admin messages have the sequence number of -1).
So what you would now have to do is either wait for another business
message to be sent by either party which would trigger a resend
request or build in other handshake logic to make sure that no
messages were dropped in the interum.
Server discards all the messages until it gets it’s expected sequence number message.
Client side sending sequence number should be same as server’s expected sequence number.
If both are not same then synchronise
So what mechanism do you use to synchronize the server and the client?
And why go to all this trouble just to “avoid” a sequence reset message?
I want to avoid the gap fill portion of it but not sequence reset.
Here sequence reset happens on client side only if the sequence number on
client (In client’s database)is greater than server’s expected sequence number (Available in the server’s databse).
Here is a synchonisation mechanism:
if ( sequence number of client is greater than sequence number of server )
then
reset client’s sequence number to server’s
if ( sequence number of client is less than sequence number of server )
serious error. Through the error and break the connection
if ( sequence number of client is equalto sequence number of server )
process the message
Thanks,
Bhasker Reddy
I don’t think that many traders would be happy hearing you say
“don’t worry you will find out what happened to those orders just as
soon as you send another!”.Hello all, If the incoming sequence number is greater than
expected,it indicates that messages are missed. We need to use Gap
fill mechanism o retieve the missed sequences. The way I
understood this is this situation occurs only if the
administrative messages (like HeartBeat) comes in between the
business messages. So, while constructing the Fix message we can
give some special number ( for eg: “-1”) as the sequence number
for the administrative messages AND at the recieving end we check
for the type of the message. If it is administrative message then
discard that messsage and expect the same number as it was before.Please correct me, if I am wrongly interpreting the mechanism.
Thanks & Regards Bhasker Reddy A.