Fix Message Recovery after session reset by acceptor

, ,
1

Imported from previous forum

2

Hi All,
We are implementing the Initiator side using Quickfix\J version 1.3.0.The FIX version is FIX.4.2.
We are getting the daily fix trades/messages from the acceptor side.The acceptor side resets the session every day at 17:30 CST.

Here are a few the setting from my cfg properties file.

[session]
BeginString=FIX.4.2
ConnectionType=initiator
SenderCompID=XXXX
TargetCompID=YYYY
StartTime=00:00:00
EndTime=23:59:59
#Reset the sequence number
ResetOnDisconnect=Y

The fix cleint application (initiator) at our end saves the messages into a table in the database as soon as a message is available.

I have a few questions on how to implement the following logic:

  1. The acceptor sends in the messages.Lets say that the application at our end which listens and saves the messages into the DB breaks down, but the the acceptor keeps on sending the message.
    To recover these messages we restart the fix client and at at logon ,we send a RESEND request from startseqNum 1 to endSeqNum 0 and get in all the messages for the day.Check for duplicates using the unique key ExecID and save into the database the missing messages.

Since the session is reset everyday at 17:30 CST on the acceptor side.What if we want to process the trades for the day after the session is reset at the acceptor end ? How do we get the missing trades for the day after 17:30 CST ?

Appreciate all the help.

Thanks
Bharath

3

[ original email was from Vern Kennedy - vkennedy@uniserve.com ]
Bharath

Have you given any thought to just sending Resend requests for only the messages you missed while down, rather than getting every single message for the day each time you log on? Since it sounds like your side is storing all the messages in the database, it sounds like it would be more efficient for you to just ask for the messages that you missed while your side was down.

Also, assuming that you are connected right up until the reset at 17:30, under what circumstance would you need to get an entire replay of all the messages from the day? Wouldn’t these already be stored in your database?

Hi All, We are implementing the Initiator side using Quickfix\J version
1.3.0.The FIX version is FIX.4.2. We are getting the daily fix
trades/messages from the acceptor side.The acceptor side resets the
session every day at 17:30 CST.

Here are a few the setting from my cfg properties file.

[session] BeginString=FIX.4.2 ConnectionType=initiator SenderCompID=XXXX
TargetCompID=YYYY StartTime=00:00:00 EndTime=23:59:59 #Reset the
sequence number ResetOnDisconnect=Y

The fix cleint application (initiator) at our end saves the messages
into a table in the database as soon as a message is available.

I have a few questions on how to implement the following logic:

  1. The acceptor sends in the messages.Lets say that the application at
    our end which listens and saves the messages into the DB breaks down,
    but the the acceptor keeps on sending the message. To recover these
    messages we restart the fix client and at at logon ,we send a RESEND
    request from startseqNum 1 to endSeqNum 0 and get in all the messages
    for the day.Check for duplicates using the unique key ExecID and save
    into the database the missing messages.

Since the session is reset everyday at 17:30 CST on the acceptor
side.What if we want to process the trades for the day after the session
is reset at the acceptor end ? How do we get the missing trades for the
day after 17:30 CST ?

Appreciate all the help.

Thanks Bharath

4

Bharath

Have you given any thought to just sending Resend requests for only the
messages you missed while down, rather than getting every single message
for the day each time you log on? Since it sounds like your side is
storing all the messages in the database, it sounds like it would be
more efficient for you to just ask for the messages that you missed
while your side was down.

Also, assuming that you are connected right up until the reset at
17:30, under what circumstance would you need to get an entire replay
of all the messages from the day? Wouldn’t these already be stored in
your database?

Hi All, We are implementing the Initiator side using Quickfix\J
version
1.3…The FIX version is FIX.4.2. We are getting the daily fix
trades/messages from the acceptor side.The acceptor side resets
the session every day at 17:30 CST.

Here are a few the setting from my cfg properties file.

[session] BeginString=FIX.4.2 ConnectionType=initiator
SenderCompID=XXXX TargetCompID=YYYY StartTime=00:00:00
EndTime=23:59:59 #Reset the sequence number ResetOnDisconnect=Y

The fix cleint application (initiator) at our end saves the messages
into a table in the database as soon as a message is available.

I have a few questions on how to implement the following logic:

  1. The acceptor sends in the messages.Lets say that the application at
    our end which listens and saves the messages into the DB breaks
    down, but the the acceptor keeps on sending the message. To recover
    these messages we restart the fix client and at at logon ,we send a
    RESEND request from startseqNum 1 to endSeqNum 0 and get in all the
    messages for the day.Check for duplicates using the unique key
    ExecID and save into the database the missing messages.

Since the session is reset everyday at 17:30 CST on the acceptor
side.What if we want to process the trades for the day after the
session is reset at the acceptor end ? How do we get the missing
trades for the day after 17:30 CST ?

Appreciate all the help.

Thanks Bharath

Vern,
Thanks for the quick response.
You are right the best way to recover messages is to keep track of the sequence numbers and recover starting form the last sequence number.I have used the same logic in my application.

Coming to the second point-

If I am connected to the session until 17:30CST, then there should not be any missing trades.But,say somebody brought down the fix application from our side and forgot to restart it.Say the whole day has passed by and we haven’t restarted the fix client from our side and the session got reset.Now In this scenario,I will need to replay all the messages for the entire day.
I know this is a humane error, but how do we recover from it ?

I couldn’t find a way to do it currently in fix.

Thanks
Bharath

5

[ original email was from Vern Kennedy - vkennedy@uniserve.com ]
It seems to me that to recover from that kind of thing you would need some kind of special arrangement between yourself and your counterparty. This almost sounds like you are getting into a disaster recovery scenario, so I am sure you would have to have some prearranged plan set with the other side to get a replay sent to you. The replay would probably have to be in a file that you would have to parse and interpret yourself, not part of the regular FIX connection session.

Bharath

Have you given any thought to just sending Resend requests for only
the messages you missed while down, rather than getting every single
message for the day each time you log on? Since it sounds like your
side is storing all the messages in the database, it sounds like it
would be more efficient for you to just ask for the messages that you
missed while your side was down.

Also, assuming that you are connected right up until the reset at
17:30, under what circumstance would you need to get an entire replay
of all the messages from the day? Wouldn’t these already be stored
in your database?

Hi All, We are implementing the Initiator side using Quickfix\J
version
1.3…The FIX version is FIX.4.2. We are getting the daily fix
trades/messages from the acceptor side.The acceptor side resets
the session every day at 17:30 CST.

Here are a few the setting from my cfg properties file.

[session] BeginString=FIX.4.2 ConnectionType=initiator
SenderCompID=XXXX TargetCompID=YYYY StartTime=00:00:00
EndTime=23:59:59 #Reset the sequence number ResetOnDisconnect=Y

The fix cleint application (initiator) at our end saves the messages
into a table in the database as soon as a message is available.

I have a few questions on how to implement the following logic:

  1. The acceptor sends in the messages.Lets say that the application
    at our end which listens and saves the messages into the DB
    breaks down, but the the acceptor keeps on sending the message.
    To recover these messages we restart the fix client and at at
    logon ,we send a RESEND request from startseqNum 1 to endSeqNum 0
    and get in all the messages for the day.Check for duplicates
    using the unique key ExecID and save into the database the
    missing messages.

Since the session is reset everyday at 17:30 CST on the acceptor
side.What if we want to process the trades for the day after the
session is reset at the acceptor end ? How do we get the missing
trades for the day after 17:30 CST ?

Appreciate all the help.

Thanks Bharath

Vern, Thanks for the quick response. You are right the best way to
recover messages is to keep track of the sequence numbers and recover
starting form the last sequence number.I have used the same logic in my
application.

Coming to the second point-

If I am connected to the session until 17:30CST, then there should not
be any missing trades.But,say somebody brought down the fix application
from our side and forgot to restart it.Say the whole day has passed by
and we haven’t restarted the fix client from our side and the session
got reset.Now In this scenario,I will need to replay all the messages
for the entire day. I know this is a humane error, but how do we
recover from it ?

I couldn’t find a way to do it currently in fix.

Thanks Bharath

6

It seems to me that to recover from that kind of thing you would need
some kind of special arrangement between yourself and your counterparty.
This almost sounds like you are getting into a disaster recovery
scenario, so I am sure you would have to have some prearranged plan set
with the other side to get a replay sent to you. The replay would
probably have to be in a file that you would have to parse and interpret
yourself, not part of the regular FIX connection session.

Bharath

Have you given any thought to just sending Resend requests for only
the messages you missed while down, rather than getting every single
message for the day each time you log on? Since it sounds like your
side is storing all the messages in the database, it sounds like it
would be more efficient for you to just ask for the messages that
you missed while your side was down.

Also, assuming that you are connected right up until the reset at
17:30, under what circumstance would you need to get an entire
replay of all the messages from the day? Wouldn’t these already
be stored in your database?

Hi All, We are implementing the Initiator side using Quickfix\J
version
1.3…The FIX version is FIX.4.2. We are getting the daily fix
trades/messages from the acceptor side.The acceptor side
resets the session every day at 17:30 CST.

Here are a few the setting from my cfg properties file.

[session] BeginString=FIX.4.2 ConnectionType=initiator
SenderCompID=XXXX TargetCompID=YYYY StartTime=00:00:00
EndTime=23:59:59 #Reset the sequence number ResetOnDisconnect=Y

The fix cleint application (initiator) at our end saves the
messages into a table in the database as soon as a message is
available.

I have a few questions on how to implement the following logic:

  1. The acceptor sends in the messages.Lets say that the
    application at our end which listens and saves the messages
    into the DB breaks down, but the the acceptor keeps on sending
    the message. To recover these messages we restart the fix
    client and at at logon ,we send a RESEND request from
    startseqNum 1 to endSeqNum 0 and get in all the messages for
    the day.Check for duplicates using the unique key ExecID and
    save into the database the missing messages.

Since the session is reset everyday at 17:30 CST on the acceptor
side.What if we want to process the trades for the day after the
session is reset at the acceptor end ? How do we get the missing
trades for the day after 17:30 CST ?

Appreciate all the help.

Thanks Bharath

Vern, Thanks for the quick response. You are right the best way to
recover messages is to keep track of the sequence numbers and recover
starting form the last sequence number.I have used the same logic in
my application.

Coming to the second point-

If I am connected to the session until 17:30CST, then there should not
be any missing trades.But,say somebody brought down the fix
application from our side and forgot to restart it.Say the whole day
has passed by and we haven’t restarted the fix client from our side
and the session got reset.Now In this scenario,I will need to replay
all the messages for the entire day. I know this is a humane error,
but how do we recover from it ?

I couldn’t find a way to do it currently in fix.

Thanks Bharath

Thanks again Vern.Just wanted to check if FIX offered a way to recover from it.I will talk to my counterparty about it.

Thanks
Bharath