Imported from previous forum
Hi,
It appears that for clients using QuickFIX/J, if they try to connect to a Cameron Engine with 141=Y in the Logon field, then Cameron replies without the 141 tag. This causes the engines to go in a strange state - QFG sends a second Logon as it does not recognize the Acknowledgment, and then sends messages with SeqNum=1 again.
Anyone else experienced this issue too ?
Thanks. Nataraj.
[ original email was from Kalpak Gadre - kalpak_gadre@omniscient.co.in ]
>
Hi,
It appears that for clients using QuickFIX/J, if they try to connect to
a Cameron Engine with 141=Y in the Logon field, then Cameron replies
without the 141 tag. This causes the engines to go in a strange state -
QFG sends a second Logon as it does not recognize the Acknowledgment,
and then sends messages with SeqNum=1 again.Anyone else experienced this issue too ?
Thanks. Nataraj.
Do you really need the ResetSeqNumFlag(141) set? I would recommend trying to connect without it. Use the session settings “quickfix.SessionSettings” and set “resetOnLogon” to “N”
It is not a necessity, but the Initiator might need to connect in this manner on occasion. If that ever happens, then we experience issues, which cannot be afforded. … .
The Specifications do not state what fields to send as Acknowledgment if Logon has 141=Y.
QuickFIX assumes that the Acceptor will reply with this flag but Cameron sends Logon Acknowledgment without the flag. QuickFIX is unable to handle this reponse and –
-
QuickFIX resends another Logon with SeqNum 2 although it is already connected - Engines go in a strange state, some requests fail; and/or
-
QuickFIX subsequently replies with TestRequest/Heartbeat also with SeqNum=1 (Errors with MsgSeqNum lower than Expected).
…
Thanks.
Hi,
It appears that for clients using QuickFIX/J, if they try to connect
to a Cameron Engine with 141=Y in the Logon field, then Cameron
replies without the 141 tag. This causes the engines to go in a
strange state - QFG sends a second Logon as it does not recognize the
Acknowledgment, and then sends messages with SeqNum=1 again.Anyone else experienced this issue too ?
Thanks. Nataraj.
Do you really need the ResetSeqNumFlag(141) set? I would recommend
trying to connect without it. Use the session settings
“quickfix.SessionSettings” and set “resetOnLogon” to “N”
The Specifications do not state what fields to send as Acknowledgment > if Logon has 141=Y.
Specification states what to do when such Logon is sent when during the session. Imho similar approach should be used when such Logon is sent as initial Logon of the day:
======= From fix-44_VOL-2_w_Errata_20030618.doc =======================
…
the initiator should send a Logon with ResetSeqNumFlag set to Y and with MsgSeqNum of 1. The acceptor should respond with a Logon with ResetSeqNumFlag set to Y and with MsgSeqNum of 1. At this point new messages from either side should continue with MsgSeqNum of 2.
…
[ original email was from Kalpak Gadre - kalpak_gadre@omniscient.co.in ]
I understand you that it maybe necessary at times to set the ResetSeqNoFlag.
As of FIX 4.2, the spec reads,
“One side will initiate the process by sending a TestRequest and wait for a Heartbeat in response to ensure of no sequence number gaps. Once the Heartbeat has been received, the initiator should send a Logon with
ResetSeqNumFlag set to Y and with MsgSeqNum of 1. The acceptor should respond with a Logon with ResetSeqNumFlag set to Y and with MsgSeqNum of 1.”
So it seems that expecting 141=Y is not abnormal. It needs to be set conditionally. If You could probably set the ResetSeqNoFlag at last stage through Cameron engine somehow?
It is not a necessity, but the Initiator might need to connect in this
manner on occasion. If that ever happens, then we experience issues,
which cannot be afforded. … .The Specifications do not state what fields to send as Acknowledgment if
Logon has 141=Y.QuickFIX assumes that the Acceptor will reply with this flag but Cameron
sends Logon Acknowledgment without the flag. QuickFIX is unable to
handle this reponse and –
QuickFIX resends another Logon with SeqNum 2 although it is already
connected - Engines go in a strange state, some requests fail; and/orQuickFIX subsequently replies with TestRequest/Heartbeat also with
SeqNum=1 (Errors with MsgSeqNum lower than Expected).…
Thanks.
Cameron does not seem to respond with 141=Y if it receives Initial Logon with 141=Y. This is the main issue as explained. Have you/anyone experienced this ?
Note that the client does not always logon with 141=Y. Without this flag the logon works fine. So, to force a reset to Seq 1 at logon or logoff cannot be implemented … .
I understand you that it maybe necessary at times to set the
ResetSeqNoFlag.As of FIX 4.2, the spec reads,
“One side will initiate the process by sending a TestRequest and wait
for a Heartbeat in response to ensure of no sequence number gaps. Once
the Heartbeat has been received, the initiator should send a Logon
with ResetSeqNumFlag set to Y and with MsgSeqNum of 1. The acceptor
should respond with a Logon with ResetSeqNumFlag set to Y and with
MsgSeqNum of 1.”So it seems that expecting 141=Y is not abnormal. It needs to be set
conditionally. If You could probably set the ResetSeqNoFlag at last
stage through Cameron engine somehow?It is not a necessity, but the Initiator might need to connect in this
manner on occasion. If that ever happens, then we experience issues,
which cannot be afforded. … .The Specifications do not state what fields to send as Acknowledgment
if Logon has 141=Y.QuickFIX assumes that the Acceptor will reply with this flag but
Cameron sends Logon Acknowledgment without the flag. QuickFIX is
unable to handle this reponse and –
QuickFIX resends another Logon with SeqNum 2 although it is already
connected - Engines go in a strange state, some requests fail;
and/orQuickFIX subsequently replies with TestRequest/Heartbeat also with
SeqNum=1 (Errors with MsgSeqNum lower than Expected).…
Thanks.
[ original email was from Daniel Plasto - daniel.plasto@orcsoftware.com ]
Hi Nataraj,
Please contact your local CameronFIX support office about this.
If you’re not sure who to contact send me an email: daniel dot plasto at orcsoftware dot com
I’ll direct you to someone who can help you.
Regards,
Daniel Plasto
Orc Software
Cameron does not seem to respond with 141=Y if it receives Initial Logon
with 141=Y. This is the main issue as explained. Have you/anyone
experienced this ?Note that the client does not always logon with 141=Y. Without this
flag the logon works fine. So, to force a reset to Seq 1 at logon or
logoff cannot be implemented … .I understand you that it maybe necessary at times to set the
ResetSeqNoFlag.As of FIX 4.2, the spec reads,
“One side will initiate the process by sending a TestRequest and wait
for a Heartbeat in response to ensure of no sequence number gaps. Once
the Heartbeat has been received, the initiator should send a Logon
with ResetSeqNumFlag set to Y and with MsgSeqNum of 1. The acceptor
should respond with a Logon with ResetSeqNumFlag set to Y and with
MsgSeqNum of 1.”So it seems that expecting 141=Y is not abnormal. It needs to be set
conditionally. If You could probably set the ResetSeqNoFlag at last
stage through Cameron engine somehow?It is not a necessity, but the Initiator might need to connect in
this manner on occasion. If that ever happens, then we experience
issues, which cannot be afforded. … .The Specifications do not state what fields to send as
Acknowledgment if Logon has 141=Y.QuickFIX assumes that the Acceptor will reply with this flag but
Cameron sends Logon Acknowledgment without the flag. QuickFIX is
unable to handle this reponse and –
QuickFIX resends another Logon with SeqNum 2 although it is
already connected - Engines go in a strange state, some requests
fail; and/orQuickFIX subsequently replies with TestRequest/Heartbeat also with
SeqNum=1 (Errors with MsgSeqNum lower than Expected).…
Thanks.
[ original email was from Ryan Pierce (FPL Technical Director) - ryan.pierce@fixprotocol.org ]
> It appears that for clients using QuickFIX/J, if they try to connect to
a Cameron Engine with 141=Y in the Logon field, then Cameron replies
without the 141 tag. This causes the engines to go in a strange state -
QFG sends a second Logon as it does not recognize the Acknowledgment,
and then sends messages with SeqNum=1 again.
141=Y was added to the specification so 24 hour sessions could be supported and sequence numbers re-established at 1 without the need to disconnect and reconnect. Using this flag when establishing a new session seems highly unusual. Most counterparties define a reset time. If you disconnect before the reset time and connect afterwards, you are using sequence number 1, and 141=Y is superfluous. If you are reconnecting intraday, in most cases 141=Y would be disastrous, as it would prevent recovery of missed messages while you were down.
It appears that for clients using QuickFIX/J, if they try to connect
to a Cameron Engine with 141=Y in the Logon field, then Cameron
replies without the 141 tag. This causes the engines to go in a
strange state - QFG sends a second Logon as it does not recognize the
Acknowledgment, and then sends messages with SeqNum=1 again.141=Y was added to the specification so 24 hour sessions could be
supported and sequence numbers re-established at 1 without the need to
disconnect and reconnect. Using this flag when establishing a new
session seems highly unusual. Most counterparties define a reset time.
If you disconnect before the reset time and connect afterwards, you are
using sequence number 1, and 141=Y is superfluous. If you are
reconnecting intraday, in most cases 141=Y would be disastrous, as it
would prevent recovery of missed messages while you were down.
Agreed. What I am getting to is that, there are some Initiators who inisist on re-establishing a new connection with SeqNum=1 and not the last known SeqNum. However on the Acceptor side, the SeqNum is still the last SeqNum at disconnection. Acceptor says “MesgSeqNum too low, expecting xxx but received 1” and disconnects the Initiator. The Initiator then tries to reconnect again, but this time with 141=Y to sync Seqs on both sides. This is when the problem occurs, and QFJ/Cameron communication runs into trouble and all these errors pop up.
Please note that this issue is mainly for counterparties who opt out of a “scheduled (bilateral) reset at a specific time of the day” and prefer sending a client-side request for resetting sequence numbers instead. Thanks in advance for your suggestions !
[ original email was from Ryan Pierce (FPL Technical Director) - ryan.pierce@fixprotocol.org ]
> Agreed. What I am getting to is that, there are some Initiators who
inisist on re-establishing a new connection with SeqNum=1 and not the
last known SeqNum. However on the Acceptor side, the SeqNum is still the
last SeqNum at disconnection. Acceptor says “MesgSeqNum too low,
expecting xxx but received 1” and disconnects the Initiator. The
Initiator then tries to reconnect again, but this time with 141=Y to
sync Seqs on both sides. This is when the problem occurs, and
QFJ/Cameron communication runs into trouble and all these errors pop up.
I’m not sure why an Initiator would do that. Doing so loses all possibility of session-level recovery of missed messages. This makes sense to me only with messages that are “disposable” like a FIX session solely used for IOIs, Market Data, etc. but it could be a disaster for order flow.
What does Cameron do when it receives a 141=Y Logon? You’ve said it doesn’t respond with 141=Y Logon. But does it reset its own sequence number to 1 and appear to accept your sequence number 1 Logon?
If not, I’d say that is the Acceptor’s right. I don’t believe the spec forces someone to implement 24-hour session semantics, or use 141=Y as a way to avoid persistence. If I were a sell-side broker or an exchange, and someone sent me 141=Y on an order flow connection, I probably wouldn’t want to accept it. Doing so means I could have queued fills for the client that the client would never see, and I’d have a whole mess on my hands about who is liable for them.
If so, it sounds like QFJ is having trouble, too. Yes, the spec does say 141=Y is the proper response. But I’m a big fan of making FIX engines bomb-proof and not assuming others will be 100% technically correct. If I send a Logon with seq no 1 and 141=Y, and my counterparty sends me a Logon with seq no 1, even if 141=Y is missing which might not be technically correct, I should still assume that this was a success.
I’m not sure why an Initiator would do that. Doing so loses all
possibility of session-level recovery of missed messages. This makes
sense to me only with messages that are “disposable” like a FIX session
solely used for IOIs, Market Data, etc. but it could be a disaster for
order flow.
I understand your concern and it is definitely something the customer must avoid for Order Sessions. I’d add that, as you have noted, this is done mainly for the Market Data session only, not the separate Order session.
What does Cameron do when it receives a 141=Y Logon? You’ve said it
doesn’t respond with 141=Y Logon. But does it reset its own sequence
number to 1 and appear to accept your sequence number 1 Logon?
There are 2 scenarios –
Seq1 A -----> B (Logon with 141=Y)
Seq1 B -----> A (Logon Acknowledgment without 141=Y)
Seq2 A -----> B (Logon message again with 141=Y)
B issues error mesg saying that it has received a second logon messages, whilst already connected. The session, at this point, is still connected.
Seq3 A -------> B (MarketDataRequest)
And B, the acceptor, throws an error saying A is “not logged into FIX session” - although when when A, the initiator, subsequently sends a TestRequest/Heartbeat over the same session, B does reply back - indicating that the session IS actually connected. Nowhere in the FIX engine logs does it say that there was any Logoff. The message only appears when the client is trying to send a Market Data Request, not for other message types such as TestRequest or Heartbeat.
Seq1 A -----> B (Logon with 141=Y)
Seq1 B -----> A (Logon Acknowledgment without 141=Y)
Seq1 A -----> B (TestRequest)
B now throws an error as it has received the second message with the same SeqNum 1 as Logon and disconnects.
This most likely happens as A has failed to recognize that B had actually acknowledged the Logon message … it sends a new message such as a TestRequest with the same Seq as the Logon, i.e. 1.
If so, it sounds like QFJ is having trouble, too. Yes, the spec does say
141=Y is the proper response. But I’m a big fan of making FIX engines
bomb-proof and not assuming others will be 100% technically correct. If
I send a Logon with seq no 1 and 141=Y, and my counterparty sends me a
Logon with seq no 1, even if 141=Y is missing which might not be
technically correct, I should still assume that this was a success.
[ original email was from Ryan Pierce (FPL Technical Director) - ryan.pierce@fixprotocol.org ]
While I believe in following the spec closely, I’m also a big fan of defensive coding in FIX engines, much like defensive driving. (This is very important for me, living in Chicago where turn signals are considered novelty items. 
In my mind, an engine that cannot tolerate slightly non-standard behavior is overly fragile.
In your examples:
Seq1 A -----> B (Logon with 141=Y)
Seq1 B -----> A (Logon Acknowledgment without 141=Y)
Seq2 A -----> B (Logon message again with 141=Y)
Yes, B should have sent 141=Y. But A has just received, in response, a Logon with Seq1. It should be clear, even though 141=Y isn’t set, that a session was established and the sequence numbers were reset. There is no reason A should send a second Logon. And things go down hill from there.
Seq1 A -----> B (Logon with 141=Y)
Seq1 B -----> A (Logon Acknowledgment without 141=Y)
Seq1 A -----> B (TestRequest) B now throws an error as it
has received the second message with the same SeqNum 1 as Logon and
disconnects.
Again, A has received a Logon with Seq1, admittedly without 141=Y. It should be obvious to A that a successful Logon has occurred at this point. But A responds by reusing Seq1! This is blatantly non-standard, and really isn’t recoverable; B has no choice but to terminate the session.
Yes, you can make an argument that B isn’t following the spec by failing to send 141=Y, and B sending 141=Y would fix the problem. But I don’t think this error necessarily should be fatal. As long as B’s sequence number on its Logon is 1, then it is evident that sequence numbers were successfully reset. I think it would only be a benefit for A to be a little more flexible.
Now if B’s Logon is not 1, then that indicates sequence numbers were not reset, and this is an entirely different issue.
Isn’t QuickFIX/J (which I assume is A in this example) open source? How hard would it be to make a small code change to infer 141=Y on B’s Logon if both A’s Logon had 141=Y and B’s Logon seq number is 1? Wouldn’t that fix the problem?
[ original email was from Steve Bate - steve@technoetic.com ]
> If so, it sounds like QFJ is having trouble, too. Yes, the spec does say
141=Y is the proper response. But I’m a big fan of making FIX engines
bomb-proof and not assuming others will be 100% technically correct. If
I send a Logon with seq no 1 and 141=Y, and my counterparty sends me a
Logon with seq no 1, even if 141=Y is missing which might not be
technically correct, I should still assume that this was a success.
The trouble on the QFJ side is that it’s not dropping the connection
when the sequence reset is not properly implemented by the
counterparty FIX engine.
According to the 4.4 specification, “the connection should be shutdown
and manual intervention taken if this [sequence reset] process is initiated but not followed properly”. I haven’t checked all the
versions of the specification but I assume this is the same or
unspecified in other FIX versions. To be 100% technically correct
per the specification this is the behavior that should be implemented
by default in QFJ. Of course, a configuration option also could be
added to support the assumption you described so that QFJ can
compensate for nonstandard counterparty FIX engines.
[ original email was from Kalpak Gadre - kalpak_gadre@omniscient.co.in ]
You could also detect this special case by writing some code and plugging it into QFJ and disconnect the session. But I agree, it rather makes sense to drop the connection instead of sending subsequent logons at the QFJ Engine level.
Natraj, maybe you could log this as a bug on QFJ?
According to the 4.4 specification, “the connection should be shutdown
and manual intervention taken if this [sequence reset] process is
initiated but not followed properly”. I haven’t checked all the versions
of the specification but I assume this is the same or unspecified in
other FIX versions. To be 100% technically correct per the specification
this is the behavior that should be implemented by default in QFJ. Of
course, a configuration option also could be added to support the
assumption you described so that QFJ can compensate for nonstandard
counterparty FIX engines.
[ original email was from Steve Bate - steve@technoetic.com ]
> You could also detect this special case by writing some code and
plugging it into QFJ and disconnect the session. But I agree, it rather
makes sense to drop the connection instead of sending subsequent logons
at the QFJ Engine level.Natraj, maybe you could log this as a bug on QFJ?
Nataraj had already submitted a QFJ bug report before this thread
was started.
http://www.quickfixj.org/jira/browse/QFJ-380
Hopefully, he will do some same for Cameron FIX. In the meantime,
a change to next version of QFJ will not completely solve his problem.
It seems that developers often don’t upgrade their FIX engine
versions quickly so old QFJ versions will still be out there. It’s
also possible that other FIX engines either implement the standard
behavior (dropping the connection) or some other nonstandard
undesirable behavior when Cameron FIX doesn’t respond correctly to
the sequence reset. I second the advice someone gave to add a
Cameron FIX message transformer to add the 141=Y to the outgoing
logon message when it’s needed.