Imported from previous forum
[ original email was from Urs Flueck - urs.flueck@sungard.com ]
We are connecting many applications via FIX Engines to our FIX Hub that forwards the message to SWIFTNet. We are using FIX 4.2. Now we have the following problem:
- SWIFT is sending a FIX Message and the Sequence number (Tag 34) is 512.
- We store the message in our FIX Hub and check for which application it is.
- We now forward the message to the application using another FIX Session. During this process the Message gets a different Sequence number (Tag 34 is now 543).
- The application rejects the message, because it doesn’t know how to handle this specific message. So this reject message receives a Sequence number 542 and the referenced sequence number Tag 45 is 543 (the previuosly received).
- The Hub now cannot process this message because it doesn’t know the message with sequence number 543.
Is there a way to store somewhere else additional sequencing information. I guess in FIX 4.4 the tags to use would be 630. Is there also such an information to be used for FIX 4.2? Or any other solution?
cheers
Urs
Tag 630 is an interesting idea, although as you note, it’s not available in versions 4.2 and earlier. One approach we’ve seen taken in situations like this is to persist the original incoming sequence number (512 in your example) along with the outgoing message (543 in your example). Then if you get a MsgType=3 reject, it will have RefSeqNum=543, so you know which message to retrieve. When you get back to that message, you can then pull off the original sequence number you stored with it (512), put that in RefSeqNum, and forward it along to the original sender.
When I say persist this along with the message, I don’t mean as a FIX tag=value that would actually get sent along with the message - definitely don’t want to do that. I mean as a custom tag=value property that your FIX engine or application could persist along with the actual FIX message content.
Dave Rhodes
TransactTools
We are connecting many applications via FIX Engines to our FIX Hub that forwards the message to SWIFTNet. We are using FIX 4.2. Now we have the following problem:
- SWIFT is sending a FIX Message and the Sequence number (Tag
34) is 512.- We store the message in our FIX Hub and check for which
application it is.- We now forward the message to the application using another FIX
Session. During this process the Message gets a different Sequence
number (Tag 34 is now 543).- The application rejects the message, because it doesn’t know how to
handle this specific message. So this reject message receives a
Sequence number 542 and the referenced sequence number Tag 45 is 543
(the previuosly received).- The Hub now cannot process this message because it doesn’t know the
message with sequence number 543.Is there a way to store somewhere else additional sequencing
information. I guess in FIX 4.4 the tags to use would be 630. Is there
also such an information to be used for FIX 4.2? Or any other solution?cheers Urs
[ original email was from Urs Flueck - urs.flueck@sungard.com ]
I’m not sure if I understand you correctly. What we have tried is to create a custom tag (5000) and store the reference there. We wanted to do exactly what you proposed comparing the value in there with the other sequence number. Unfortunately when sending the message out to SWIFTNet it gets rejected since it seems that you cannot send FIX Messages with custom tags to SWIFTNet.
cheers
Urs
Tag 630 is an interesting idea, although as you note, it’s not available
in versions 4.2 and earlier. One approach we’ve seen taken in situations
like this is to persist the original incoming sequence number (512 in
your example) along with the outgoing message (543 in your example).
Then if you get a MsgType=3 reject, it will have RefSeqNum=543, so you
know which message to retrieve. When you get back to that message, you
can then pull off the original sequence number you stored with it (512),
put that in RefSeqNum, and forward it along to the original sender.When I say persist this along with the message, I don’t mean as a FIX
tag=value that would actually get sent along with the message -
definitely don’t want to do that. I mean as a custom tag=value property
that your FIX engine or application could persist along with the actual
FIX message content.Dave Rhodes TransactTools
We are connecting many applications via FIX Engines to our FIX Hub
that forwards the message to SWIFTNet. We are using FIX 4.2. Now we
have the following problem:
- SWIFT is sending a FIX Message and the Sequence number (Tag
34) is 512.- We store the message in our FIX Hub and check for which application
it is.- We now forward the message to the application using another FIX
Session. During this process the Message gets a different Sequence
number (Tag 34 is now 543).- The application rejects the message, because it doesn’t know how to
handle this specific message. So this reject message receives a
Sequence number 542 and the referenced sequence number Tag 45 is
543 (the previuosly received).- The Hub now cannot process this message because it doesn’t know the
message with sequence number 543.Is there a way to store somewhere else additional sequencing
information. I guess in FIX 4.4 the tags to use would be 630. Is there
also such an information to be used for FIX 4.2? Or any other
solution?cheers Urs
Your approach is exactly what I was attempting to describe, with one exception - instead of actually sending the custom tag out to SWIFTNet, simply store it as a custom property in your own persistence along with the FIX message, but don’t actually send it. Then when you get back RefSeqNum=543, you would grab the original outgoing 543 messaage from your persistence, and get the value of the custom property you stored along with it.
Does that make more sense? Not all FIX engines can persist additional properties like this by default, but if you have a way to do this, it can be a very useful tool in complex message routing apps.
- Dave
I’m not sure if I understand you correctly. What we have tried is to
create a custom tag (5000) and store the reference there. We wanted to
do exactly what you proposed comparing the value in there with the other
sequence number. Unfortunately when sending the message out to SWIFTNet
it gets rejected since it seems that you cannot send FIX Messages with
custom tags to SWIFTNet.cheers Urs
Tag 630 is an interesting idea, although as you note, it’s not
available in versions 4.2 and earlier. One approach we’ve seen taken
in situations like this is to persist the original incoming sequence
number (512 in your example) along with the outgoing message (543 in
your example). Then if you get a MsgType=3 reject, it will have
RefSeqNum=543, so you know which message to retrieve. When you get
back to that message, you can then pull off the original sequence
number you stored with it (512), put that in RefSeqNum, and forward it
along to the original sender.When I say persist this along with the message, I don’t mean as a FIX
tag=value that would actually get sent along with the message -
definitely don’t want to do that. I mean as a custom tag=value
property that your FIX engine or application could persist along with
the actual FIX message content.Dave Rhodes TransactTools
We are connecting many applications via FIX Engines to our FIX Hub
that forwards the message to SWIFTNet. We are using FIX 4.2. Now we
have the following problem:
- SWIFT is sending a FIX Message and the Sequence number (Tag
34) is 512.- We store the message in our FIX Hub and check for which
application it is.- We now forward the message to the application using another FIX
Session. During this process the Message gets a different
Sequence number (Tag 34 is now 543).- The application rejects the message, because it doesn’t know how
to handle this specific message. So this reject message receives
a Sequence number 542 and the referenced sequence number Tag 45
is 543 (the previuosly received).- The Hub now cannot process this message because it doesn’t know
the message with sequence number 543.Is there a way to store somewhere else additional sequencing
information. I guess in FIX 4.4 the tags to use would be 630. Is
there also such an information to be used for FIX 4.2? Or any other
solution?cheers Urs
[ original email was from John Prewett - jprewett@lavatrading.com ]
This is an interesting subject.
I fundamentally concur with Dave Rhodes on this issue.
The generic problem is that application developers often need to store some private data along with an outgoing message such that both items can be subsequently retrieved due to some eventuality.
A cheap implementation of this problem is to send the private data in the outgoing message. The good news is that the private data is definitely stored with the outgoing message - simply because it is a part of the outgoing message itself. The bad news is that you expose your private data to your customers and also consume unnecessary bandwidth sending them data that they don’t need. In the case of Urs’s problem, the implementation was using custom tags which caused problems at the customer’s end. This could probably be circumvented by abusing unused standard tags, but some customer could still get upset.
A more thorough implementation of this problem is to design your outgoing message store as having two components:
-
The first component is the outgoing message. Unsurprisingly, this will be sent to the customer.
-
The second component is the private data. This is stored along with the message but not sent to the customer.
When you send a message, you supply the outgoing message and an optional secondary component. The FIX engine stores both components in the outgoing message store indexed by message sequence number and then only sends the message itself.
At some future point, you can retrieve both components from your outgoing message store by supplying the message sequence number.
Handling FIX rejects received by a forwarding application then becomes a trivial exercise.
I don’t know if any third party FIX engines support this concept, but they most certainly should. It makes the application developer’s job considerably simpler and more versatile.
I hope this helps.
JohnP
Thanks, John - that definitely makes the generic case much clearer than I did.
Since you asked about third-party engines… I should mention that there is one that does this. ttCONNECT, now part of the TCM solution from TransactTools (www.transacttools.net), offers this functionality and many more features to enable complex multi-protocol message routing applications.
- Dave
This is an interesting subject.
I fundamentally concur with Dave Rhodes on this issue.
The generic problem is that application developers often need to store
some private data along with an outgoing message such that both items
can be subsequently retrieved due to some eventuality.A cheap implementation of this problem is to send the private data in
the outgoing message. The good news is that the private data is
definitely stored with the outgoing message - simply because it is a
part of the outgoing message itself. The bad news is that you expose
your private data to your customers and also consume unnecessary
bandwidth sending them data that they don’t need. In the case of Urs’s
problem, the implementation was using custom tags which caused problems
at the customer’s end. This could probably be circumvented by abusing
unused standard tags, but some customer could still get upset.A more thorough implementation of this problem is to design your
outgoing message store as having two components:
The first component is the outgoing message. Unsurprisingly, this
will be sent to the customer.The second component is the private data. This is stored along with
the message but not sent to the customer.When you send a message, you supply the outgoing message and an optional
secondary component. The FIX engine stores both components in the
outgoing message store indexed by message sequence number and then only
sends the message itself.At some future point, you can retrieve both components from your
outgoing message store by supplying the message sequence number.Handling FIX rejects received by a forwarding application then becomes a
trivial exercise.I don’t know if any third party FIX engines support this concept, but
they most certainly should. It makes the application developer’s job
considerably simpler and more versatile.I hope this helps.
JohnP
[ original email was from Jeremy Sutton - jezza.sutton@patsystems.com ]
That is certain the way I would recommend doing this. Storing some context private data for your message in this case is much better than either custom tags or abusing standard tags with non-standard values. This is a hub application problem rather than a message content problem.