Question about Fragmentation of FIX Allocation message in FIX 4.4

, ,
1

Imported from previous forum

2

I have a question about Fragmentation of Allocation message in FIX 4.4
protocol specification. My question is why the number sequence (e.g. NoSeq)is not included in supporting the allocation fragments. The number sequence can be used to indicate the sequencing of allocation fragments and can help the application to safeguard the integrity of the complete FIX message.

For example, when fragmentation is applied, allocation instruction with 200 allocations can be fragmented as follows:

1st fragment:
TotNoAlloc=200
NoAllocs=80
NoSeq=1

2nd fragment:
TotNoAlloc=200
NoAllocs=80
NoSeq=2

3rd fragment:
TotNoAlloc=200
NoAllocs=40
NoSeq=3

If any one of fragments is missing or duplicate, the application side can take appropriate measurement and actions accordingly.

Appreciate if someone can reply back or point me to the right direction.

3

[ original email was from Dean Kauffman - dean.kauffman@tradeweb.com ]
Richard,

Your suggestion is a good one and it’s one we tried when we modelled the flow of fragmented allocation messages prior to introducing it in FIX 4.4. We found that the potential problems with fragmentation did NOT include messages getting out of order, duplicated, or dropping out - which is what sequencing resolves. That is taken care of at the session level. The big problem is ensuring that the sender’s and receiver’s expectations are in sync, and the pair of fields TotNoAllocs and LastFragment achieve that.

Dean

I have a question about Fragmentation of Allocation message in FIX 4.4
protocol specification. My question is why the number sequence (e.g.
NoSeq)is not included in supporting the allocation fragments. The number
sequence can be used to indicate the sequencing of allocation fragments
and can help the application to safeguard the integrity of the complete
FIX message.

For example, when fragmentation is applied, allocation instruction with
200 allocations can be fragmented as follows:

1st fragment: TotNoAlloc=200 NoAllocs=80 NoSeq=1

2nd fragment: TotNoAlloc=200 NoAllocs=80 NoSeq=2

3rd fragment: TotNoAlloc=200 NoAllocs=40 NoSeq=3

If any one of fragments is missing or duplicate, the application side
can take appropriate measurement and actions accordingly.

Appreciate if someone can reply back or point me to the right direction.

4

Dean,

Thanks for your reply.

Looks like the assumption was made that sender and receiver abide the 4 key rules that are recommended in the FIX 4.4 spec. But in reality, application that is behind the FIX engine may be serving multiple clients and receiving different FIX messages all in one pipe. The FIX messages may be inter-mingled including the fragments. In this case, application would have problem in assembling the fragments accordingly without the proper tags included in the FIX protocol especially the sequencing is required by the receiving end.

Also, as part of message assembling and recovery process, application has to take into consideration that messages may be disrrupted, duplicated, or missing. Without proper tag, number sequence, in this case, it is hard to assemble the fragments back to its original or intended message.

Thanks,
Richard

Richard,

Your suggestion is a good one and it’s one we tried when we modelled the
flow of fragmented allocation messages prior to introducing it in FIX
4.4. We found that the potential problems with fragmentation did NOT
include messages getting out of order, duplicated, or dropping out -
which is what sequencing resolves. That is taken care of at the session
level. The big problem is ensuring that the sender’s and receiver’s
expectations are in sync, and the pair of fields TotNoAllocs and
LastFragment achieve that.

Dean

I have a question about Fragmentation of Allocation message in FIX 4.4
protocol specification. My question is why the number sequence (e.g.
NoSeq)is not included in supporting the allocation fragments. The
number sequence can be used to indicate the sequencing of allocation
fragments and can help the application to safeguard the integrity of
the complete FIX message.

For example, when fragmentation is applied, allocation instruction
with 200 allocations can be fragmented as follows:

1st fragment: TotNoAlloc=200 NoAllocs=80 NoSeq=1

2nd fragment: TotNoAlloc=200 NoAllocs=80 NoSeq=2

3rd fragment: TotNoAlloc=200 NoAllocs=40 NoSeq=3

If any one of fragments is missing or duplicate, the application side
can take appropriate measurement and actions accordingly.

Appreciate if someone can reply back or point me to the right
direction.

5

[ original email was from Dean Kauffman - dean.kauffman@tradeweb.com ]
Richard,

Yes, the receiving app must be prepared to handle incoming messages comingled from multiple clients, thus it needs to maintain a persistent state table relative to each allocation that’s in reassembly. That would be true regardless of what controls you’ve introduced into the protocol. As for message reordering, loss, or duplication, the FIX session protocol protects the message stream from these things. Be sure you don’t implement a process architecture behind the engine that reintroduces them.

Dean

Dean,

Thanks for your reply.

Looks like the assumption was made that sender and receiver abide the 4
key rules that are recommended in the FIX 4.4 spec. But in reality,
application that is behind the FIX engine may be serving multiple
clients and receiving different FIX messages all in one pipe. The FIX
messages may be inter-mingled including the fragments. In this case,
application would have problem in assembling the fragments accordingly
without the proper tags included in the FIX protocol especially the
sequencing is required by the receiving end.

Also, as part of message assembling and recovery process, application
has to take into consideration that messages may be disrrupted,
duplicated, or missing. Without proper tag, number sequence, in this
case, it is hard to assemble the fragments back to its original or
intended message.

Thanks, Richard

Richard,

Your suggestion is a good one and it’s one we tried when we
modelled the flow of fragmented allocation messages prior to
introducing it in FIX
4.4. We found that the potential problems with fragmentation did NOT
include messages getting out of order, duplicated, or dropping
out - which is what sequencing resolves. That is taken care of at
the session level. The big problem is ensuring that the sender’s
and receiver’s expectations are in sync, and the pair of fields
TotNoAllocs and LastFragment achieve that.

Dean

I have a question about Fragmentation of Allocation message in FIX
4.4 protocol specification. My question is why the number sequence
(e.g. NoSeq)is not included in supporting the allocation fragments.
The number sequence can be used to indicate the sequencing of
allocation fragments and can help the application to safeguard the
integrity of the complete FIX message.

For example, when fragmentation is applied, allocation instruction
with 200 allocations can be fragmented as follows:

1st fragment: TotNoAlloc=200 NoAllocs=80 NoSeq=1

2nd fragment: TotNoAlloc=200 NoAllocs=80 NoSeq=2

3rd fragment: TotNoAlloc=200 NoAllocs=40 NoSeq=3

If any one of fragments is missing or duplicate, the application
side can take appropriate measurement and actions accordingly.

Appreciate if someone can reply back or point me to the right
direction.