Imported from previous forum
For some time now, we have been writing a Java engine for FIX 4.2. Much of our time has been devoted to parsing repeated groups. We realise that we have little experience with FIX, but we have found the specification for repeated groups in FIX 4.2 to be somewhat unclear.
Some of the problems that we have encountered are the following:
Consider the message List Status. Tags 354 and 355 in the message body are defined but not required. These tags are also defined as the last tags in the repeated group. Since fields in the body may occur in any order, they might in a particular message occur just after the repeated groups. Suppose that in the last instance of the repeated group these tags have not been set. When this message is parsed, it is impossible to say whether the tags belong to the repeated group or to the body. Therefore, we have assumed that as long as fields are defined in a particular repeated group, they should be considered part of it. However, as far as we can see, this procedure is not described in the FIX 4.2 specification. This procedure also makes it impossible to allow fields to occur “in any order” since, if one would like to set tags 354 and 355 in the body but not in the repeated group, they must not be placed immediately after the repeated group.
In the 4.2 specification, it is not always stated that the “number of”-tag (i.e. the tag which indicates the number of repeats) is required if the corresponding repeating group is used. When implementing FIX 4.2, we have found it necessary either to state that this field must indeed be given, or to state that the delimiter tag must not occur anywhere else in the message body (perhaps in reality they don’t). If not, in some cases, these other occurrences of the delimiter tag will be considered instances of the repeated group. It would be helpful if either procedure (preferably the first) were given in the protocol specification.
It seems to us, that the 4.2 specification does not explicitly state that the instances of a repeated group must follow each other in a message. One could get the impression that there may be other tags in between. Neither does it state that the “number of”-tag must immediately precede the repeated group. If this were stated, an implementer would not need to worry about the possibility that the delimiter tag (the first tag in the repeated group) occur in the message as something else than the specific delimiter. To illustrate the problem, suppose that field x is the delimiter for a repeated group, for which no other fields are required, and that x is also a defined field elsewhere in the message body. Suppose that in a particular message the “number of”-field indicates that there is only one group in the section of repeated groups, and that the only tag of the repeated group is the delimiter (i.e. none of the tags which are not required are set). Now, when a parser of this message finds a field x, it cannot say whether it is the single field, or the repeated group (which consists of only one field).
Many of our problems probably have to do with the fact that we are not aware of how FIX is usually implemented. However, it would have saved us quite a lot of work if the above procedures were given in the protocol specification, and were not a matter for the parties implementing the protocol.
[ original email was from Ryan Pierce - rpierce@taltrade.com ]
The generally accepted way in FIX of implementing repeating groups is for the "number of" field to immediately preceed the repeating group, then the entirety of one record in the group is sent, then the next record, etc. without intervening fields not part of the repeating group. Generally, there are parsing aids, such as the conditional requirement that a certain field appear first in the group.
This can cause issue given that FIX is extensible; a firm may elect to send a user-defined field, or to make matters more confusing, a field not normally associated with that message, within a repeating group. This is why anything user-defined must be made extremely explicit when repeating groups are concerned.