Imported from previous forum
Should it matter if a Header field is contained within the Body. My example is field (43) being passed at the end of the Body, right in front of the checksum (10), rather than in the Header of the message. This is FIX 4.0.
[ original email was from Oren Miller - omiller@thoughtworks.com ]
> Should it matter if a Header field is contained within the Body. My example is field (43) being passed at the end of the Body, right in front of the checksum (10), rather than in the Header of the message. This is FIX 4.0.
>
This is how the FIX expected behavior document (http://www.fixprotocol.org/ORGANIZATIONS/928202077/FIX_TestCase_SessionLevel20010710.doc)
says this should be handled:
"g. Receive a message in which the following is not true: Standard Header fields appear before Body fields which appear before Standard Trailer fields.
- Send Reject (session-level) message referencing incorrect message structure header+body+trailer
- Increment inbound MsgSeqNum
- Generate an "error" condition in test output."
That being said, there may be some systems beyond your control that do mix up the order of fields. In that situation, hopefully your engine has a setting that allows you to process such messages.
> > Should it matter if a Header field is contained within the Body. My example is field (43) being passed at the end of the Body, right in front of the checksum (10), rather than in the Header of the message. This is FIX 4.0.
> >
>
> This is how the FIX expected behavior document (http://www.fixprotocol.org/ORGANIZATIONS/928202077/FIX_TestCase_SessionLevel20010710.doc)
> says this should be handled:
>
> "g. Receive a message in which the following is not true: Standard Header fields appear before Body fields which appear before Standard Trailer fields.
>
> 1. Send Reject (session-level) message referencing incorrect message structure header+body+trailer
> 2. Increment inbound MsgSeqNum
> 3. Generate an "error" condition in test output."
>
> That being said, there may be some systems beyond your control that do mix up the order of fields. In that situation, hopefully your engine has a setting that allows you to process such messages.
>
>
Sorry, three commercially available engines. These session-level tests do not refer to 4.0.
[ original email was from Oren Miller - omiller@thoughtworks.com ]
> Sorry, three commercially available engines. These session-level tests do not refer to 4.0.
Really? On the third page of the document after the table of contents it reads:
"This document is applicable to all versions of FIX 4.X (4.0, 4.1, 4.2, and 4.3) except where explicitly indicated."
Since that test makes no explicit exemption for FIX 4.0, it seems to be included.
> > Sorry, three commercially available engines. These session-level tests do not refer to 4.0.
>
> Really? On the third page of the document after the table of contents it reads:
>
> "This document is applicable to all versions of FIX 4.X (4.0, 4.1, 4.2, and 4.3) except where explicitly indicated."
>
> Since that test makes no explicit exemption for FIX 4.0, it seems to be included.
>
The header-body-trailer rule only came in the errata of FIX.4.1, two years after FIX.4.0. So the session level tests contradict a spec that is well over five years old. If an engine is still running 4.0 then it is likely to only obey this rule by accident or due to an upgrade for later versions of FIX. If it does not then after this period of time you are going to need leverage on the order of compromising photos with animals to get them to change on the basis of quoting these tests.
Maybe Ryan or someone else can say if the intention was to retroactively impose a restriction over five years after the original spec.
What I was saying is that for FIX.4.0, to avoid pain, be generous what you accept and strict what you send.
[ original email was from Oren Miller - omiller@thoughtworks.com ]
> The header-body-trailer rule only came in the errata of FIX.4.1, two years after FIX.4.0. So the session level tests contradict a spec that is well over five years old. If an engine is still running 4.0 then it is likely to only obey this rule by accident or due to an upgrade for later versions of FIX. If it does not then after this period of time you are going to need leverage on the order of compromising photos with animals to get them to change on the basis of quoting these tests.
>
> Maybe Ryan or someone else can say if the intention was to retroactively impose a restriction over five years after the original spec.
>
> What I was saying is that for FIX.4.0, to avoid pain, be generous what you accept and strict what you send.
>
I agree with you 100% on that point. Evey engine should at least provide the option to ignore field ordering. But keep in mind in the 4.0 spec under message format it does read:
"The general format of a FIX message is a standard header followed by the message body fields and terminated with a standard trailer. "
Which is the exact same sentence found in all later versions of FIX. The later versions explicitly point out that because of that rule you cannot move fields out of the header etc, but the rule was there. The intention was ambigous originally so they clarified it. Even the terminology used (header, body, trailer) seems to indicate an intended order. So all I am saying is that I don’t think it is accurate to call an engine that implements the 4.0 standard in this way as non-compliant. I will agree that it isn’t an extremely useful engine, simply because in the real world it simply won’t be able to talk to as many counterparties, which I know we both agree is the real goal. And that is exactly why I originally indicated that it could be a problem, depending on what engines are involved.
I don’t really think we disagree on anything at all except the intention of an ambiguous/contradictory portion of the FIX 4.0 spec. It doesn’t much change the fact, as you correctly pointed out, that in these situations we must implement the less restrictive interpretation at the very least.
I would welcome an official interpretation of the original intention just for the sake of curiosity. Is there anyone who was involved in the writing of that section that would know?
The FIX spec has always had the header-body-trailer statement. I have been the editor of the spec since FIX 4.1 which was released over 4 years ago. The specific statement that you are referring to dates back to the very first version of the FIX Spec, version 2.7(a), with a January 1995 date. The editors of the original spec were Chris Morstatt and Bob Lamoureux.
In researching this, the original FIX spec, FIX 2.7(a) states:
- "The general format of a message is a standard message header followed by message body data and a standard trailer."
- "Each message, administrative or application, is preceded by a standard header."
- "The application message is composed of the standard header followed by the message body and trailer."
- and "Except where noted, fields within a message can be defined in any sequence (i.e. relative position of a field within a record is inconsequential except where explicitly defined otherwise)."
======
I imagine this could be debated for some time as to whether or not this is an ambiguity… My belief is that the original intention was header then body then trailer. My personal interpretation of the English words "followed by" and "preceded by" implies a specific order and 3 references to it within a spec this constitutes "where noted".
I recommend the philosophy of construct in order and be flexible in parsing.
===
Also, this discussion thread references the clarification made to the FIX 4.1 Errata 19960630. That Errata change was:
[E19990129_14] Message construction clarification:
Message construction specifically header + body + trailer vs. "any order" needs clarification. Note the “Message Format” section currently states: “The general format of a FIX message is a standard header followed by the message body fields and terminated with a standard trailer”, however it also states “Except where noted, fields within a message can be defined in any sequence…Any exceptions are explicitly defined otherwise…” The first statement should apply as a noted exception but needs to be stressed.
Add “, and general message format of standard header followed by body followed by standard trailer.” to the list of bolded, stated exceptions to fields in any sequence. Add “Note if encryption used, the SecureData field is a header field and thus the body data will appear between the header and trailer.”
> > The header-body-trailer rule only came in the errata of FIX.4.1, two years after FIX.4.0. So the session level tests contradict a spec that is well over five years old. If an engine is still running 4.0 then it is likely to only obey this rule by accident or due to an upgrade for later versions of FIX. If it does not then after this period of time you are going to need leverage on the order of compromising photos with animals to get them to change on the basis of quoting these tests.
> >
> > Maybe Ryan or someone else can say if the intention was to retroactively impose a restriction over five years after the original spec.
> >
> > What I was saying is that for FIX.4.0, to avoid pain, be generous what you accept and strict what you send.
> >
>
> I agree with you 100% on that point. Evey engine should at least provide the option to ignore field ordering. But keep in mind in the 4.0 spec under message format it does read:
>
> "The general format of a FIX message is a standard header followed by the message body fields and terminated with a standard trailer. "
>
> Which is the exact same sentence found in all later versions of FIX. The later versions explicitly point out that because of that rule you cannot move fields out of the header etc, but the rule was there. The intention was ambigous originally so they clarified it. Even the terminology used (header, body, trailer) seems to indicate an intended order. So all I am saying is that I don’t think it is accurate to call an engine that implements the 4.0 standard in this way as non-compliant. I will agree that it isn’t an extremely useful engine, simply because in the real world it simply won’t be able to talk to as many counterparties, which I know we both agree is the real goal. And that is exactly why I originally indicated that it could be a problem, depending on what engines are involved.
>
> I don’t really think we disagree on anything at all except the intention of an ambiguous/contradictory portion of the FIX 4.0 spec. It doesn’t much change the fact, as you correctly pointed out, that in these situations we must implement the less restrictive interpretation at the very least.
>
> I would welcome an official interpretation of the original intention just for the sake of curiosity. Is there anyone who was involved in the writing of that section that would know?
>
> Should it matter if a Header field is contained within the Body. My example is field (43) being passed at the end of the Body, right in front of the checksum (10), rather than in the Header of the message. This is FIX 4.0.
>
FIX.4.0 does not order message fields. FIX.4.1 and above do. I know of at least two commercial engines for which this is a major problem, although one has a work around. It is typically an issue with the newer or rarely used engines.
My advice is order what you send in header-body-trailer and be prepared to receive in any order.
FIX 4.0 suggests that "the general format of a FIX message is a standard header followed by the message body fields and terminated with a standard trailer." However, it goes on to state "except where noted, fields within a message can be defined in any sequence…". The main exceptions are tags 8, 9 and 35 must be first, second and third tags of the message, tag 10 must be the last tag of the message.
I’m in agreement with Danny Shobrook that you should expect certain allowable header fields to be “out of order”. FIX 4.2 and 4.3 was very explicit with the header-body-trailer rule (and probably 4.1 also).
> Should it matter if a Header field is contained within the Body. My example is field (43) being passed at the end of the Body, right in front of the checksum (10), rather than in the Header of the message. This is FIX 4.0.
>