Hi all,
Appreciate some pointers on the preferred approach to using the FIX SBE with SOFH over a FIXP Session. The decoupled nature of these standards is a great thing, where it offers utmost flexibility to mix and match the technical specifications to best meet the requirements. However, I have come across few grey areas, where I look forward to others thoughts and overall recommendation please.
Example use-case : Implementing a bi-directional un-sequenced flow of FIXP Session of FIX SBE encoded application messages over TCP point to point transport.
I would like to know where exactly the SOFH is expected to be in the layered structure. As per the FIXP Spec (FIXP-Technical-Standard-v1.1-RC1.pdf) in p8, it is mentioned as “may be used” in the presentation layer.
Which, if we choose to use, would in my interpretation sit as follows -
This would mean that there cannot be any optionality around having a FIXP header to facilitate proper decoding.
If the above holds, I have a question on how to decode the message sequence as described in the same FIXP document in section 7.3.3, that explains “transferring” an un-sequenced flow.
Even though, this example illustrates an idle keep alive mechanism, is it appropriate to assume that any SOFH framed FIX SBE message sent in such an un-sequenced flow, should be superseded by an appropriate FIXP header block?
On a related note, section 3.3.3 in the same spec mentions the below text to optionally define a FIXP session layer header for application messages -
FIXP does not require application messages to have a session layer header. Application messages may
have their own presentation layer header, depending on encoding. However, application messages may
immediately follow Sequence without any intervening session layer prologue.
Is there any recommendation to define an appropriate session layer header for application messages?
Appreciate any pointers / recommendations on this please.
The session layer and utilities such as message persistence or protocol analysis need not crack all messages to determine their boundaries.
Mixed encodings are supported on a transport. FIXP session messages and application messages may use different encodings. (Implementations are commonly using SBE for both, but that is not a requirement of FIXP.) Also, application messages need not be all of the same encoding. For example, high performance SBE messages could be interleaved with legacy tagvalue encoding on a back office feed.
For this to work properly, every message on the transport should be prefixed by SOFH, both session and application messages. Your example should be: SOFH / FIXP session message / SOFH / application message. This is not a big burden since SOFH is so small and simple.
The use of SOFH is independent of FIXP flow type. After a session is negotiated, an unsequenced flow uses UnsequencedHeartbeat instead of Sequence, but each of those would be prefixed with SOFH.
Many thanks Donald for your detailed description, which I found very useful!
A follow-up question I have on this recommended approach is, should the SOFH carry an indicator as to whether the payload is going to be a FIXP session message or a FIX SBE Application message (in addition to the standard message length and encoding type as specified in ‘Simple Open Framing Header - Draft Standard v1.0’) ?
I’m asking this because the FIXP Session messages start with the “Message Type” field, whereas the FIX SBE application messages start with an “SBE Root Block Length”.
An interesting thought, but currently SOFH only identifies the length and encoding of a message, not its purpose. It allows a system to then select the correct message decoder to gather more information.
If both FIXP session messages and application messages are encoded with SBE, the decoder can examine an SBE header to determine what kind of message it is. Typically, session and application messages would have a different schemaId in the header, and they would be further differentiated by templateId.
Just to add to Don’s comment. It is important to note again that FIXP does not define an encoding, i.e. “FIXP session message” cannot be compared with a “FIX SBE application message”. You can only compare it to a “FIX application message” (without identifying an encoding). Using SOFH you can use different encodings for session and application messages.
If both are to be encoded with SBE, you have no need for SOFH and you can separate session from application messages as described by Don.
We decided to use SOFH for the benefit of not having to walk through the elements of the message body to determine boundaries over a stream oriented transport. We decided to use big endian FIX SBE encoding across all the layers (the session, SOFH and presentation) for simplicity, because SOFH is mandated to be always FIX SBE big endian.
In this particular approach, if I am to summarize my understanding based on your recommendations:
Both FIXP session and FIX Application messages should be separately framed using the SOFH.
The FIXP Session messages defined in the “FIXP-Technical-Standard-v1.1-RC1”, encoding agnostic specification should, in this case, be modified to have a header, that provides uniformity for decoding. Such header should include ‘schemaId’ and ‘templateId’. (This is the part I was missing, which led to the confusion on decoding)
For this purpose, could we prepend the “Message header schema” defined in sec. 3.2.1 of the “FIX Simple Binary Encoding Technical Specification - Version 2.0 Release Candidate 2 August 13, 2019” to the messages defined in the encoding agnostic FIXP Technical standard?
i.e the following will be prepended to the FIXP messages?
Upon closer look, I believe, we can keep the first four fields in the message header schema invariant and make “numGroups” and “numVarDataFields” optional. We wouldn’t need the latter two for the FIXP session messages.
Does this approach make sense?
@donaldmendelson,
As for the FIXP Session messages, such header with ‘schemaId’ and ‘templateId’ could make the “MessageType” field (which is the first field in the session body) redundant. I guess this is acceptable because the FIXP spec is encoding agnostic.
@thaya, just to confirm that your approach makes sense. The slight redundancy is acceptable and theoretically, you could have multiple templates for a single FIXP message type. This is more applicable to FIX application layer messages like the ExecutionReport(35=8) that is used for many different purposes that do not use the same fields of the message, e.g. confirm order add vs report order execution.
