On the subject of decimal encoding, we should define the members of the composite encoding, as suggested by H Chan. It seems that the term "mantissa" is discuouraged by IEEE and others in this context in favor of "significand" or "coefficient". We can define it as the part of the encoding that represents the significant digits and sign of a decimal number. Exponent represents scale as a power of 10.
Is it a typo that Multi-value choice encoding uses <enum>...</enum> in section 2.12.4? If not, how can I differeniate Multi-value choice encoding from Enumeration encoding?
Yes, that was a typo in the spec that has been corrected in subsequent drafts. Multi-value choice uses tag <set> to distinguish it from enumeration (single choice). It is encoded as a bitset. Here is an example (not all values shown):
<type name="bitsetType" primitiveType="uint8"/>
<set name="QuoteConditionSet" encodingType="bitsetType">
<choice name="Open" description="Open/Active">0</choice>
<choice name="Closed" description="Closed/Inactive">1</choice>
<choice name="ExchangeBest" description="Exchange Best">2</choice>
</set>
sort of a side question -- is there a version of spec newer than the RC1?
RC2 is being worked on - Don is making updates as he gets feedback from the community.
Don't know when he plans to release it. I think it will be discussed at today's GTC meeting.
/Rolf
I have received questions about how to test whether an optional field is null in SBE when it has a composite encoding. Two cases are explicitly given in the spec:
- An optional Decimal field is null if its mantissa is null. (The other part of Decimal is exponent.)
- An optional MonthYear field is null if its year subfield is null. (The subfields in the encoding are year, month, day and week, in that order.)
Users have suggested that we give a generic rule, such as: for a composite type, test its first member for null value. This is consistent with the two cases above and seems reasonable for any composite types that may be added in the future.
In the SBE spec, we have tried hard to separate protocol layers. There is no dependency on lower layers such as session or transport protocols. Encoding is presentation layer (layer 6 in the OSI model), but we also have mapping to application layer in these message schema attributes (all are optional in XML):
- The "messageSchema" element has attribute "fixVersion", which would be populated with something like "FIX.5.0_SP2".
- The "message" element has attribute has attribute "fixMsgType", which would be populated with something like "D" for a single order message. (MsgType is not encoded in SBE on the wire; it is only documented in the schema.)
- Fields have attribute "fixUsage" for semantic type. For example, the encoding of a field could be uint64 while its fixUsage is UTCTimesamp.
It has been suggested by a reviewer that we make a stronger separation of encoding from application layer by naming the attributes "semantic" or something similar. Attribute "fixVersion" would change to "sematicVersion" while every other element would have optional attribute "semanticType" to distinguish from encoding type.
Feedback from an implementer:
- Elements within a composite type should allow an "offset" attribute, like fields, in order to control byte alignment. It would represent offset from the beginning of the composite
- Composite types should allow enums and bitsets as members.
#1 Should be easy to add to the spec and does not conflict with any other requirement.
#2 I don't know of a FIX use case that requires this, but it seems reasonable to make SBE more general purpose. This would require a bit of a change in the way enums are declared, however. To be a member of a composite, an object needs to be a kind of "type" ("is a" relationship). Enums currently list valid values but refer to a type ("has a" relationship with a wire format).
Feedback from an implementer:
How can a variable-length data field be set to null? The length can be set to zero to represent an empty a string, but there may be a semantic difference between an empty string and a null string.
Also, there is no need to offer a choice of char or uint8 for the primitive type of a data field since the "characterEncoding" attribute (formerly "encoding") tells how the data is formatted. This gives two ways to do the same thing. That is, if the data is a variable-length ASCII string, it can be specified as characterEncoding="US-ASCII".
Version 1.0 of SBE should be focussed on use cases related to FIX. In the same way that we want to offer a solution for nested groups although we do not recommend to use them we should initially not offer solutions for problems that do not exist when using the FIX application layer. Maybe there is a use case but it needs to be described first.
What is the semantic difference between an empty string (if this is not about a string of blanks) and a null string?
present / not present
As far as I can tell, every field of "data" datatype in FIX is optional. Users may also choose to implement String fields in SBE as variable-length encoding, and they may be either required or optional. So a null indicator (not present) may be desirable.
So the empty string is about a string of blanks being present whereas the null value indicates absence of any string. In case of a variable length field I suggest to use length 1 and expect a blank character on the wire to indicate an empty string and use length 0 for a null value. The user can choose to any length bigger than 1 to express an empty string as long as he only uses blanks on the wire. For a fixed length field the spec already defines the NUL control character to be used as a delimiter, i.e. for the first character of the string to express a null value, correct?
Should an empty fixed length string require all characters to be set to blank? The space is available on the wire anyway but the processing may be different. Semantically I do not see a difference between empty strings of different lengths.
one drawback of the encoding proposed by Hanno is that we have to add another rule "one space means empty" which is both less generic and more costly than needed. Why not use an optional integer as length and use a null value for the length? As you know I have my reservations about the null encoding for integers, but if we decide to use this encoding, why not use it in the variable-length string case as well?
There is also the case with a fixed-length optional string, where we cannot rely on a null encoding of the length field.
It would be useful to have a few use cases to analyze. I would guess that fixed length will be used for short strings such as symbols, whereas variable-length strings will primarily be used for string fields that can be very long, but may be short on average? Is there any obvious example of an optional field that should / needs to be fixed length?
My view is that a fixed length field will primarily be used to improve direct access performance.
It may be that users of the encoding will choose to define a default / null value within the value range of the field. For example, spaces may be used to indicate a default, and NULL (ascii 0) may be used to indicate absence. In the case that there is no default value, spaces may be used to indicate absence, but that would be another rule to support. One alternative could be to allow the default and null values to be specified.
As I noted in my previous post, these rules will complicate a generic implementation. It is important to analyze the different options carefully as design decisions will need to be handled by every full featured implementation.
Hello,
Where can I find the SBE specification document? I don't find it on the old web site (http://www.fixprotocol.org/encodings#Simple_Binary_Encoding).
Xavier.
The URL of SBE RC1 specification is http://www.fixtradingcommunity.org/pg/file/fplpo/read/46939/simple-binary-encoding-specification
The High Performance Working Group is striving to publish Release Candidate 2 soon.
The URL of SBE RC1 specification is http://www.fixtradingcommunity.org/pg/file/fplpo/read/46939/simple-binary-encoding-specification
The High Performance Working Group is striving to publish Release Candidate 2 soon.
discussion thread re ASN/GPB/SBE encodings
http://www.fixtradingcommunity.org/pg/discussions/topicpost/165290/proposals-from-the-high-performance-working-group-sbe-gpb-asn1