Imported from previous forum
It seems that there have been a lot of issues when implementing decimal fields with individual operators for the mantissa and exponent. I know from experience with OpenFAST that implementing this took a lot of time and caused a refactor of the entire code base between 1.0 and 1.1. At the time it didn’t seem like a big enough issue to bring up, because of the benefit to possibly decreasing the message size. It is natural when creating a specification for compressing data to not having any useless bits.
However, there have been many posts about this single issue and it seems to be the largest obstacle to building a FAST implementation. Supporting this part of the specification prevents FAST implementations from providing many features which decrease encoding and decoding latency (e.g. random access messages, known pmap sizes).
The parts of the specification in question are as follows:
From 6.2.2 Decimal Field Instructions
“If the decimal field has optional presence and has individual operators, the presence of the mantissa is dependent on the presence of the exponent. See the section Presence Map and NULL Utilization for the definition.”
From 10.5.1 Presence Map and NULL Utilization
“If the mantissa has an operator that requires a bit in the presence map, this bit is present iff the exponent value is considered present.”
I recommend changing the line in 10.5.1 to: “If the mantissa has an operator that requires a bit in the presence map, this bit is present but ignored when the exponent value is considered absent.”
Jake
The rationale for the current definitions is sanity and consistency.
When implementing a FAST processor it may help to think of it this way:
In the general case the pmap is a stream of bits. Processing an operator that requires a presence bit will consume a bit from the stream. Handling variable number of bits gets trivial with this model.
For some (actually most) templates it is possible to statically identify an optimization opportunity: bits are allocated at deterministic positions. A processor may use this information to improve performance.
/David
It seems that there have been a lot of issues when implementing
decimal fields with individual operators for the mantissa and
exponent. I know from experience with OpenFAST that implementing this
took a lot of time and caused a refactor of the entire code base
between 1.0 and 1.1. At the time it didn’t seem like a big enough
issue to bring up, because of the benefit to possibly decreasing the
message size. It is natural when creating a specification for
compressing data to not having any useless bits.However, there have been many posts about this single issue and it seems
to be the largest obstacle to building a FAST implementation. Supporting
this part of the specification prevents FAST implementations from
providing many features which decrease encoding and decoding latency
(e.g. random access messages, known pmap sizes).The parts of the specification in question are as follows:
From 6.2.2 Decimal Field Instructions
“If the decimal field has optional presence and has individual
operators, the presence of the mantissa is dependent on the presence of
the exponent. See the section Presence Map and NULL Utilization for the
definition.”From 10.5.1 Presence Map and NULL Utilization
“If the mantissa has an operator that requires a bit in the presence
map, this bit is present iff the exponent value is considered present.”I recommend changing the line in 10.5.1 to: “If the mantissa has an
operator that requires a bit in the presence map, this bit is present
but ignored when the exponent value is considered absent.”Jake