Imported from previous forum
Quote:
"The delta operator has the following presence map utilization:
- Mandatory integer, decimal, string and byte vector fields – no bit.
- Optional integer, decimal, string and byte vector fields – no bit.
The delta appears in the stream in a nullable representation.
A NULL indicates that the delta is absent. Note that the previous
value is not set to empty but is left untouched if the value is
absent." (Section 10.5.1 of FAST Specification)
My question is with the last sentence. If a NULL value is found in the stream, the previous is left untouched. However, what is returned to the user? In other words is the “current value” after decoding? Is it a NULL value or the previous value?
[ original email was from Greg Orsini - greg.orsini@orcsoftware.com ]
A NULL value indicates that the field is not present in the message.
This allows a template to define a message that may sometimes include a field. For example, a FIX OrderSingle template could have an optional LimitPrice field.
Quote: "The delta operator has the following presence map utilization:
- Mandatory integer, decimal, string and byte vector fields – no bit.
- Optional integer, decimal, string and byte vector fields – no bit. The
delta appears in the stream in a nullable representation. A NULL
indicates that the delta is absent. Note that the previous value is
not set to empty but is left untouched if the value is absent."
(Section 10.5.1 of FAST Specification)My question is with the last sentence. If a NULL value is found in the
stream, the previous is left untouched. However, what is returned to the
user? In other words is the “current value” after decoding? Is it a NULL
value or the previous value?
Hi Joshua,
a NULL value should be passed to the user.
The reason for not changing previous value is a supposed
affinitiy to a next non-NULL value. The benefit of this
scheme is obviously data dependent. If you have a field
with a large frequency of NULL values, then you would have
a benefit from updating the previous value to NULL. This
is however not what 1.1 specifies.
It’s another example of a data dependent trade-off.
There may have been other considerations, but I don’t
remember off-hand. David may have something to add.
Best,
Rolf
Quote: "The delta operator has the following presence map utilization:
- Mandatory integer, decimal, string and byte vector fields – no bit.
- Optional integer, decimal, string and byte vector fields – no bit.
The
delta appears in the stream in a nullable representation. A NULL
indicates that the delta is absent. Note that the previous value is
not set to empty but is left untouched if the value is absent."
(Section 10.5.1 of FAST Specification)My question is with the last sentence. If a NULL value is found in the
stream, the previous is left untouched. However, what is returned to the
user? In other words is the “current value” after decoding?
Is it a NULL value or the previous value?
Since the DELTA operator does not use a slot in the Presence Map a value must always exist in the stream. Therefore how would an incoming message reuse the previous value as is?
Example:
For simplicity, suppose you had message with one DECIMAL field using the DELTA operator.
Message#1 has a value of 2.25. Previous value is 2.25
Message#2 has a value of NULL. Previous value is still 2.25
*Now, message #3 comes in and after decoding it should return a value of 2.25 to the user.
How do we accomplish that? Since a value must always be sent, it must send a non NULL value. Therefore, it seems to me that a value of 0.0 (mantissa=0 exponent=0) must be sent in the stream in order for the user to get 2.25.
Although that is minimal data being transmitted, why should data be sent at all?
Lastly, why does the delta operator not occupy a slot in the presence map? It seems in consistent because it’s very similar to the TAIL operator.
Joshua, you’re making some good points.
Pls find my comments below
/Rolf
Since the DELTA operator does not use a slot in the Presence
Map a value must always exist in the stream. Therefore how
would an incoming message reuse the previous value as is?
And this is the reason why updating the prev value doesn’t
buy you anything.
Example:
For simplicity, suppose you had message with one DECIMAL
field using the DELTA operator.Message#1 has a value of 2.25. Previous value is 2.25
Message#2 has a value of NULL. Previous value is still 2.25*Now, message #3 comes in and after decoding it should
return a value of 2.25 to the user.How do we accomplish that? Since a value must always be
sent, it must send a non NULL value. Therefore, it seems
to me that a value of 0.0 (mantissa=0 exponent=0) must
be sent in the stream in order for the user to get 2.25.Although that is minimal data being transmitted, why
should data be sent at all?
Because you don’t have a way (in 1.1) to express “unchanged”.
(ie. default=0 / zero)
Lastly, why does the delta operator not occupy a slot in
the presence map? It seems inconsistent because it’s very
similar to the TAIL operator.
This is a possible extension:
A combination of delta and default or copy maybe.
A field that is frequently NULL or successive values are within
a narrow range could use delta + default NULL, or delta + copy.
A field that is frequently zero could use default 0 (zero).
Anyway, this is not in the current spec.
[ original email was from John Cameron - john.cameron@orcsoftware.com ]
> > Since the DELTA operator does not use a slot in the Presence Map a
value must always exist in the stream. Therefore how would an incoming
message reuse the previous value as is?And this is the reason why updating the prev value doesn’t buy
you anything.
Therefore I assume that the first line of example 3 in Appendix 3.2.5 of the FAST 1.1 spec is incorrect?
It has no value appearing in the stream
John
You’re right, the example is incorrect or at least confusing.
If the first line represents the first value transferred (which would be the most apparent interpretation) The first line should read
12000 | 3 | 12 | 0 | 0 | N/A | 0x80 | 0x80
On the other hand, it could be that the first line represents the initial value as hinted in the table header (it should then have had N/A in the prev value colums too).
Nevertheless, I think the first line should be changed or simply removed. We’ll have to add that to the list of issues going into the errata.
/David
Since the DELTA operator does not use a slot in the Presence Map a
value must always exist in the stream. Therefore how would an
incoming message reuse the previous value as is?And this is the reason why updating the prev value doesn’t buy you
anything.Therefore I assume that the first line of example 3 in Appendix 3.2.5 of
the FAST 1.1 spec is incorrect?It has no value appearing in the stream
John
[ original email was from Matt Simpson - msimpson@cme.com ]
The primary intent of the example is to illustrate the normalization rule for decimal fields when an initial value is specified. Note that 12000 is expressed as a mantissa of 12 as required by the spec.
The secondary intent is to illustrate how initial values can be used to prime the encoder. The first row is showing the state of the encoder at “start up”. 12000 is the initial value rather than an input value. Prior Values should be set to N/A as well.
The example will be changed to clarify this.
You’re right, the example is incorrect or at least confusing.
If the first line represents the first value transferred (which would be
the most apparent interpretation) The first line should read12000 | 3 | 12 | 0 | 0 | N/A | 0x80 | 0x80
On the other hand, it could be that the first line represents the
initial value as hinted in the table header (it should then have had N/A
in the prev value colums too).Nevertheless, I think the first line should be changed or simply
removed. We’ll have to add that to the list of issues going into
the errata./David
Since the DELTA operator does not use a slot in the Presence Map a
value must always exist in the stream. Therefore how would an
incoming message reuse the previous value as is?And this is the reason why updating the prev value doesn’t buy you
anything.Therefore I assume that the first line of example 3 in Appendix 3.2.5
of the FAST 1.1 spec is incorrect?It has no value appearing in the stream
John
The usage for the delta operator is when you have a value that is present most of the time but seldom the same as the previous value. In this case a bit in the presence map would be unnecessary, hence the decision not to use one. On the other hand, if your value is often the same or at least often the same as the previous, then you should use the default or copy operators as Rolf suggested.
The definition of the tail operator origins from the 1.0 specification and is retained for compatibility reasons. It’s specifically designed for equal length strings that changes in the tail part only. The most apparent use case is ISIN codes.
A note about NULL values: NULL is an artifact of the encoding and is used to express absence of a value. It is implementation dependent how the absence of a value is communicated from the decoder to the application. A particular implementation may use a special NULL value to do this. Another may simply do nothing upon receiving a NULL. In FIX for example, the absence of a value is indicated by simply leaving the field out.
/David
Since the DELTA operator does not use a slot in the Presence Map a value
must always exist in the stream. Therefore how would an incoming message
reuse the previous value as is?Example:
For simplicity, suppose you had message with one DECIMAL field using the
DELTA operator.Message#1 has a value of 2.25. Previous value is 2.25 Message#2 has a
value of NULL. Previous value is still 2.25*Now, message #3 comes in and after decoding it should return a value of
2.25 to the user.How do we accomplish that? Since a value must always be sent, it must
send a non NULL value. Therefore, it seems to me that a value of 0.0
(mantissa=0 exponent=0) must be sent in the stream in order for the user
to get 2.25.Although that is minimal data being transmitted, why should data be
sent at all?Lastly, why does the delta operator not occupy a slot in the
presence map? It seems in consistent because it’s very similar to
the TAIL operator.