Imported from previous forum
On the call the need to represent calculated fields in meta data was discussed by Hanno and Greg. The feeling was, in a tunneling scenario where a FIX tag=value message is being reconstructed, there should be a standard way of indicating fields, such as checksum, message length need to be recalculated.
Hanno added that sometimes one FIX field might be segments, such as CFICode into constituent parts. Would it be useful to have meta data on how to reconstruct the data from its constituent FAST fields?
Response from Hanno:
Another example for the concatenation is again the UTCTimestamp datatype in FIX. If it can be split into date+time on the wire, then date can be coded very efficiently as a constant if it does not change during the session, requiring only time to be sent on the wire. The receiving end can put the two fields back together to create a valid FIX datatype again.
Further Response from Hanno:
I agree, you could even combine the two to have a smaller integer value for msec since midnight instead of epoch. We do not need to define a single method, rather a toolbox of options to be chosen from. Greg made the case for ISE where members want to pick up the binary version whereas CME has stuck to the string representation and omitted the punctuation marks. The latter will benefit more from a concatenation approach than the former but there is no need to rule one or the other out. As you said, this is up to testing to determine the most efficient way.
The language to describe such conversions is what is needed for any of this. It is only needed in the context of standard messages where datatypes, field orders and field content are typically defined from a non-technical perspective. I believe this is a good thing. However, it does not preclude optimization on a technical level which is what FAST is intended for.
On the call the need to represent calculated fields in meta data was
discussed by Hanno and Greg. The feeling was, in a tunneling scenario
where a FIX tag=value message is being reconstructed, there should be a
standard way of indicating fields, such as checksum, message length need
to be recalculated.Hanno added that sometimes one FIX field might be segments, such as
CFICode into constituent parts. Would it be useful to have meta data on
how to reconstruct the data from its constituent FAST fields?Response from Hanno:
Another example for the concatenation is again the UTCTimestamp
datatype in FIX. If it can be split into date+time on the wire, then
date can be coded very efficiently as a constant if it does not change
during the session, requiring only time to be sent on the wire. The
receiving end can put the two fields back together to create a valid
FIX datatype again.
No formal notation would be necessary to express that for example checksum and message length should be recalculated in a tunneling scenario. The FIX-over-FAST document should mention that such fields are not to be present in the FAST version of the message since they are artifacts of the tag=value encoding.
In earlier drafts of FAST there was the notion of a computed field, later renamed to implicit field, which was later droped alltogether since the construct was considered purely informational (and with very little practical value)
As discussed earlier, timestamps are best (most compact and efficiently) represented as a number of milli/nano/seconds since an EPOCH, together with the delta operator. What we need to discuss further is how to indicate the resolution and epoch used for a particular field.
Sending timestamps as strings should still be an option, but designing string de-/composition trickery seems suboptimal and defeats the purpose of FAST beeing efficient. If you want a simple solution in an otherwise string based context stick with full tsp strings and a suitable operator. If you want best overall performance, send integers with e.g. milliseconds since EPOCH.
For the timestamp handling I agree we’ll need some formal notation. Ideally the next revision of FAST will include a special timestamp datatype for this purpose. In the meantime, some simple heuristcs and documentation in the FIX-over-FAST doucment would cover the most basic and immediate needs.
/David
On the call the need to represent calculated fields in meta data was
discussed by Hanno and Greg. The feeling was, in a tunneling scenario
where a FIX tag=value message is being reconstructed, there should be a
standard way of indicating fields, such as checksum, message length need
to be recalculated.Hanno added that sometimes one FIX field might be segments, such as
CFICode into constituent parts. Would it be useful to have meta data on
how to reconstruct the data from its constituent FAST fields?Response from Hanno:
Another example for the concatenation is again the UTCTimestamp
datatype in FIX. If it can be split into date+time on the wire, then
date can be coded very efficiently as a constant if it does not change
during the session, requiring only time to be sent on the wire. The
receiving end can put the two fields back together to create a valid
FIX datatype again.
David,
that sounds simple for computable fields but I believe some more detail needs to be explained. If we assume a FAST decoder and a FIX engine (and nothing else) then the FAST decoder does not know it is decoding a FIX message (and hence does not produce ChkSum) and the FIX engine assumes that it receives a valid FIX message (and hence also does not produce ChkSum). Conceptually, who takes care of the missing bit? Is it a third, separate component or an integral part of one of the other two?
Regards,
Hanno.
No formal notation would be necessary to express that for example
checksum and message length should be recalculated in a tunneling
scenario. The FIX-over-FAST document should mention that such fields are
not to be present in the FAST version of the message since they are
artifacts of the tag=value encoding.In earlier drafts of FAST there was the notion of a computed field,
later renamed to implicit field, which was later droped alltogether
since the construct was considered purely informational (and with very
little practical value)As discussed earlier, timestamps are best (most compact and efficiently)
represented as a number of milli/nano/seconds since an EPOCH, together
with the delta operator. What we need to discuss further is how to
indicate the resolution and epoch used for a particular field.Sending timestamps as strings should still be an option, but designing
string de-/composition trickery seems suboptimal and defeats the purpose
of FAST beeing efficient. If you want a simple solution in an otherwise
string based context stick with full tsp strings and a suitable
operator. If you want best overall performance, send integers with e.g.
milliseconds since EPOCH.For the timestamp handling I agree we’ll need some formal notation.
Ideally the next revision of FAST will include a special timestamp
datatype for this purpose. In the meantime, some simple heuristcs and
documentation in the FIX-over-FAST doucment would cover the most basic
and immediate needs./David
On the call the need to represent calculated fields in meta data was
discussed by Hanno and Greg. The feeling was, in a tunneling scenario
where a FIX tag=value message is being reconstructed, there should be
a standard way of indicating fields, such as checksum, message length
need to be recalculated.Hanno added that sometimes one FIX field might be segments, such as
CFICode into constituent parts. Would it be useful to have meta data
on how to reconstruct the data from its constituent FAST fields?Response from Hanno:
Another example for the concatenation is again the UTCTimestamp
datatype in FIX. If it can be split into date+time on the wire, then
date can be coded very efficiently as a constant if it does not change
during the session, requiring only time to be sent on the wire. The
receiving end can put the two fields back together to create a valid
FIX datatype again.
The part that is composing the tag=value message is responsible for supplying the body length and checksum fields. If that component is directly integrated with the FAST decoder or is implemented as a separate component that works off some other intermediate format that the FAST decoder produces, doesn’t matter. The tag=value encoder must obviously know it is encoding the tag=value format and should have no problem supplying a checksum. No metadata should be needed to state this fact.
/David
David,
that sounds simple for computable fields but I believe some more detail
needs to be explained. If we assume a FAST decoder and a FIX engine (and
nothing else) then the FAST decoder does not know it is decoding a FIX
message (and hence does not produce ChkSum) and the FIX engine assumes
that it receives a valid FIX message (and hence also does not produce
ChkSum). Conceptually, who takes care of the missing bit? Is it a third,
separate component or an integral part of one of the other two?Regards, Hanno.
No formal notation would be necessary to express that for example
checksum and message length should be recalculated in a tunneling
scenario. The FIX-over-FAST document should mention that such fields
are not to be present in the FAST version of the message since they
are artifacts of the tag=value encoding.In earlier drafts of FAST there was the notion of a computed field,
later renamed to implicit field, which was later droped alltogether
since the construct was considered purely informational (and with very
little practical value)As discussed earlier, timestamps are best (most compact and
efficiently) represented as a number of milli/nano/seconds since an
EPOCH, together with the delta operator. What we need to discuss
further is how to indicate the resolution and epoch used for a
particular field.Sending timestamps as strings should still be an option, but designing
string de-/composition trickery seems suboptimal and defeats the
purpose of FAST beeing efficient. If you want a simple solution in an
otherwise string based context stick with full tsp strings and a
suitable operator. If you want best overall performance, send integers
with e.g. milliseconds since EPOCH.For the timestamp handling I agree we’ll need some formal notation.
Ideally the next revision of FAST will include a special timestamp
datatype for this purpose. In the meantime, some simple heuristcs and
documentation in the FIX-over-FAST doucment would cover the most basic
and immediate needs./David
On the call the need to represent calculated fields in meta data was
discussed by Hanno and Greg. The feeling was, in a tunneling
scenario where a FIX tag=value message is being reconstructed, there
should be a standard way of indicating fields, such as checksum,
message length need to be recalculated.Hanno added that sometimes one FIX field might be segments, such as
CFICode into constituent parts. Would it be useful to have meta data
on how to reconstruct the data from its constituent FAST fields?Response from Hanno:
Another example for the concatenation is again the UTCTimestamp
datatype in FIX. If it can be split into date+time on the wire, then
date can be coded very efficiently as a constant if it does not
change during the session, requiring only time to be sent on the
wire. The receiving end can put the two fields back together to
create a valid FIX datatype again.