Imported from previous forum
In Fast 1.0, it was possible to assign two different fields (say, BID_PRICE and TRADE_PRICE) to the same PMAP slot, but different templates, with the particular template chosen at encode/decode time. So, a user could decode the first field (QUOTE_TRADE_INDICATOR, for example), and use its value to determine how a later field in the same message should be treated (continuing this example, as a BID_PRICE or a TRADE_PRICE).
How would you structure this in Fast 1.1 using the templates? It seems that you would need a template to become active during encoding or decoding, based on the VALUE of an already encoded/decoded field. This is different from a dynamic template, where the identifier of the template to be activated is explicitly coded in the stream.
The motivation behind this, by the way, is to try to decode an existing Fast 1.0 stream using a Fast 1.1 decoder. I believe that this could be done relatively easily if the non-existent Template Identifier field is skipped when encoding/decoding messages; but I have run into the issue described above.
David, which feed are you referring to?
/Rolf
In Fast 1.0, it was possible to assign two different fields (say,
BID_PRICE and TRADE_PRICE) to the same PMAP slot, but different
templates, with the particular template chosen at encode/decode time.
So, a user could decode the first field (QUOTE_TRADE_INDICATOR, for
example), and use its value to determine how a later field in the same
message should be treated (continuing this example, as a BID_PRICE or a
TRADE_PRICE).How would you structure this in Fast 1.1 using the templates? It seems
that you would need a template to become active during encoding or
decoding, based on the VALUE of an already encoded/decoded field. This
is different from a dynamic template, where the identifier of the
template to be activated is explicitly coded in the stream.The motivation behind this, by the way, is to try to decode an existing
Fast 1.0 stream using a Fast 1.1 decoder. I believe that this could be
done relatively easily if the non-existent Template Identifier field is
skipped when encoding/decoding messages; but I have run into the issue
described above.
Rolf, this was a custom feed provided by one of our vendors.
David, which feed are you referring to? /Rolf
In Fast 1.0, it was possible to assign two different fields (say,
BID_PRICE and TRADE_PRICE) to the same PMAP slot, but different
templates, with the particular template chosen at encode/decode time.
So, a user could decode the first field (QUOTE_TRADE_INDICATOR, for
example), and use its value to determine how a later field in the same
message should be treated (continuing this example, as a BID_PRICE or
a TRADE_PRICE).How would you structure this in Fast 1.1 using the templates? It seems
that you would need a template to become active during encoding or
decoding, based on the VALUE of an already encoded/decoded field. This
is different from a dynamic template, where the identifier of the
template to be activated is explicitly coded in the stream.The motivation behind this, by the way, is to try to decode an
existing Fast 1.0 stream using a Fast 1.1 decoder. I believe that this
could be done relatively easily if the non-existent Template
Identifier field is skipped when encoding/decoding messages; but I
have run into the issue described above.
David,
pmap slot allocation is local to a specific template.
each message identifies which template to use in decoding.
the following rules apply for a FAST encoded stream:
- each message begins with a pmap.
- the first field (following the pmap) is a tid (template id).
- the tid uses copy coding.
- the first slot in the message level pmap is used for the tid.
otherwise the format is not FAST compliant.
/Rolf
Rolf, this was a custom feed provided by one of our vendors.
David, which feed are you referring to? /Rolf
In Fast 1.0, it was possible to assign two different fields (say,
BID_PRICE and TRADE_PRICE) to the same PMAP slot, but different
templates, with the particular template chosen at encode/decode
time. So, a user could decode the first field
(QUOTE_TRADE_INDICATOR, for example), and use its value to determine
how a later field in the same message should be treated (continuing
this example, as a BID_PRICE or a TRADE_PRICE).How would you structure this in Fast 1.1 using the templates? It
seems that you would need a template to become active during
encoding or decoding, based on the VALUE of an already
encoded/decoded field. This is different from a dynamic template,
where the identifier of the template to be activated is explicitly
coded in the stream.The motivation behind this, by the way, is to try to decode an
existing Fast 1.0 stream using a Fast 1.1 decoder. I believe that
this could be done relatively easily if the non-existent Template
Identifier field is skipped when encoding/decoding messages; but I
have run into the issue described above.
Rolf, you are describing Fast 1.1 compliance. I have a Fast 1.0 data source I would like to continue supporting, preferably using a Fast 1.1 code base with as little hacking as possible.
The question is really this:
Is there a way of using FAST 1.1 templates to describe an encoding scheme where the template for the second half of the message is determined by the value of a field in the first part of the message?
David,
pmap slot allocation is local to a specific template. each message
identifies which template to use in decoding.the following rules apply for a FAST encoded stream:
- each message begins with a pmap.
- the first field (following the pmap) is a tid (template id).
- the tid uses copy coding.
- the first slot in the message level pmap is used for the tid.
otherwise the format is not FAST compliant.
/Rolf
Rolf, this was a custom feed provided by one of our vendors.
David, which feed are you referring to? /Rolf
In Fast 1.0, it was possible to assign two different fields (say,
BID_PRICE and TRADE_PRICE) to the same PMAP slot, but different
templates, with the particular template chosen at encode/decode
time. So, a user could decode the first field
(QUOTE_TRADE_INDICATOR, for example), and use its value to
determine how a later field in the same message should be treated
(continuing this example, as a BID_PRICE or a TRADE_PRICE).How would you structure this in Fast 1.1 using the templates? It
seems that you would need a template to become active during
encoding or decoding, based on the VALUE of an already
encoded/decoded field. This is different from a dynamic template,
where the identifier of the template to be activated is explicitly
coded in the stream.The motivation behind this, by the way, is to try to decode an
existing Fast 1.0 stream using a Fast 1.1 decoder. I believe that
this could be done relatively easily if the non-existent Template
Identifier field is skipped when encoding/decoding messages; but I
have run into the issue described above.
Well, the 1.0 specs were incomplete and some of the implementors at the time communicated with the mdowg to get clarifications. One of the issues discussed repeatedly was the existence, placement and operator for a template identifier. So, yes, only reading the 1.0 specs you can argue that you’re still 1.0 compliant without a tid. I only know of one 1.0 impl and that’s arca. They use a tid in way that is compatible with 1.1. Seems the implementor of your source didn’t check or decided not to use a tid for some reason.
Best,
Rolf
Rolf, you are describing Fast 1.1 compliance. I have a Fast 1.0 data
source I would like to continue supporting, preferably using a Fast 1.1
code base with as little hacking as possible.The question is really this:
Is there a way of using FAST 1.1 templates to describe an encoding
scheme where the template for the second half of the message is
determined by the value of a field in the first part of the message?David,
pmap slot allocation is local to a specific template. each message
identifies which template to use in decoding.the following rules apply for a FAST encoded stream:
- each message begins with a pmap.
- the first field (following the pmap) is a tid (template id).
- the tid uses copy coding.
- the first slot in the message level pmap is used for the tid.
otherwise the format is not FAST compliant.
/Rolf
Rolf, this was a custom feed provided by one of our vendors.
David, which feed are you referring to? /Rolf
In Fast 1.0, it was possible to assign two different fields
(say, BID_PRICE and TRADE_PRICE) to the same PMAP slot, but
different templates, with the particular template chosen at
encode/decode time. So, a user could decode the first field
(QUOTE_TRADE_INDICATOR, for example), and use its value to
determine how a later field in the same message should be
treated (continuing this example, as a BID_PRICE or a
TRADE_PRICE).How would you structure this in Fast 1.1 using the templates? It
seems that you would need a template to become active during
encoding or decoding, based on the VALUE of an already
encoded/decoded field. This is different from a dynamic
template, where the identifier of the template to be activated
is explicitly coded in the stream.The motivation behind this, by the way, is to try to decode an
existing Fast 1.0 stream using a Fast 1.1 decoder. I believe
that this could be done relatively easily if the non-existent
Template Identifier field is skipped when encoding/decoding
messages; but I have run into the issue described above.
I seem to remember that your company did a deal with Spryware some time ago. If this Q is related to the FASTOR I suggest you discuss with Dan.
/Rolf
Rolf, you are describing Fast 1.1 compliance. I have a Fast 1.0 data
source I would like to continue supporting, preferably using a Fast 1.1
code base with as little hacking as possible.The question is really this:
Is there a way of using FAST 1.1 templates to describe an encoding
scheme where the template for the second half of the message is
determined by the value of a field in the first part of the message?
Hi David,
No, the only “field” that has this power is the template identifier. Once you’re decoding a given template, the number of logical fields cannot change.
Is the field that indicates the structure the first field of the message?
If so, it could be possible to pretend it actually is the TID and have the real value of that field constant coded in the corresponding template. But I’ve too little information to say if it would fly in your particular case.
/David
The question is really this:
Is there a way of using FAST 1.1 templates to describe an encoding
scheme where the template for the second half of the message is
determined by the value of a field in the first part of the message?
Hi David –
Thanks for great idea! In this case the message identifier is in fact the first field, so your trick should work nicely.
But in general, this seemed like a nice thing you could do with 1.0 (albeit only through coding); it’s too bad it didn’t make it somehow into the template definitions for 1.1.
Perhaps while we are talking about enhancements to the standard, we might consider some sort of ability to map a field to a template identifer?
Hi David,
No, the only “field” that has this power is the template identifier.
Once you’re decoding a given template, the number of logical fields
cannot change.Is the field that indicates the structure the first field of the
message?If so, it could be possible to pretend it actually is the TID and have
the real value of that field constant coded in the corresponding
template. But I’ve too little information to say if it would fly in your
particular case./David
The question is really this:
Is there a way of using FAST 1.1 templates to describe an encoding
scheme where the template for the second half of the message is
determined by the value of a field in the first part of the message?
David,
Are you talking about FASTOR ? Or OPRA ? Opra uses the scheme where the second half of the message is defined by a field in the message itself, but FASTOR does not do this. Each FASTOR message has one of 4 possible template ID’s, ‘T’, ‘Q’, ‘B’, ‘A’. The FAST template for each of these messages is unique. The encoding scope however for some of the fields may be either global or local to the tid. When implementing this using the fastapi 1.0, it was a bit misleading if you try to read the definition from the source code. To implement global and local encoding, the code reads as if you need to branch on fields within a messages to decode later fields. That is not the case, it is only needed to support the scope of the encoding. As far as I know, the FASTOR template definitions are 1.1 compliant.
I do believe at one point Jacob Northy put together a set of FAST 1.1 xml templates to define the FASTOR schema, and was able to decode the data with his “off the shelf” decoder. Let me see if I can find those.
/Daniel
Hi David –
Thanks for great idea! In this case the message identifier is in fact
the first field, so your trick should work nicely.But in general, this seemed like a nice thing you could do with 1.0
(albeit only through coding); it’s too bad it didn’t make it somehow
into the template definitions for 1.1.Perhaps while we are talking about enhancements to the standard, we
might consider some sort of ability to map a field to a template
identifer?Hi David,
No, the only “field” that has this power is the template identifier.
Once you’re decoding a given template, the number of logical fields
cannot change.Is the field that indicates the structure the first field of the
message?If so, it could be possible to pretend it actually is the TID and have
the real value of that field constant coded in the corresponding
template. But I’ve too little information to say if it would fly in
your particular case./David
The question is really this:
Is there a way of using FAST 1.1 templates to describe an encoding
scheme where the template for the second half of the message is
determined by the value of a field in the first part of the message?
Rolf,
I do not recall that it was mandatory to use copy encoding on the TID. I know I used an encoding of “NONE” in the ARCA implementation for the tid (which was called MSG_TYPE in those days). Do you recall why or when copy encoding on the tid was made mandatory ? I recall ARCA liked the idea if having a predictable MSG_TYPE.
As far as David Hait, I think I see the confusion between what he is saying, at what actually is going on. If he is referring to our FASTOR templates, they are not “self describing” as he says, but the encoder and decoder sample code based on fastapi 1.0 may give that impression.
Daniel
David,
pmap slot allocation is local to a specific template. each message
identifies which template to use in decoding.the following rules apply for a FAST encoded stream:
- each message begins with a pmap.
- the first field (following the pmap) is a tid (template id).
- the tid uses copy coding.
- the first slot in the message level pmap is used for the tid.
otherwise the format is not FAST compliant.
/Rolf
Dan -
I’m not sure I can answer when, but hopefully why: the decoder must know that the TID uses a copy operator before it tries to read it. So it cannot be a property of the template itself and must therefore be mandatory. However, an encoder is free to not utilize the operator and always set the first bit in the pmap and include the TID in every message.
/David
Rolf, I do not recall that it was mandatory to use copy encoding on the
TID. I know I used an encoding of “NONE” in the ARCA implementation for
the tid (which was called MSG_TYPE in those days). Do you recall why or
when copy encoding on the tid was made mandatory ? I recall ARCA liked
the idea if having a predictable MSG_TYPE.As far as David Hait, I think I see the confusion between what he is
saying, at what actually is going on. If he is referring to our FASTOR
templates, they are not “self describing” as he says, but the
encoder and decoder sample code based on fastapi 1.0 may give that
impression.Daniel
David,
pmap slot allocation is local to a specific template. each message
identifies which template to use in decoding.the following rules apply for a FAST encoded stream:
- each message begins with a pmap.
- the first field (following the pmap) is a tid (template id).
- the tid uses copy coding.
- the first slot in the message level pmap is used for the tid.
otherwise the format is not FAST compliant.
/Rolf
IIRC, the TID in the ARCA feed had a pmap bit allocated although it was specified to have the operator NONE.
Dan -
I’m not sure I can answer when, but hopefully why: the decoder must
know that the TID uses a copy operator before it tries to read it.
So it cannot be a property of the template itself and must therefore
be mandatory. However, an encoder is free to not utilize the
operator and always set the first bit in the pmap and include the
TID in every message./David
Rolf, I do not recall that it was mandatory to use copy encoding on
the TID. I know I used an encoding of “NONE” in the ARCA
implementation for the tid (which was called MSG_TYPE in those days).
Do you recall why or when copy encoding on the tid was made mandatory
? I recall ARCA liked the idea if having a predictable MSG_TYPE.As far as David Hait, I think I see the confusion between what he is
saying, at what actually is going on. If he is referring to our FASTOR
templates, they are not “self describing” as he says, but the
encoder and decoder sample code based on fastapi 1.0 may give that
impression.Daniel
David,
pmap slot allocation is local to a specific template. each message
identifies which template to use in decoding.the following rules apply for a FAST encoded stream:
- each message begins with a pmap.
- the first field (following the pmap) is a tid (template id).
- the tid uses copy coding.
- the first slot in the message level pmap is used for the tid.
otherwise the format is not FAST compliant.
/Rolf