Proposal For New FAST Operator Default-Delta

, ,
1

Imported from previous forum

2

[ original email was from Clive Browning - clive@rapidaddition.com ]

We are seeing the following “general” conclusions for FAST field operators:

Defaults, Increments, Copys and Deltas (when sequence is always increasing) all work well for numerics.

Defaults and Copys are also good for strings, but Deltas do not work so well in practice…

e.g.

        AAAA (transmit AAAA)
        AAAAB (transmit B)
        AAAABC (transmit C)
        AAAABC (transmit nothing)
        AAAABCD (transmit D)

Above does work fine for string delta operator

But…

        AAAA
        AAAAB
        AAAABC
        AA

Does not work as there is currently no way to “reset” to a new starting delta value.

What might be useful is a Default-Delta operator – e.g. just pass the difference from a default value …

Default = AAAA

                    AAAA (transmit nothing)
                    AAAAB (transmit B)
                    AAAABB (transmit BB)
                    AAAAC (transmit C)

This might also be useful for numerics, e.g. default = 225

                    227 (transmit 2)
                    229 (transmit 4)
                    226 (transmit 1)
                    230 (transmit 5)

Is it worth investigating this for the next FAST release?

Clive Browning
Rapid Addition Ltd

3

[ original email was from Clive Browning - clive@rapidaddition.com ]

Sorry to reply to my own message…

Obviously the default-delta operator for numerics would only come in to its own for “larger” numbers using more than 1 byte of data! My example would have resulted in 1 byte being transmitted either way!

Clive

We are seeing the following “general” conclusions for FAST field
operators:

Defaults, Increments, Copys and Deltas (when sequence is always
increasing) all work well for numerics.

Defaults and Copys are also good for strings, but Deltas do not work so
well in practice…

e.g.

        AAAA (transmit AAAA) AAAAB (transmit B) AAAABC (transmit C)
        AAAABC (transmit nothing) AAAABCD (transmit D)

Above does work fine for string delta operator

But…

        AAAA AAAAB AAAABC AA

Does not work as there is currently no way to “reset” to a new starting
delta value.

What might be useful is a Default-Delta operator – e.g. just pass the
difference from a default value …

Default = AAAA

                    AAAA (transmit nothing) AAAAB (transmit B)
                    AAAABB (transmit BB) AAAAC (transmit C)

This might also be useful for numerics, e.g. default = 225

                    227 (transmit 2) 229 (transmit 4) 226 (transmit
                    1) 230 (transmit 5)

Is it worth investigating this for the next FAST release?

Clive Browning Rapid Addition Ltd

4

[ original email was from Wei Koek - Wei.Koek@nyfix.com ]
Clive,

Correct me if I am wrong, according to the FE spec section 2.1.6, shouldn’t the example you use look more like the following:

AAAA (transmit AAAA)
AAAB (transmit B)
AAAC (transmit C)
AAAC (transmit nothing)
AAAD (transmit D)

Would sending a NULL value works as resetting? I seem to remember having such conversation with Rolf and Matt a while back…

wk

We are seeing the following “general” conclusions for FAST field
operators:

Defaults, Increments, Copys and Deltas (when sequence is always
increasing) all work well for numerics.

Defaults and Copys are also good for strings, but Deltas do not work so
well in practice…

e.g.

        AAAA (transmit AAAA) AAAAB (transmit B) AAAABC (transmit C)
        AAAABC (transmit nothing) AAAABCD (transmit D)

Above does work fine for string delta operator

But…

        AAAA AAAAB AAAABC AA

Does not work as there is currently no way to “reset” to a new starting
delta value.

What might be useful is a Default-Delta operator – e.g. just pass the
difference from a default value …

Default = AAAA

                    AAAA (transmit nothing) AAAAB (transmit B)
                    AAAABB (transmit BB) AAAAC (transmit C)

This might also be useful for numerics, e.g. default = 225

                    227 (transmit 2) 229 (transmit 4) 226 (transmit
                    1) 230 (transmit 5)

Is it worth investigating this for the next FAST release?

Clive Browning Rapid Addition Ltd

5

[ original email was from Matt Simpson - msimpson@cme.com ]
Clive/Wei -

Delta Value Coding for strings assumes a fixed length field and that all positions in that field are used. The example provided by Wei is actually the correct interpretation of the spec while what Clive shows would be an extension.

The FastReset message (See SCP v099) can be used to reset state for a Delta and all other field encoding operators. Nulls cannot be used for this.

Clive’s use of Delta is an interesting one and could be part of a future extension if we see data patterns that would benefit from this. A default value is something that could be integrated as well.

Matt

Clive,

Correct me if I am wrong, according to the FE spec section 2.1.6,
shouldn’t the example you use look more like the following:

AAAA (transmit AAAA) AAAB (transmit B) AAAC (transmit C) AAAC (transmit
nothing) AAAD (transmit D)

Would sending a NULL value works as resetting? I seem to remember having
such conversation with Rolf and Matt a while back…

wk

We are seeing the following “general” conclusions for FAST field
operators:

Defaults, Increments, Copys and Deltas (when sequence is always
increasing) all work well for numerics.

Defaults and Copys are also good for strings, but Deltas do not work
so well in practice…

e.g.

        AAAA (transmit AAAA) AAAAB (transmit B) AAAABC (transmit
        C) AAAABC (transmit nothing) AAAABCD (transmit D)

Above does work fine for string delta operator

But…

        AAAA AAAAB AAAABC AA

Does not work as there is currently no way to “reset” to a new
starting delta value.

What might be useful is a Default-Delta operator – e.g. just pass the
difference from a default value …

Default = AAAA

                    AAAA (transmit nothing) AAAAB (transmit B)
                    AAAABB (transmit BB) AAAAC (transmit C)

This might also be useful for numerics, e.g. default = 225

                    227 (transmit 2) 229 (transmit 4) 226
                    (transmit
                    1) 230 (transmit 5)

Is it worth investigating this for the next FAST release?

Clive Browning Rapid Addition Ltd

6

[ original email was from David Rosenborg - David.Rosenborg@pantor.com ]
The delta operator propsed in TD is more generic than the FE one and can handle the case

AAAA
AA

The generalized delta operator has a signed length preamble indicating how many characters to remove from the back (or front if negative). The preamble is followed by the string to append (or prepend). So the above case would be encoded as

AAAA (transmit 0AAAA)
AA (transmit 2e, where e is the empty string)

In addition, the delta operator of TD lets you specify an initial value
so that even the first occurence of the field can be delta coded. This is obviously only beneficial if the first value is predictable.
The operator would render the following sequence

AAAA (transmit 0e)
AAAAB (transmit 0B)
AAAABC (transmit 0C)
AA (transmit 4e)

And the initial value works for numbers too, assuming :

227 (transmit 2)
229 (transmit 2)
226 (transmit -3)
230 (transmit 4)

/David

        AAAA (transmit AAAA) AAAAB (transmit B) AAAABC (transmit C)
        AAAABC (transmit nothing) AAAABCD (transmit D)

Above does work fine for string delta operator

But…

        AAAA AAAAB AAAABC AA

Does not work as there is currently no way to “reset” to a new starting
delta value.

What might be useful is a Default-Delta operator – e.g. just pass the
difference from a default value …

Default = AAAA

                    AAAA (transmit nothing) AAAAB (transmit B)
                    AAAABB (transmit BB) AAAAC (transmit C)

This might also be useful for numerics, e.g. default = 225

                    227 (transmit 2) 229 (transmit 4) 226 (transmit
                    1) 230 (transmit 5)

Is it worth investigating this for the next FAST release?

Clive Browning Rapid Addition Ltd

7

[ original email was from Clive Browning - clive@rapidaddition.com ]
David

Thanks for the feedback. The new delta operator from the TD will be just what we need.

Clive

The delta operator propsed in TD is more generic than the FE one and can
handle the case

AAAA AA

The generalized delta operator has a signed length preamble indicating
how many characters to remove from the back (or front if negative). The
preamble is followed by the string to append (or prepend). So the above
case would be encoded as

AAAA (transmit 0AAAA) AA (transmit 2e, where e is the empty string)

In addition, the delta operator of TD lets you specify an initial value
so that even the first occurence of the field can be delta coded. This
is obviously only beneficial if the first value is predictable. The
operator would render the following sequence

AAAA (transmit 0e) AAAAB (transmit 0B) AAAABC (transmit 0C) AA
(transmit 4e)

And the initial value works for numbers too, assuming :

227 (transmit 2) 229 (transmit 2) 226 (transmit -3) 230 (transmit 4)

/David

        AAAA (transmit AAAA) AAAAB (transmit B) AAAABC (transmit
        C) AAAABC (transmit nothing) AAAABCD (transmit D)

Above does work fine for string delta operator

But…

        AAAA AAAAB AAAABC AA

Does not work as there is currently no way to “reset” to a new
starting delta value.

What might be useful is a Default-Delta operator – e.g. just pass the
difference from a default value …

Default = AAAA

                    AAAA (transmit nothing) AAAAB (transmit B)
                    AAAABB (transmit BB) AAAAC (transmit C)

This might also be useful for numerics, e.g. default = 225

                    227 (transmit 2) 229 (transmit 4) 226
                    (transmit
                    1) 230 (transmit 5)

Is it worth investigating this for the next FAST release?

Clive Browning Rapid Addition Ltd