FAST Extension proposal - BITGROUP

, ,
1

Imported from previous forum

2

All,

this is the last of four proposals announced in a previous post.
Please review and comment.
/Rolf

EXTENSION PROPOSAL - BITGROUP (packed field)

Background

There are several examples of real FAST feeds where it would
be possible to gain compaction efficiency if FAST would allow
several fields to be packed into one or more SBIT-encoded
bytes.

Introduction

A BITGROUP packs two or more fields into one SBIT-encoded
field. Each of the bit fields will:

  • have a fixed size (this is a requirement)
  • normally be less than 8 bits in size (not a requirement).

Example

For example, one live feed uses tags 279, 269, 461 with
limited value domains for each of the fields.

279 UpdateAction takes one of three values (requiring 2 bits)
269 EntryType takes one of two values (requiring 1 bit)
461 CFICode takes one of two values (requiring 1 bit)

As a consequence, all three field could be encoded into one
byte if several fields of one or more bits could be grouped
together.

(As a further extension bitgroups could support embedded enums
but this isn’t proposed here)

Wire representation

No new representation needed.
SBIT-encoded fields will be used.

Template Syntax

3

Is it possible or even beneficial to combine the Bitgroup, Set and Enum extensions? If we are striving for bit-level compression it might make sense to allow our bit-level operators to work together. For a NewOrderSingle message the following enumerations are used: Side, ExecInst, OrdType, TimeInForce. An example template definition might be:

<template name="NewOrderSingle" id="80">
…
<bitGroup>
<enum name="Side">
<value>1</value>
<value>2</value>
</enum>
<enum name="TimeInForce">
<value>0</value>
<value>1</value>
<value>2</value>
<value>3</value>
<value>4</value>
<value>5</value>
<value>6</value>
<value>7</value>
</enum>
<set name="ExecInst">
<value>G</value>
<option presence="mandatory">
<value>J</value>
<value>K</value>
</option>
</set>
<enum name="OrdType">
<value>1</value>
<value>2</value>
<value>3</value>
<value>4</value>
</enum>
</bitGroup>
…
</template>

The above bitGroup would require a single byte to represent 4 fields. The byte 10010101 would mean: Side: Sell, OrdType: Market, TimeInForce: GoodTillCancel, ExecInst: CancelOnTradingHalt.

4

Couple of questions:

  • syntax: you use the attribute “value” as opposed to “element”, any reason?
  • bits needed: I thought sets need as many bits as elements, i.e. ExecInst in your example would need 3 bits instead of 2. I understand the purpose of having either J or K. Where is the syntax for “option” defined? I guess this is a single element and thus only 2 bits are needed for ExecInst.

In general I think it is beneficial and maybe even necessary to combine the three extensions as shown. Rolf’s initial example of UpdateAction, EntryType and CFICode is now not 100% clear to me. Even if you only use two values for EntryType, they need to be defined in the template as they are a subset of all possible values. A FIX boolean field will not need additional information in the template.

Regards,
Hanno.

Is it possible or even beneficial to combine the Bitgroup, Set and Enum
extensions? If we are striving for bit-level compression it might make
sense to allow our bit-level operators to work together. For a
NewOrderSingle message the following enumerations are used: Side,
ExecInst, OrdType, TimeInForce. An example template definition might be:

… 1 2 0 1 2
3 4 5 6
7 G J K
1 2 3
4 …

The above bitGroup would require a single byte to represent 4 fields.
The byte 10010101 would mean: Side: Sell, OrdType: Market, TimeInForce:
GoodTillCancel, ExecInst: CancelOnTradingHalt.

5
  • syntax: you use the attribute “value” as opposed to “element”,
    any reason?

This is just a matter of choice. The syntax can be worked out by the MDOWG once the extension proposals are agreed to.

  • bits needed: I thought sets need as many bits as elements, i.e.
    ExecInst in your example would need 3 bits instead of 2. I understand
    the purpose of having either J or K. Where is the syntax for “option”
    defined? I guess this is a single element and thus only 2 bits are
    needed for ExecInst.

I have added comments to the Set extension proposal about the option group of mutually exclusive values. In the example here, ExecInst must have J or K and thus only a single bit is needed. The value 0 represents J. The value 1 represents K. Since these values are in an option only one value can be present for a given message. Since the option is mandatory one of the values must be present every time.

6

The BitGroup could act similar to the existing Group instruction in terms of optional presence. If the BitGroup is mandatory then no pmap bit is used. If it is optional a single pmap bit indicates whether the BitGroup is encoded in the stream.

Similarly each field in the BitGroup could have optional presence. Since the BitGroup does not have a presence map, a null value for an enum field is encoded as 0.