Book Management

, ,
1

Imported from previous forum

2

[ original email was from Colin Cullen - colin.cullen@us.icap.com ]
Hi,

I have a question specifically regarding the MDWG’s “Recommended Practices for Book Management” (RPBM) 2.00.

We currently provide a single market data “view” via the Snapshot/Full Refresh (W) and Incremental Refresh-Update (X) messages. We are investigating expanding our Market Data messages to provide more Order Book information, with three (and possible more) “views” or book types (as in MDBookType, tag=1021). Several of these book types are similar the ones discussed in sections 5.2 - 5.4 of the RPBM: Top of Book, Price Depth, and Order Depth. The RPBM speaks to these book types as individual market data messages. We would like to consolidate multiple book types into a single market data message by creating a repeating group out of MDBookType, pushing the book type and its associated fields down a “level” in the message. We would define a new field, NoBookTypes to specify the count for the repeating group and then MDBookType would become the first field within the repeating group, the second field of the repeating group would be NoMDEntries, which starts the market data repeating group.

The reason for this modification to the recommended usage is to provide the client with all market data for a specific timeslice in a single atomic message.

FYI, we are the exchange in this scenario.

I would like to know people’s thoughts on this approach.

A more general question is: does FIX “allow” for this type of flexibility in its specification?

Has anyone seen a need for something similar?

Best Regards,
Colin Cullen
ICAP

3

The simple answer is “no” and I would like to explain it.

First, MDBookType does not allow user-defined enum values, i.e. you can only use TOB/PD/OD but none of your own that might be similar to the standard book types. You would have to use MDSubBookType (1173) for your own flavors of the three standard book types.

Secondly, and more importantly, you are planning to make a structural change to the message by moving standard fields into different levels of nesting and adding a nesting level. This is not possible if you want to stay within the standard.

You say your requirement is to have all data for one timeslice. It might be sufficient to have all of the information in a single UDP packet (if you use multicast), albeit in multiple FIX messages. Overhead should not be an issue, especially with FAST encoding being used.

MDBookType assumes that the information is generated by independent sources and just needs to be transported over the same connection. Your design has different types of data that seem to be compiled centrally and can thus be linked to a specific timeslice. I assume that all of the information pertains to a single instrument.

Note that the snapshot can only convey information about a single security as is at the root level of the message. Incremental updates can have multiple instruments in the MDEntry group.

Maybe you could use MDReportID (963) to emulate a single report split across multiple messages of type W or X. MsgSeqNum in the header would be different for all messages whereas MDReportID would be the same for groups of messages that were created by you in the same timeslice.

With the suggested structural change, your messages might get rather large, possibly having to split logical messages across UDP packets. It also does not allow users to quickly discard book types they are not interested in.

In short, I would solve the requirement to link market data of different book types by other means than to physically make everything part of a large, nested repeating group.

Regards,
Hanno.

Hi,

I have a question specifically regarding the MDWG’s “Recommended Practices for Book Management” (RPBM) 2.00.

We currently provide a single market data “view” via the Snapshot/Full Refresh (W) and Incremental Refresh-Update (X) messages. We are investigating expanding our Market Data messages to provide more Order Book information, with three (and possible more) “views” or book types (as in MDBookType, tag=1021). Several of these book types are similar the ones discussed in sections 5.2 - 5.4 of the RPBM: Top of Book, Price Depth, and Order Depth. The RPBM speaks to these book types as individual market data messages. We would like to consolidate multiple book types into a single market data message by creating a repeating group out of MDBookType, pushing the book type and its associated fields down a “level” in the message. We would define a new field, NoBookTypes to specify the count for the repeating group and then MDBookType would become the first field within the repeating group, the second field of the repeating group would be NoMDEntries, which starts the market data repeating group.

The reason for this modification to the recommended usage is to provide the client with all market data for a specific timeslice in a single atomic message.

FYI, we are the exchange in this scenario.

I would like to know people’s thoughts on this approach.

A more general question is: does FIX “allow” for this type of flexibility in its specification?

Has anyone seen a need for something similar?

Best Regards,
Colin Cullen
ICAP

4

[ original email was from Colin Cullen - colin.cullen@us.icap.com ]
Hanno,

Thanks for your detailed response. I have added my [comments] for the sake of other readers.

The simple answer is “no” and I would like to explain it.

First, MDBookType does not allow user-defined enum values, i.e. you can only use TOB/PD/OD but none of your own that might be similar to the standard book types. You would have to use MDSubBookType (1173) for your own flavors of the three standard book types.

[I thought that the protocol allowed for adding user defined enums to a standard enum field - I will need to research this more.]

Secondly, and more importantly, you are planning to make a structural change to the message by moving standard fields into different levels of nesting and adding a nesting level. This is not possible if you want to stay within the standard.

[That makes sense, I continue to question what flexibility is “allowed” by the standard messages, but would prefer to stay as close to the standard as possible.]

You say your requirement is to have all data for one timeslice. It might be sufficient to have all of the information in a single UDP packet (if you use multicast), albeit in multiple FIX messages. Overhead should not be an issue, especially with FAST encoding being used.

MDBookType assumes that the information is generated by independent sources and just needs to be transported over the same connection.

[good to know, our market data comes from a single source]

Your design has different types of data that seem to be compiled centrally and can thus be linked to a specific timeslice.

[The data is the same, it is the views of the data that are different. They are all market depth views (book types) that differ slightly from the offerings in the the MDWG’s “Recommended Practices for Book Management” (RPBM) 2.00 (and in one case we have a view that has no counterpart in the RPBM)]

I assume that all of the information pertains to a single instrument.

[per snapshot, yes]

Note that the snapshot can only convey information about a single security as is at the root level of the message. Incremental updates can have multiple instruments in the MDEntry group.

[yes, that is what we do]

Maybe you could use MDReportID (963) to emulate a single report split across multiple messages of type W or X. MsgSeqNum in the header would be different for all messages whereas MDReportID would be the same for groups of messages that were created by you in the same timeslice.

With the suggested structural change, your messages might get rather large, possibly having to split logical messages across UDP packets.

[It makes sense to split up the single logical into multiple physical messages]

It also does not allow users to quickly discard book types they are not interested in.

[the subscription will be by instrument and book type, so the client will only get what he is interested in]

In short, I would solve the requirement to link market data of different book types by other means than to physically make everything part of a large, nested repeating group.

[that makes sense, thanks]

Regards,
Colin

The simple answer is “no” and I would like to explain it.

First, MDBookType does not allow user-defined enum values, i.e. you can only use TOB/PD/OD but none of your own that might be similar to the standard book types. You would have to use MDSubBookType (1173) for your own flavors of the three standard book types.

Secondly, and more importantly, you are planning to make a structural change to the message by moving standard fields into different levels of nesting and adding a nesting level. This is not possible if you want to stay within the standard.

You say your requirement is to have all data for one timeslice. It might be sufficient to have all of the information in a single UDP packet (if you use multicast), albeit in multiple FIX messages. Overhead should not be an issue, especially with FAST encoding being used.

MDBookType assumes that the information is generated by independent sources and just needs to be transported over the same connection. Your design has different types of data that seem to be compiled centrally and can thus be linked to a specific timeslice. I assume that all of the information pertains to a single instrument.

Note that the snapshot can only convey information about a single security as is at the root level of the message. Incremental updates can have multiple instruments in the MDEntry group.

Maybe you could use MDReportID (963) to emulate a single report split across multiple messages of type W or X. MsgSeqNum in the header would be different for all messages whereas MDReportID would be the same for groups of messages that were created by you in the same timeslice.

With the suggested structural change, your messages might get rather large, possibly having to split logical messages across UDP packets. It also does not allow users to quickly discard book types they are not interested in.

In short, I would solve the requirement to link market data of different book types by other means than to physically make everything part of a large, nested repeating group.

Regards,
Hanno.

Hi,

I have a question specifically regarding the MDWG’s “Recommended Practices for Book Management” (RPBM) 2.00.

We currently provide a single market data “view” via the Snapshot/Full Refresh (W) and Incremental Refresh-Update (X) messages. We are investigating expanding our Market Data messages to provide more Order Book information, with three (and possible more) “views” or book types (as in MDBookType, tag=1021). Several of these book types are similar the ones discussed in sections 5.2 - 5.4 of the RPBM: Top of Book, Price Depth, and Order Depth. The RPBM speaks to these book types as individual market data messages. We would like to consolidate multiple book types into a single market data message by creating a repeating group out of MDBookType, pushing the book type and its associated fields down a “level” in the message. We would define a new field, NoBookTypes to specify the count for the repeating group and then MDBookType would become the first field within the repeating group, the second field of the repeating group would be NoMDEntries, which starts the market data repeating group.

The reason for this modification to the recommended usage is to provide the client with all market data for a specific timeslice in a single atomic message.

FYI, we are the exchange in this scenario.

I would like to know people’s thoughts on this approach.

A more general question is: does FIX “allow” for this type of flexibility in its specification?

Has anyone seen a need for something similar?

Best Regards,
Colin Cullen
ICAP

5

[I thought that the protocol allowed for adding user defined enums to a standard enum field - I will need to research this more.]

I think of this in three categories:

  1. Fields where the FIX Protocol explicitly states that they can be extended. Doing so is fully compliant. Two sub-cateogries:

1A. Integer fields that are represented in the Repository with a union datatype. E.g. OrdRejReason(103) has the union datatype Reserved100Plus, which means users can, with bilateral agreement, add enums starting at 100. The data dictionary in Volume 6 of the spec shows this by appending the text: “or any value conforming to the data type Reserved100Plus” at the end of the list of enums. Unfortunately, I don’t think FIXimate 3.0 on the website shows this.

1B. String fields that spell out in their descriptions how they should be extended. E.g. SecurityIDSource(22) is a String; it says in its description: “100+ are reserved for private security identifications”. And MsgType(35) says: “Note: A “U” as the first character in the MsgType field (i.e. U, U2, etc) indicates that the message format is privately defined between the sender and receiver.”

  1. Taking standard enums from later versions and pulling them into a previous version. This is usually considered an acceptable practice when there is a need, and when doing so isn’t duplicating existing functionality. E.g. ExecInst = f (Intermarket Sweep) was added to support Reg NMS in the US and was intended to support similar functions in other markets as requirements emerge (e.g. DAO orders in Canada.) I see nothing wrong with using ExecInst=f in FIX 4.2, even though it technically isn’t part of 4.2. But this doesn’t apply when functionality does exist. E.g. FIX 4.2 uses ExecType = 1 (Partial Fill) and 2 (Fill). FIX 4.4 uses ExecType = F (Trade). I would consider use of ExecType=F in FIX 4.2 a mistake.

  2. Firms add their own enums to standard FIX fields that don’t support user extensions all the time. This isn’t compliant. But it happens anyway. Two things to consider:

  • Counterparty impact. Generally, adding new data (e.g. a new MDEntryType) has less impact than adding new state (e.g. a new OrdStatus/ExecType.)

  • The value you choose may be unique now, but may cease being unique in the future. If a field has 0-9, A-Z, and a-c assigned today, taking d isn’t a good idea, because that’s the next value FPL will likely assign. Taking z, y, and x doesn’t guarantee uniqueness, but it is less likely to result in a collision for some time.