Imported from previous forum
[ original email was from Ryan Pierce - rpierce@taltrade.com ]
I’ve tried to produce a very rough outline of book functionality, leaning more towards abstract terms rather than getting into actual mappings to FIX messages. Given that different ECNs have different policies concerning how much of their book they are willing to display, the protocol should accomodate the different modes.
I’ve also noticed that these kinds of messages could, in addition to allowing ECNs to communicate their book, optionally be used to provide real-time data (i.e. NASDAQ Level II data, or regional data, as well as simple time and sales / tickers) to trading terminals, and possibly historical data as well. Whether anyone wishes to use FIX in this way, or whether this working group wishes to address this issue, is another thing entirely. I’ve noted a few simple additions to these messages which would facilitate this.
Existing FIX Quote and IOI messages seem unsuitable for implementing books without extensive changes, and the changes required to make them useable could cause incompatibilities with existing implementations. Thus it might be advantageous to create new message types for handling book queries and real-time book feeds.
Book Types / Attributes:
- Top Of File - only shows best bid, offer, and sizes.
- Full Book - can show multiple entries per side, but only one per price.
- Complete - shows full market depth.
- Truncated - shows just first N price tiers.
- Trades Only - just show completed trades.
A user should be able to request any of these book attributes. The ECN may choose to support only a subset of these due to technical limitations or policies, and must have the ability [to reject the request.] I am wondering whether any ECN would care to announce individual orders per side per price tier, and if it warrants inclusion in the spec as separate options, i.e. Aggregated vs. Unaggregated.
The Trades Only mode I suggest might be duplicating the purpose of the Advertisement message, but that message doesn’t seem to have any way of selectively turning on and off what advertisements you want to receive, or requesting historical trades.
Possible Book Queries:
- Single Symbol
- List of Symbols
- All Symbols
- All Symbols for a list of exchanges
- All Symbols in an index (i.e. all DJIA stocks, or all S&P500 stocks)
Book Request Modes:
- Request Snapshot for given query
- Request Snapshot + Turn On Updates for given query
- Turn Off Updates for given query
- Request historical data for given query
- Request historical data + updates for a given query
Book Entry Attributes:
- Type = Bid, Offer, Trade
- Volume
- Price
- Text Tag
- Time stamp
- Primary Market Maker (Y/N)
For ECN books, only the Type (Bid/Offer), Volume, and Price would be required. Timestamps could be optional.
For using this type of message to display regional / NQDS data, a text tag could indicate the Market Maker ID, and possibly whether the person in question was a primary market maker for that security. (Would anyone need this info? Should it be sent only once in the snapshot and the client would cache it? Would anyone also need an uptick / downtick indicator?) The Trade type would be used only for Trades Only updates, to provide ticker / time & sales data.
Book Update Functionality:
I would imagine that unlike an IOI, a book update would remain valid up until the time it is somehow refreshed, changed, or removed.
Implementing Top Of File book broadcasts should be relatively straightforward. But full books pose a different problem. I see two approaches:
Each time the book changes, broadcast the full book for a given instrument. The advantage here is it is simple; you get an update for MSFT, throw out your old MSFT book, and replace it with the new book. The disadvantage is that it is extremely wasteful of bandwidth.
The other method is to give each book entry an ID. Then you can correct the price, volume, and possibly timestamp for the ID, or you can drop the entry entirely. This method would require a means of dropping all quotes for a given instrument, or dropping all quotes for all instruments.
I’m guessing the choice of update strategy should be up to the client, subject to the server’s approval.
Ryan Pierce
Townsend Analytics Ltd. / Archipelago LLC
[ original email was from Ryan Pierce - rpierce@taltrade.com ]
Sorry to those who received the last message twice; it was recommended that I put it in a separate thread.
So I’d like to throw out a few questions to everyone.
Full Book vs. Top Of Book
Top of Book could be implemented with the functionality that I’m suggesting by making some small changes and additions, mostly to the Quote Request message to allow for requesting asynchronous updates.
To do Full Book in a bandwidth-friendly way would best be handled by a new message type. (Neither IOI nor Quote would be well-suited without doing a major overhaul of those messages.)
So should we:
A. Extend Quote / Quote Request for top of book data only, and develop a separate set of book-related messages for showing market depth, or
B. Leave Quote / Quote Request alone, and do both Top of Book and Full Book via a new set of book-related messages.
I can see advantages and disadvantages to both; what do you all think would be best?
Data Feed Redistribution
If we’re creating messages to display an ECN’s book, it’s only a small hop to extend that to allow people to redistribute the NASDAQ book, regional quotes for listed stocks, foreign exchanges, etc. (Obviously it requires that appropriate exchange fees have been paid, proper agreements are in place, etc.) Would there be any interest in this?
Ryan Pierce
Townsend Analytics Ltd. / Archipelago LLC
[ original email was from johna@ms.com - ]
> To do Full Book in a bandwidth-friendly way would best be handled by a new message type. (Neither IOI nor Quote would be well-suited without doing a major overhaul of those messages.)
I’d like to explore using Quote/Quote Request since this is
really what we are talking about with depth of book, i.e a bunch of quotes. I think we could extend the messages to be
backward compatable.
>
> So should we:
>
> A. Extend Quote / Quote Request for top of book data only, and develop a separate set of book-related messages for showing market depth, or
>
> B. Leave Quote / Quote Request alone, and do both Top of Book and Full Book via a new set of book-related messages.
>
> I can see advantages and disadvantages to both; what do you all think would be best?
>
> Data Feed Redistribution
>
> If we’re creating messages to display an ECN’s book, it’s only a small hop to extend that to allow people to redistribute the NASDAQ book, regional quotes for listed stocks, foreign exchanges, etc. (Obviously it requires that appropriate exchange fees have been paid, proper agreements are in place, etc.) Would there be any interest in this?
>
> Ryan Pierce
> Townsend Analytics Ltd. / Archipelago LLC
>
>
[ original email was from Mike Gibb - Mike.Gibb@csfb.com ]
> So should we:
>
> A. Extend Quote / Quote Request for top of book data only, and develop a separate set of book-related messages for showing market depth, or
>
> B. Leave Quote / Quote Request alone, and do both Top of Book and Full Book via a new set of book-related messages.
You could reverse the question: Given that at least one new message will have to be created for subscription-based, asynchronous market information data, could/should a general purpose MarketInfo message (that can hold Top of Book information as well as Market Depth) be used?
Mike.Gibb@CSFB.com (London)
[ original email was from Ryan Pierce - rpierce@taltrade.com ]
> > So should we:
> >
> > A. Extend Quote / Quote Request for top of book data only, and develop a separate set of book-related messages for showing market depth, or
> >
> > B. Leave Quote / Quote Request alone, and do both Top of Book and Full Book via a new set of book-related messages.
>
> You could reverse the question: Given that at least one new message will have to be created for subscription-based, asynchronous market information data, could/should a general purpose MarketInfo message (that can hold Top of Book information as well as Market Depth) be used?
I think I’m leaning more towards John Armstrong’s suggestion - that all this can be accomplished within the Quote / Quote Request. Assuming we can add a new mode to Quote Request and Quote to allow for depth of book, and the Quote message can be used to express a quote request reject reason (much like Execution can convey that an order was rejected), then we don’t have to add any new message types.
[ original email was from John Lopez - john.e.lopez@usa.net ]
For consolidated feeds, Quotes and quote request need a contributor id field. The contributor specifies the source of each book entry within the quote (ECN, exchange, broker id, quote vendor, wildcard, etc).
Two basic use cases:
- Snapshot quotes: may extend the current spec to include book depth.
- Real-time pub/sub semantics.
I think the real-time pub/sub can’t send the full book for each update (if there is a >1 depth). There needs to be a stateful mechanism. I can see a quote request followed by full quote and new update (add) and update (delete). There will also need to be a mechanism to turn off updates.
I used to work at Microsoft on DNAfs for securities. Their STPML (XML securities messaging) is addressing the same problem and working with quote Vendors Bridge and Reuters. There may be value in collaborating with them. Contact Derek LaSalle derekla@microsoft.com.
John
> I’ve tried to produce a very rough outline of book functionality, leaning more towards abstract terms rather than getting into actual mappings to FIX messages. Given that different ECNs have different policies concerning how much of their book they are willing to display, the protocol should accomodate the different modes.
>
> I’ve also noticed that these kinds of messages could, in addition to allowing ECNs to communicate their book, optionally be used to provide real-time data (i.e. NASDAQ Level II data, or regional data, as well as simple time and sales / tickers) to trading terminals, and possibly historical data as well. Whether anyone wishes to use FIX in this way, or whether this working group wishes to address this issue, is another thing entirely. I’ve noted a few simple additions to these messages which would facilitate this.
>
> Existing FIX Quote and IOI messages seem unsuitable for implementing books without extensive changes, and the changes required to make them useable could cause incompatibilities with existing implementations. Thus it might be advantageous to create new message types for handling book queries and real-time book feeds.
>
> Book Types / Attributes:
>
> * Top Of File - only shows best bid, offer, and sizes.
> * Full Book - can show multiple entries per side, but only one per price.
> * Complete - shows full market depth.
> * Truncated - shows just first N price tiers.
> * Trades Only - just show completed trades.
>
> A user should be able to request any of these book attributes. The ECN may choose to support only a subset of these due to technical limitations or policies, and must have the ability [to reject the request.] I am wondering whether any ECN would care to announce individual orders per side per price tier, and if it warrants inclusion in the spec as separate options, i.e. Aggregated vs. Unaggregated.
>
> The Trades Only mode I suggest might be duplicating the purpose of the Advertisement message, but that message doesn’t seem to have any way of selectively turning on and off what advertisements you want to receive, or requesting historical trades.
>
> Possible Book Queries:
>
> * Single Symbol
> * List of Symbols
> * All Symbols
> * All Symbols for a list of exchanges
> * All Symbols in an index (i.e. all DJIA stocks, or all S&P500 stocks)
>
> Book Request Modes:
>
> * Request Snapshot for given query
> * Request Snapshot + Turn On Updates for given query
> * Turn Off Updates for given query
> * Request historical data for given query
> * Request historical data + updates for a given query
>
> Book Entry Attributes:
>
> * Type = Bid, Offer, Trade
> * Volume
> * Price
> * Text Tag
> * Time stamp
> * Primary Market Maker (Y/N)
>
> For ECN books, only the Type (Bid/Offer), Volume, and Price would be required. Timestamps could be optional.
>
> For using this type of message to display regional / NQDS data, a text tag could indicate the Market Maker ID, and possibly whether the person in question was a primary market maker for that security. (Would anyone need this info? Should it be sent only once in the snapshot and the client would cache it? Would anyone also need an uptick / downtick indicator?) The Trade type would be used only for Trades Only updates, to provide ticker / time & sales data.
>
> Book Update Functionality:
>
> I would imagine that unlike an IOI, a book update would remain valid up until the time it is somehow refreshed, changed, or removed.
>
> Implementing Top Of File book broadcasts should be relatively straightforward. But full books pose a different problem. I see two approaches:
>
> Each time the book changes, broadcast the full book for a given instrument. The advantage here is it is simple; you get an update for MSFT, throw out your old MSFT book, and replace it with the new book. The disadvantage is that it is extremely wasteful of bandwidth.
>
> The other method is to give each book entry an ID. Then you can correct the price, volume, and possibly timestamp for the ID, or you can drop the entry entirely. This method would require a means of dropping all quotes for a given instrument, or dropping all quotes for all instruments.
>
> I’m guessing the choice of update strategy should be up to the client, subject to the server’s approval.
>
> Ryan Pierce
> Townsend Analytics Ltd. / Archipelago LLC
>
[ original email was from Ryan Pierce - rpierce@taltrade.com ]
> For consolidated feeds, Quotes and quote request need a contributor id field. The contributor specifies the source of each book entry within the quote (ECN, exchange, broker id, quote vendor, wildcard, etc).
For addressing the needs of market data distribution, the Quote message probably isn’t the best choice. On the FIX web site, under Specifications / Drafts you will find a Book Messages draft document which is much more suited for distributing market data.
The Book Draft has undergone many changes within a subcommittee of the ECNs and Exchanges Working Group to make it much more comprehensive, and has been renamed to the Market Data draft to reflect its new uses.
The draft includes, among other things, the following fields to identify quotes in a consolidated feed:
MDMkt (Uses Appendix C values to designate the market)
MDEntryOriginator (Useful for markets like NASDAQ; this can represent the Market Maker posting a quote)
LocationID and DeskID (Again, useful in markets like NASDAQ to show MMGLs and MMDLs )
> Two basic use cases:
> 1. Snapshot quotes: may extend the current spec to include book depth.
> 2. Real-time pub/sub semantics.
>
> I think the real-time pub/sub can’t send the full book for each update (if there is a >1 depth). There needs to be a stateful mechanism. I can see a quote request followed by full quote and new update (add) and update (delete). There will also need to be a mechanism to turn off updates.
Again, these are all addressed in the draft I referenced. While a lot of things have changed in it, the basic concepts have mostly remained the same.
One of the additions to the Market Data draft is that unlike the Book draft, unsolicited Market Data messages are allowed, which makes the messages more useful for broadcasting / multicasting data feeds.
[ original email was from Ryan Pierce - rpierce@taltrade.com ]
> The Book Draft has undergone many changes within a subcommittee of the ECNs and Exchanges Working Group to make it much more comprehensive, and has been renamed to the Market Data draft to reflect its new uses.
This Market Data draft is now available for comment on the FIX web site under Specifications / Drafts.
[ original email was from John Lopez - john.e.lopez@usa.net ]
Is there a posted doc for publishing the book?
John
> I’ve tried to produce a very rough outline of book functionality, leaning more towards abstract terms rather than getting into actual mappings to FIX messages. Given that different ECNs have different policies concerning how much of their book they are willing to display, the protocol should accomodate the different modes.
>
> I’ve also noticed that these kinds of messages could, in addition to allowing ECNs to communicate their book, optionally be used to provide real-time data (i.e. NASDAQ Level II data, or regional data, as well as simple time and sales / tickers) to trading terminals, and possibly historical data as well. Whether anyone wishes to use FIX in this way, or whether this working group wishes to address this issue, is another thing entirely. I’ve noted a few simple additions to these messages which would facilitate this.
>
> Existing FIX Quote and IOI messages seem unsuitable for implementing books without extensive changes, and the changes required to make them useable could cause incompatibilities with existing implementations. Thus it might be advantageous to create new message types for handling book queries and real-time book feeds.
>
> Book Types / Attributes:
>
> * Top Of File - only shows best bid, offer, and sizes.
> * Full Book - can show multiple entries per side, but only one per price.
> * Complete - shows full market depth.
> * Truncated - shows just first N price tiers.
> * Trades Only - just show completed trades.
>
> A user should be able to request any of these book attributes. The ECN may choose to support only a subset of these due to technical limitations or policies, and must have the ability [to reject the request.] I am wondering whether any ECN would care to announce individual orders per side per price tier, and if it warrants inclusion in the spec as separate options, i.e. Aggregated vs. Unaggregated.
>
> The Trades Only mode I suggest might be duplicating the purpose of the Advertisement message, but that message doesn’t seem to have any way of selectively turning on and off what advertisements you want to receive, or requesting historical trades.
>
> Possible Book Queries:
>
> * Single Symbol
> * List of Symbols
> * All Symbols
> * All Symbols for a list of exchanges
> * All Symbols in an index (i.e. all DJIA stocks, or all S&P500 stocks)
>
> Book Request Modes:
>
> * Request Snapshot for given query
> * Request Snapshot + Turn On Updates for given query
> * Turn Off Updates for given query
> * Request historical data for given query
> * Request historical data + updates for a given query
>
> Book Entry Attributes:
>
> * Type = Bid, Offer, Trade
> * Volume
> * Price
> * Text Tag
> * Time stamp
> * Primary Market Maker (Y/N)
>
> For ECN books, only the Type (Bid/Offer), Volume, and Price would be required. Timestamps could be optional.
>
> For using this type of message to display regional / NQDS data, a text tag could indicate the Market Maker ID, and possibly whether the person in question was a primary market maker for that security. (Would anyone need this info? Should it be sent only once in the snapshot and the client would cache it? Would anyone also need an uptick / downtick indicator?) The Trade type would be used only for Trades Only updates, to provide ticker / time & sales data.
>
> Book Update Functionality:
>
> I would imagine that unlike an IOI, a book update would remain valid up until the time it is somehow refreshed, changed, or removed.
>
> Implementing Top Of File book broadcasts should be relatively straightforward. But full books pose a different problem. I see two approaches:
>
> Each time the book changes, broadcast the full book for a given instrument. The advantage here is it is simple; you get an update for MSFT, throw out your old MSFT book, and replace it with the new book. The disadvantage is that it is extremely wasteful of bandwidth.
>
> The other method is to give each book entry an ID. Then you can correct the price, volume, and possibly timestamp for the ID, or you can drop the entry entirely. This method would require a means of dropping all quotes for a given instrument, or dropping all quotes for all instruments.
>
> I’m guessing the choice of update strategy should be up to the client, subject to the server’s approval.
>
> Ryan Pierce
> Townsend Analytics Ltd. / Archipelago LLC
>
[ original email was from John Lopez - john.e.lopez@usa.net ]
Does the current proposal account for sharing of the order book (cross listing books) among execution systems in a peer-to-peer environment. I think a 2 phase commit for distributed matching across execution systems is needed.
(Logically not FIX specific)
I can see a use case where:
I as an ECN list by books with other ECNs. I own these orders and the other ECNs. Another ECN matches the order and sends me a match request; I must send acknowledge the match in ‘x’ seconds. No ACK and the match are broken. The ack, positive or negative (because it was internally traded or canceled) received by the remote party either rolls back the trade or sends a match confirmation (binding and done trade). Broken trades could either be handled externally (manually) or based on some automated rules.
[ original email was from Ryan Pierce - rpierce@taltrade.com ]
> Does the current proposal account for sharing of the order book (cross listing books) among execution systems in a peer-to-peer environment. I think a 2 phase commit for distributed matching across execution systems is needed.
>
> (Logically not FIX specific)
> I can see a use case where:
>
> I as an ECN list by books with other ECNs. I own these orders and the other ECNs. Another ECN matches the order and sends me a match request; I must send acknowledge the match in ‘x’ seconds. No ACK and the match are broken. The ack, positive or negative (because it was internally traded or canceled) received by the remote party either rolls back the trade or sends a match confirmation (binding and done trade). Broken trades could either be handled externally (manually) or based on some automated rules.
This "2 phase commit" already happens using IOC orders.
For instance, Customer 1 sends a buy order to ECN A. The order does not lock or cross the market, and is the best bid for ECN A, so ECN A advertises it to ECN B in a Market Data message. Customer 2 places a sell order to ECN B which happens to lock the market and matches the order on ECN A. ECN B then sends an IOC order to ECN A. If the match is still there, ECN A sends back an execution. If Customer 1 already cancelled the order by the time that ECN B’s order arrives, then ECN A will report that ECN B’s order is Cancelled.
IOC orders are what the ECNs/Exchanges committee has recommended as the way to take liquidity from an ECN’s book, and requires an immediate, or near-immediate, response. If the ECN has a matching order in its book, it trades it. If the ECN only has some of the shares to match, it partially fills the IOC and reports the rest as cancelled. If the ECN has no match, it reports the order as cancelled.
The FIX protocol doesn’t define any X second limit for IOCs or what should happen if an IOC order isn’t acknowledged in a timely fashion. One’s business department could negotiate that if ECN A sends ECN B an IOC order, and ECN B does not respond in X seconds, ECN A is free to assume that no match happened and ignore any late trades, or alternately, that a match did happen.
[ original email was from John Lopez - john.e.lopez@usa.net ]
Thanks Ryan for your comment.
Again on 2 phase commit with 3 party trades:
If I was to do aggregated trades that matched several orders (external) to get an average price and quantity to satisfy an internal order. Is it possible to do this with the IOC?
I hit two orders advertised from two different ECNs to satisfy my own unpublished limit order with AON. Can I back out of an early leg if the later leg refuses the IOC?
[ original email was from Ryan Pierce - rpierce@taltrade.com ]
> Thanks Ryan for your comment.
>
> Again on 2 phase commit with 3 party trades:
>
> If I was to do aggregated trades that matched several orders (external) to get an average price and quantity to satisfy an internal order. Is it possible to do this with the IOC?
I’m not sure I understand what you are saying, and I think I need to take back what I said about a 2-phase commit.
I’m speaking from a perspective of the US OTC marketplace and ECNs. What follows is a gross simplification of business practices.
When an ECN posts a bid or offer to SelectNet or sends a Market Data message adding an entry to its book, the ECN is saying that it has liquidity and is willing to do a trade at that price.
If you send a SelectNet preference to an ECN, or send an IOC order which is effectively the FIX equivalent, you are asking to hit that bid or offer. The ECN is bound by the firm quote rule; it can’t just arbitrarily decide to back away from its quote when it sees your order. But sending an order/preference does NOT guarantee you a trade. Someone else may have traded with the ECN before you, or the user may have cancelled the order before the ECN received your FIX IOC order or SelectNet preference.
Once you send that SelectNet preference or send an IOC order, you are, in general, obligated to accept a trade if the ECN makes a match. Yes, it is possible to try to withdraw a preference or cancel the order, but it may not be feasible to do so before a trade happens.
Thus, if you have an order to buy 2000 shares at 10, and you see ECN A with an offer of 1000 at 9 15/16 and ECN B with an offer at 10 1/16, sending an IOC order to both may result in you getting filled at 10 1/16 and declined at 9 15/16, forcing you to eat 1/16 on 1000.
> I hit two orders advertised from two different ECNs to satisfy my own unpublished limit order with AON. Can I back out of an early leg if the later leg refuses the IOC?
This is a business issue which would be determined between the customer and the ECN, but my guess is that you’d be very hard-pressed to find an ECN that would allow you to break a trade for that reason. The issue is that an ECN is a limit order matching engine; it risks no capital and assumes no positions in stocks it trades. As there is always a customer order on the other side of your order that matched, and the ECN really cannot bust the other customer’s side of the trade, the only way to allow you to back out of a trade would be if the ECN assumed your side of the trade, possibly at a large financial risk.