FIX Latest + Simple Binary Encoding (SBE) / JSON + FIXS + FIXP standards (crypto exchange experience)

Hi Colleagues,
please kindly assist, I have some questions to clarify how to implement messages sequencing in FIXP standard:

  1. What procedure of messages resetting? I didn’t understand how to do that using Sequence message…
  2. If client has been disconnected for a long time - as I understand, according FIXP recommendation, client can recover messages occured during that disconnection (even if server knows that client has been disconnected - i.e. server should save these unsent messages on its side)? Am I right?
  3. How client and server should sequence messages across different logical sessions in FIXP? Should it be one shared sequence order - i.e. it means that application messages are also shared? In that case sides can recover messages in any other logical session?

Thank you!

  1. FIXP is intended to be a simple, light-weight protocol. Unlike FIX4 and FIXT, there is no provision in FIXP for sequence number reset. Thus, a complex interaction is eliminated. The FIXP sequence number is an unsigned 64-bit number. It is highly unlikely that that number of messages would be sent in the context of a session.
  2. If a party provides a recoverable flow in their service offering, then yes, it is assumed that they must persist message for recovery. A firm might place a limit on the number of messages that can be recovered. A request for resend may be rejected with rejection code RequestLimitExceeded. The protocol does not currently have a feature to announce such limits; it is expected that a firm would inform its counterparties out of band.
  3. Each flow has its own sequence number; sequence numbers are not shared across sessions. Therefore, there is no need to synchronize on shared data, which would typically cause delay. A request for resend is based on the sequence of the flow being recovered.

@donaldmendelson thanks a lot for your clarifications. I just one more time read FIXP document carefully taking into account your comments, and these questions are closed for me.

I saw your project conga here: https://github.com/FIXTradingCommunity/conga

Although I’m C#-guy this project really helped me to understand some details. But one more detail I would like to clarify - how to translate order book using such stack: FIX 5.0 SP2 + SBE + FIXP over WebSocket with TLS…

The question is: as sequencing is separate for each logical session in FIXP, server should collect missed market data for each session. Too many data… Will this solution be better: make market data flow from server as unsequenced, and use sequencing as it described in FIX 5.0 SP2 - on application layer? So, on this level we may use general sequence - the same for all clients.

@analitiq FIXP supports point-to-point communications, and WebSocket is a point-to-point protocol. It would be possible to send the same market data to multiple clients using that protocol stack. However, it would be somewhat inefficient. FIXP also supports the pattern of one publisher / many listeners over multicast. The publisher sends each message only once and sets sequence numbers on packets. Listeners use the sequence numbers to detect gaps.

Typically, market data providers have a separate channel for recovery of missed data. Out-of-band recovery was discussed by the working group but has not yet been standardized.

@donaldmendelson yeap, I saw multicast option with FIXP, and also had experience with one exchange which sends data via UDP with recovery on TCP. Unfortunally, it seems like not suitable for our MVP with FIX. :wink: I think, we will first try FIXP + WebSocket.

@analitiq it would be great to hear back from you here at some point about experiences you make with FIXP and WebSocket. FIXP v1.1 is now in Draft Standard which means that we are waiting for two independent and interoperable implementations to remove the term “Draft”, proving that the spec on paper works in practice. Thank you!

@hanno.klein I have changed title of this post as our discussion has been extended.

We are working on crypto exchange now. I will share with you our document when it is ready. Also may provide you final feedback after final run. We are targeting institutional clients, and we are interesting in interoperable of our FIX implementation.

There are some thinkings about “how” currently:

  • Transport Layer - WebSocket Protocol RFC 6455 runs over a TCP transport.
  • Security Mechanisms - FIX traffic is protected by using Transport Layer Security 1.2 (TLS) protocol with recommendation from FIX-over-TLS 1.0 (FIXS). Server is authenticated by client using certificate on TLS level. Server authenticates client at the beginning of session on session layer.
  • Presentation Layer - Simple Binary Encoding 1.0 (SBE) (but I saw 2.0 RC2) and JSON RC1 (as there is no standard yet).
  • Session Layer - session logic is ensured by FIX Performance 1.1 (FIXP) semantics.
  • Application Layer - messages meet FIX 5.0 SP2.
1 Like

Messages should meet “FIX Latest”. FIX 5.0 SP2 was first published in 2006 and only includes Extension Packs EP1-EP97. FIX Latest stands for all Extension Packs that have been published, currently EP249 is the newest one (see https://www.fixtrading.org/extension-packs/).

Very excited to hear about your plans for the technical stack used for your crypto exchange! Sounds great!

Sure, I thought that FIX 5.0 SP2 includes all extensions. For example, we are considering to use EP191 to provide OHLCV history to clients.

We stopped creating new versions every few years just to include previously approved extensions and decided to publish EPs immediately as of EP98. In general the versioning of the application only makes sense for legacy versions nowadays, e.g. FIX 4.2 or FIX 4.4, because some user struggle to update their implementations to FIX 5. We do apply versions to the Technical Standards of the presentation/encoding and session layer as you cannot support only parts of it. This is different for the application layer where counterparties always agree on the supported subset of all FIX messages, fields and valid values.

Extension Packs are incremental, e.g. EP191 includes EP1 - EP190. We suggest to simply use FIX Latest. BTW, happy to see EP191 being used. I submitted that extension during my time at Group Deutsche Börse. We extended that further for MiFID RTS 27 and RTS 28, please see EP231 and EP236.

1 Like

Thank you for clarification, will use FIX Latest! And taked into account suggested EPs.

Hmm…

  1. How to mark in “MktDataInc” message that order has been deleted/changed on market as result of matching? The question is because field DeleteReason(285) has been depreciated in FIX 5.0. In general, let’s consider the following example. We have entries on market:
Current market state
MDEntityID_1001 sell 3@100
MDEntityID_1003 sell 1@90
MDEntityID_1005 sell 2@90
----------
MDEntityID_1007 buy 1@80
MDEntityID_1008 buy 1@80

Market buy order has been sent with volume 5.

What should I send to client in message “MktDataInc”? My suggestion is to send the following message with entries:

Bid/Offer entries
MDUpdateAction = New
MDEntryID(278) = 1009
MDEntryType(269) = Bid
MDEntrySize(271) = 5
OrderID(198) = 200001
OrdType(40) = Market
----------
MDUpdateAction = Change
MDEntryID(278) = 1009
MDEntrySize(271) = 3
----------
MDUpdateAction = Change
MDEntryID(278) = 1009
MDEntrySize(271) = 2
----------
MDUpdateAction = Delete
MDEntryID(278) = 1009
----------
MDUpdateAction = Delete
MDEntryID(278) = 1005
----------
MDUpdateAction = Delete
MDEntryID(278) = 1003
----------
MDUpdateAction = Change
MDEntryID(278) = 1001
MDEntrySize(271) = 1
Trade entries
MDUpdateAction = New
MDEntryID(278) = 1010
MDEntryType(269) = Trade
MDEntrySize(271) = 2
// OPTIONAL field for this case, agressor OrderID
OrderID(198) = 200001
// OPTIONAL field for this case, may be limit with higher price
OrdType(40) = Market
AggressorSide(2446) = Buy
----------
MDUpdateAction = New
MDEntryID(278) = 1011
MDEntryType(269) = Trade
MDEntrySize(271) = 1
// OPTIONAL field for this case, agressor OrderID
OrderID(198) = 200001
// OPTIONAL field for this case, may be limit with higher price
OrdType(40) = Market
AggressorSide(2446) = Buy
----------
MDUpdateAction = New
MDEntryID(278) = 1012
MDEntryType(269) = Trade
MDEntrySize(271) = 2
// OPTIONAL field for this case, agressor OrderID
OrderID(198) = 200001
// OPTIONAL field for this case, may be limit with higher price
OrdType(40) = Market
AggressorSide(2446) = Buy

Am I right?

  1. And how can I specify if two actions with the same timestamp occurred at the same MDEntryID(278) (for example, order replacement and cancellation after)? Sending them in different packages? It seems like not good idea…

Sorry, if some items are unclear in my questions…

Regarding my question #1 above. Suggestion is to include field TradeID(1003) in entities. It will mean that changes/deletion because of trade.

Is there way to specify init Qty for bid/offer in message “MktDataFull” or this message is not for this purpose?

Just a short answer here as this forum is not the right place to provide consulting for your interface design. See FIX 5.0 SP2 Spec Volume 3, page 95 which describes incremental market data updates. Here are a few things to consider:

  • MDEntryID(278) is intended for order book entries, not for trade entries.
  • Trade entries can contain OrderID(37) to refer back to the resting order that traded
  • Order modification/deletion and trade with order reference in the same message is sufficient as reason
  • Orders not resting on the book (i.e. incoming orders that match immediately) should not be added as new entries, they are only implicit
  • In general, reduce verbosity of order book actions as much as you can to increase overall performance
1 Like

Maybe you meant Volume 3, page 95? Yes, I read that. Unfortunately, some items still staid unclear for me. Your answer is very helpful to move me in right direction. Thanks!

As it is public forum, I understand that some my questions may stay unanswered. Maybe I will find answers myself and publish here for next newbie in FIX as I am today. Please, sorry that there were a lot of questions, I decided that simple example will be better than text.

Oops, thanks, changed to Volume 3. No problem with posting questions, the more specific the better to get answers. FIX does offer quite some flexibility for the design of market data messages, i.e. the design of the interface should be viewed together with your given system architecture. For example, aligning the transaction boundaries of your core matching engine with the market data messages (inc or full) sent out to customers. On the other hand, your customers should be advised to process an entire incremental message before taking any action. The order book should be consistent between any two messages but not necessarily between two entries in the repeating group.

1 Like

I’m reading SBE. One question: for example, here https://github.com/FIXTradingCommunity/fix-simple-binary-encoding/blob/master/v1-0-STANDARD/doc/07Examples.md the following type is specified:

<type name="idString" length="8" primitiveType="char" semanticType="String"/>

and it used in NewOrderSingle message:

<sbe:message name="NewOrderSingle" id="99" blockLength="54" semanticType="D">
    <field name="ClOrdID" id="11" type="idString" description="Customer Order ID"
        offset="0" semanticType="String"/>
...

Can I change type of ClOrdID to int? I.e. I changed type to int and updated semanticType. Will this be violation of standard?

<sbe:message name="NewOrderSingle" id="99" blockLength="54" semanticType="D">
    <field name="ClOrdID" id="11" type="int" description="Customer Order ID"
        offset="0" semanticType="int"/>
...

Yes, although most identifiers in FIX are specified as String datatype, in a binary protocol it is acceptable to use a numeric type for performance.

We covered that point in Usage Guidelines

1 Like