Imported from previous forum
Chaps,
I’m trying to understand the FIXP recovery model(s) better. And specifically want the answer to this question:
One exchange protocols’ recovery model is not to recover individual messages but instead restart the session with a ‘state of the world’ replay (e.g. of executions and open orders). This makes sense especially in a high frequency environment where a) the venue may have cancel-on-disconnect on, b) the number of orders involved might be huge and c) the number of recoverable objects (fills, in this case), would be relatively small. This sounds like a job for ‘idempotent’ recovery but how do you essentially recover all the executions without needing to recover everything else?
jim.
FIXP is a session layer protocol so it is only concerned with reliable, in-order delivery of messages, not with their business semantics. That does not preclude application layer recovery with aggregation or filtering as you suggest. In fact, FIXP provides an unsequenced flow type in case sequencing and recovery are handled at the application layer. Thus, duplication of effort is avoided. (For that matter, some degree of reliability may be built into a transport such as TCP. Avoid duplication of effort with a lower layer as well.)
That being said, it may be useful for FIX to suggest standardization of cancel-on-disconnect terminology and business logic as well as fail-over for disconnected trading systems.
Hi - this is very helpful - thanks. Just so I fully understand you, I can take a FIXP session and pretty much implement a recovery model of my choosing? So I’m not tied into a full-message replay (like current gen FIX has) but could instead reset my ‘session’ by, for exmaple, sending a recap of fills and open orders?
Jim.
The mechanism for requesting retransmission in FIXP is similar to FIXT. A RetransmitRequest message conveys the range of sequence numbers to resend. Replayed messages may be interleaved with real-time messages. It is up to the receiver to reconstruct a gapless sequence.
The only pattern in FIXT is guaranteed delivery in both directions. On the other hand, FIXP provides more flexiblity – not every stream must be recoverable. Session flows may be asymmetrical, e.g. idempotent flow of orders inbound and recoverable flow of executions outbound. (Trading systems emulate this today with gap fills in FIXT to avoid reprocessing of stale orders, but it’s simpler to just declare the order flow unrecoverable.)
Many alternatives are possible for recovery services. At session layer, a separate recovery channel could be provided to avoid interfering with real-time flows. This is sometimes done for market data. Anything that involves requesting by message type or summarizing of data is best left to the application layer, though.
Jim, just a note in addition to Don’s clarifications.
In a single session, if messages are recoverable, all of them are.
To create another logical stream that is filtered or manipulated in some way, that second logical stream is carried by a separate FIXP session.
A FIXP session is lightweight and is intended to be automated to a high degree, unlike the FIXT session.
This means that the same logical entity, the “CompID”, can be used to negotiate a set of FIXP sessions that comprise a higher level service.
For instance, every time you would want to do a sequence reset in FIXT, you would instead throw away the FIXP session and negotiate a new FIXP session for the same CompID to start over.
The same scheme can be used to create filtered sessions (for e.g fills) without requiring high touch involvement from an ops team.
To outline some options:
Provide fills in the form of a drop copy flow in a separate service, using the FIXP recoverable flow type.
Use the same counterparty identifiers to avoid manual session management, full automation.
Provide a snapshot service for open orders using the FIXP unsequenced flow type. If there is message loss during the transfer of this snapshot, make the client ask again. The use of the unsequenced flow type means that the snapshot is discarded after being transfered, not requiring message logs.
Again, fully automate the management of the separate FIXP session being used here by letting counterparty identification be shared across the different flows (the same CompID).