The Global Technical Committee has reviewed and preliminarily approved the submitted proposal. This document from the FIX Continuous Markets Working Group sets out a series of topics that should be addressed when using FIX for a continuous workflow over an extended period of time.
The document now enters a public comment period in which public review and feedback is encouraged. Once the public comment period closes, the GTC will review public comments and amend the document accordingly before publishing the final version.
Please post feedback, comments, and questions as replies to this discussion thread. The public comment period ends on July 5, 2023.
Resend requests that are out of range should result in a logoff as this can be considered to constitute a serious problem requiring manual intervention.
pg. 8: Would it also be possible to simply send a SequenceReset with NewSeqNo set to the earliest sequence number that could be resent? I.e. skipping over all messages that are out of range?
We therefore recommend placing an upper bound on the number of messages that can be recovered within an implementation’s rules of engagement.
Regarding “number of messages”, sometimes it makes more sense to refer to this as limit in time, e.g. “max 48 h of retransmission history is available”.
However, the “number of messages” is simpler to reason with in the following examples.
Yes, “limit in time” (message age) would be an alternative to “limit in number of messages”. They mean essentially the same thing, i.e. an upper bound on retransmission history.
For example, two clients with very different throughput, the limit of e.g. 10 000 messages may mean 10 seconds or 10 days. An exchange usually have the same rules for all members, and in this case a limit in time, of e.g. 24 h, would be more fair.
Time (or age) is also very common in data retention policies. For example in applications such as data warehouse, metrics collection and event streaming platforms (Apache Kafka). The latter might even be used as a backing store for a FIX engine implementation.
I suggest that section 5.2 have the following text added:
“This upper bound may be either expressed as number of messages (e.g. 10,000) or max message age (e.g. 24 hours).”
or
“As an alternative, this upper bound may be expressed as a max age of messages to be redelivered, e.g. 24 hours.”
Apart from this minor detail, I think the recommended practices for 24x7 sessions is a good addition to the community that hopefully will move the industry away from the day-centric approach of resetting sessions every day.
Thanks for the comment. Alongside @ejanec I am co-chair of the working group and you can see I wrote the document to which you are commenting.
The sequence reset idea was considered, but that would then allow a session to continue without any clear notification that the messages requested were potentially useful but not available.
This would then need some form of change to allow a notification “you have asked for messages from 1 to max uint16 but you can only have the last x messages”
If this is a frequently requested item in the wild then I would reconvene the working group and add this as an agenda item.
For now, I think we can note this item and continue.
Thanks for the comment. Alongside @ejanec I am co-chair of the working group and you can see I wrote the document to which you are commenting.
I would consider this as an implementation detail, not one that requires documentation. At the moment, in production, I have seen multiple different ways to handle resent requests:
(a) You can ask for anything you like
(b) You can only ask for current sequence number - N messages
(c) You can ask for anything you like but only can ask for messages in chunks of M messages. If you want 10,000 messages and M is set to 5,000 then you need to ask twice for resends.
As an implementation detail I don’t think this document needs to cover that.
If these is strong disagreement we can reopen the document…
(cc @ejanec)
The challenge here is that an exchange is part of an ecosystem. Exchanges sell market data, they don’t generally want to be in the business of persisting that data for days or another long time. Therefore for market data it does not make much sense to offer this sort of time-bound feature.
For orders/executions and drop-copy feeds, it would be rather odd to request resends from two days back, since the data should already have been sent downstream to settlement.
As such, I am not sure of the validity of the use-case for a time-bound resend request within the context of FIX.
Happy to discuss, I don’t think we spoke on the working group - if that would be helpful let me know.