Gap Fills

[andrewschwartz_x]

Imported from previous forum

[andrewschwartz_x]

In the FIX 4.4 spec, in the discussion of Message Recovery, it states:

“The SeqReset-GapFill can also be used to skip application messages that the sender chooses not to retransmit (e.g. aged orders). This leaves Reject as the only administrative message which can be resent.”

I am using the functionality of not retransmitting aged orders. I would like to make sure that if I do this, I can rely on the behavior of the order never being filled.

My concern is the following situation: I submit a NewOrderSingle. The counterparty’s FIX endpoint sends the order off to be matched before marking it processed. The counterparty’s FIX endpoint then randomly crashes. When the session is re-established, I get a ResendRequest for the order and I use the GapFill because the order is aged. I rely on the order never having been sent, and mark it done internally. Then, I get an Execution Report for the order, because it had already been sent to the matching engine.

Is this possible? If so, how can I handle this situation?

[fplpo_x]

[ original email was from John Prewett - jprewett@lavatrading.com ]
Hi Andrew,

I too have experienced this type of problem with at least one execution venue. They would appear to be breaking the spirit, if not the specification, of the protocol.

Unfortunately, I know of no simple defense to this problem. So all I can do is commiserate.

On a more positive note, there a two things you could try:

1. Always assume that the execution venue may have received the message and resend it unaltered as a result of a resend request. Immediately after processing the resend request you could then cancel the orders that you believe are stale.

2. Take a sort of hybrid approach and send gap fills and also cancel requests. If an OUT or a cancel reject is returned, it would then be safe to assume that no order is open on the market and you would have minimized the risk to the best of your ability.

Ultimately, I believe it is the responsibility of the receiver of an order to determine if it is stale or not. They have TransactTime, OrigSendingTime and SendingTime at their disposal. This is more than sufficient to implement a suitable stale rejection logic.

I hope this rambling response helps slightly.

JohnP

In the FIX 4.4 spec, in the discussion of Message Recovery, it states:

“The SeqReset-GapFill can also be used to skip application messages that
the sender chooses not to retransmit (e.g. aged orders). This leaves
Reject as the only administrative message which can be resent.”

I am using the functionality of not retransmitting aged orders. I would
like to make sure that if I do this, I can rely on the behavior of the
order never being filled.

My concern is the following situation: I submit a NewOrderSingle. The
counterparty’s FIX endpoint sends the order off to be matched before
marking it processed. The counterparty’s FIX endpoint then randomly
crashes. When the session is re-established, I get a ResendRequest for
the order and I use the GapFill because the order is aged. I rely on the
order never having been sent, and mark it done internally. Then, I get
an Execution Report for the order, because it had already been sent to
the matching engine.

Is this possible? If so, how can I handle this situation?

[andrewschwartz_x]

Thanks, John. I was thinking about something along the lines of #2, though I had been hoping it would not be necessary.

One possible issue: if I sent an Order Cancel/Replace Reject and get an Order Cancel Reject response because the order does not exist, is this information reliable?

Simply put, if I issue an order and immediately try to cancel it, will I ever get an Order Cancel Reject specifying that the order doesn’t exist, perhaps due to some sort of race?

Hi Andrew,

I too have experienced this type of problem with at least one execution
venue. They would appear to be breaking the spirit, if not the
specification, of the protocol.

Unfortunately, I know of no simple defense to this problem. So all I can
do is commiserate.

On a more positive note, there a two things you could try:

1. Always assume that the execution venue may have received the message
   and resend it unaltered as a result of a resend request. Immediately
   after processing the resend request you could then cancel the orders
   that you believe are stale.

2. Take a sort of hybrid approach and send gap fills and also cancel
   requests. If an OUT or a cancel reject is returned, it would then be
   safe to assume that no order is open on the market and you would
   have minimized the risk to the best of your ability.

Ultimately, I believe it is the responsibility of the receiver of an
order to determine if it is stale or not. They have TransactTime,
OrigSendingTime and SendingTime at their disposal. This is more than
sufficient to implement a suitable stale rejection logic.

I hope this rambling response helps slightly.

JohnP

In the FIX 4.4 spec, in the discussion of Message Recovery, it states:

“The SeqReset-GapFill can also be used to skip application messages
that the sender chooses not to retransmit (e.g. aged orders). This
leaves Reject as the only administrative message which can be resent.”

I am using the functionality of not retransmitting aged orders. I
would like to make sure that if I do this, I can rely on the behavior
of the order never being filled.

My concern is the following situation: I submit a NewOrderSingle. The
counterparty’s FIX endpoint sends the order off to be matched before
marking it processed. The counterparty’s FIX endpoint then randomly
crashes. When the session is re-established, I get a ResendRequest for
the order and I use the GapFill because the order is aged. I rely on
the order never having been sent, and mark it done internally. Then, I
get an Execution Report for the order, because it had already been
sent to the matching engine.

Is this possible? If so, how can I handle this situation?

[kaymahes_x]

Andrew,

From a FIX neutral perspective, I thought of a solution, but this requires changes in the counterpartiy’s FIX Endpoint and an enhancement in your FIX Engine.

Your counterparty receives a new order, send your FIX Engine an Ack, then starts executing / working it in the market. Assume a random crash of the counterparty’s “System” (its a “black box”):-

1. Crash occurs before your order reached them. They make a ResendRequest, you do a seq reset gap fill because you choose not to send the “stale” order.

2. Crash occurs after your order is received in their system and before they send the Ack to you. Reconnect happens, Ack and Executions if any are received by you.

3. Crash occurs after they have sent you an Ack, their system is working the order (executions get ready to be sent to you), reconnect happens, you receive executions.

Now there is a scenario in which your traders do not like the prices on the exec reports and decide to cancel the un executed quantity.

For this to work acceptably, the counterparty’s FIX engine would have to implement the following logic:-

A tag needs to be included to indicate that the order should be pulled out from market if disconnect happens. I could not find any such tag in the FIX 5.0 spec pdf (Version 5.0 - VOLUME 4, Pages 7 to 11 of 185, CATEGORY: SINGLE/GENERAL ORDER HANDLING), I have not checked other versions, I assume such a Tag is not present in FIX even in older versions because I see some Tags marked as deprecated. Let it be called “this Tag”. If I were to design a FIX Tag for this purpose, then it would be as follows:-

1. “this Tag” is an optional field of type quantity, if missing, the order maintains its status as if the crash / FIX session disconnect did not happen.

2. If “this Tag” contains 0 (zero) it means pull your order out of the market completely, ie try to make the order quantity zero on disconnect. If partly executed, cancel unexecuted quantity and transmit executions on reconnect.

3. “this Tag” contains a quantity more than zero and less than order quantity, it means, if disconnect happens, try reducing your order to the quantity specified in “this” tag. If executions already happened and

total executed quantity quantity in this Tag

Less - Reduce Order quantity to quantity in “this” tag and continue working the leaves quantity = quantity in this Tag minus executed quantity.

Equal - Cancel unexecuted quantity

More - Cancel unexecuted quantity

4. “this Tag” contains a quantity equal to order quantity, it behaves as if “this Tag” were absent ie no change to the quantity.

5. “this Tag” contains a quantity greater than order quantity - Error - invalid Data in “this Tag”.

I see this as a way to safegaurd against your order getting unfavourably executed.

If I were writing a Buy Side program (source of New orders and destination of Executions) to perform automated trading on multiple avenues, ie makes Buy/Hold/Sell decisions of securities based on market data, then this would be a way to hedge its risk against FIX Session disconnect. When the connection is alive, my program watches its order and cancels it when wanted. If “this Tag” is populated, then it knows, either there is nothing to watch or there is less to watch ie less at “risk” when disconnect happens. Just my 0.0002 cents

From my point of view I would prefer not to look inside the counterparty’s FIX Engine to matching engine interaction.

Everybody,

Is there is a standard Tag in FIX which implements this functionality?

Regards,
K. Mahesh

[hanno.klein]

Mahesh,

the execution instructions that can be attached to an order (ExecInst) are the right place to provide the basic behaviour you describe. Values K and Q ask the counterparty to cancel an order on trading halt and system failure resp. A current proposal of the Global Exchanges and Markets Committee (EEWG Phase 1) wants to add the case of connection loss. It also wants to add “Suspend” to the existing two actions “Cancel” and “Reinstate” to keep the order in the book but eliminate the risk of execution.

Andrew’s case seems to be covered already, i.e. ExecInst=Q asks to Cancel on System Failure. The counterparty’s system needs to realize it has crashed when it comes up again and can then cancel any orders marked with this execution instruction. This should trigger an Execution Report to inform about the cancellation. It is then up to the original sending party to submit the order again or to define it as stale and not submit it.

Connection loss is simply a less severe case than system failure but highly relevant for exchange environments, especially for market makers who lose their connection while the exchange system continues to operate.

Regards,
Hanno.

Everybody,

Is there is a standard Tag in FIX which implements this functionality?

Regards,
K. Mahesh

[andrewschwartz_x]

Thank you for the reply.

What I am interested in specifically is whether John’s statement “If an OUT or a cancel reject is returned, it would then be safe to assume that no order is open on the market…” is really true. If the FIX Spec guarantees the statement, then my issue is resolved.

Can anyone shed light on this?

Thanks.

Andrew,

From a FIX neutral perspective, I thought of a solution, but this
requires changes in the counterpartiy’s FIX Endpoint and an
enhancement in your FIX Engine.

Your counterparty receives a new order, send your FIX Engine an Ack,
then starts executing / working it in the market. Assume a random crash
of the counterparty’s “System” (its a “black box”):-

1. Crash occurs before your order reached them. They make a
   ResendRequest, you do a seq reset gap fill because you choose not to
   send the “stale” order.

2. Crash occurs after your order is received in their system and before
   they send the Ack to you. Reconnect happens, Ack and Executions if
   any are received by you.

3. Crash occurs after they have sent you an Ack, their system is working
   the order (executions get ready to be sent to you), reconnect happens,
   you receive executions.

Now there is a scenario in which your traders do not like the prices on
the exec reports and decide to cancel the un executed quantity.

For this to work acceptably, the counterparty’s FIX engine would have to
implement the following logic:-

A tag needs to be included to indicate that the order should be pulled
out from market if disconnect happens. I could not find any such tag in
the FIX 5.0 spec pdf (Version 5.0 - VOLUME 4, Pages 7 to 11 of 185,
CATEGORY: SINGLE/GENERAL ORDER HANDLING), I have not checked other
versions, I assume such a Tag is not present in FIX even in older
versions because I see some Tags marked as deprecated. Let it be called
“this Tag”. If I were to design a FIX Tag for this purpose, then it
would be as follows:-

3. “this Tag” is an optional field of type quantity, if missing, the
   order maintains its status as if the crash / FIX session disconnect
   did not happen.

4. If “this Tag” contains 0 (zero) it means pull your order out of the
   market completely, ie try to make the order quantity zero on
   disconnect. If partly executed, cancel unexecuted quantity and
   transmit executions on reconnect.

5. “this Tag” contains a quantity more than zero and less than order
   quantity, it means, if disconnect happens, try reducing your order
   to the quantity specified in “this” tag. If executions already
   happened and

total executed quantity quantity in this Tag

Less - Reduce Order quantity to quantity in “this” tag and
continue working the leaves quantity = quantity in this Tag minus
executed quantity.

Equal - Cancel unexecuted quantity

More - Cancel unexecuted quantity

6. “this Tag” contains a quantity equal to order quantity, it behaves as
   if “this Tag” were absent ie no change to the quantity.

7. “this Tag” contains a quantity greater than order quantity - Error -
   invalid Data in “this Tag”.

I see this as a way to safegaurd against your order getting
unfavourably executed.

If I were writing a Buy Side program (source of New orders and
destination of Executions) to perform automated trading on multiple
avenues, ie makes Buy/Hold/Sell decisions of securities based on market
data, then this would be a way to hedge its risk against FIX Session
disconnect. When the connection is alive, my program watches its order
and cancels it when wanted. If “this Tag” is populated, then it knows,
either there is nothing to watch or there is less to watch ie less at
“risk” when disconnect happens. Just my 0.0002 cents

From my point of view I would prefer not to look inside the
counterparty’s FIX Engine to matching engine interaction.

Everybody,

Is there is a standard Tag in FIX which implements this functionality?

Regards,
K. Mahesh

[fplpo_x]

[ original email was from John Prewett - jprewett@lavatrading.com ]
OK, maybe I should retract that statement.
While this would be valid from the perspective of adherence to FIX protocol, I have definitely seen execution venues where this wouldn’t be 100% true.

“Don’t cancel too quickly please” was what someone told me from one venue. I’m not sure what the definition of “quick” actually was in that case.

It is a good idea to support logic whereby you can cancel an order with or without waiting for an ExecReport(new). Then venue certification will make you aware of which venues support “quick” (don’t wait for execreport-new) and which venues support “non-quick” (wait for execreport-new) cancellation.

Of course, this entire discussion highlights the fact that adherence to FIX protocol is a wonderful thing.

JohnP

Thank you for the reply.

What I am interested in specifically is whether John’s statement “If an
OUT or a cancel reject is returned, it would then be safe to assume that
no order is open on the market…” is really true. If the FIX Spec
guarantees the statement, then my issue is resolved.

Can anyone shed light on this?

Thanks.

[andrewschwartz_x]

OK, maybe I should retract that statement. While this would be valid
from the perspective of adherence to FIX protocol, I have definitely
seen execution venues where this wouldn’t be 100% true.

this is encouraging. for my reference, where could I find this in the spec?