Compiling use cases / test cases for HFT

, ,
1

Imported from previous forum

2

What could we use as the starting point for compiling usecases / testcases for HFT? Could we start with testcases in one of the appendices of FIXProtocol specifications. Or should we start with a blank sheet because HFT design is being done from scratch rather that incrementaly refining / enhancing existing Tag=Value^ FIX and many new / different usecases could be found by taking a “from scratch” approach.

I think the use cases / test cases could be classified into three categories :-

  1. Buy side - the High frequency trader perspective
  2. Order router - the Broker algorithim perspective
  3. Exchange - the high volume trade matching engine perspective

Regards,
Mahesh

3

real life transaction traces are very valuable but hard to come by for obvious reasons. The next best thing is to synthesize traces using a statistical distribution gathered from real life data. This approach has lower fidelity and requires more work to verify the validity of the generated data. A third approach is to use public data as a basis and then “enrich” the data with synthesized data such as client ids, order ids etc.

However we do it, we will need trading firms and exchanges to verify the validity of the test data sets.

/Rolf

What could we use as the starting point for compiling usecases / testcases for HFT? Could we start with testcases in one of the appendices of FIXProtocol specifications. Or should we start with a blank sheet because HFT design is being done from scratch rather that incrementaly refining / enhancing existing Tag=Value^ FIX and many new / different usecases could be found by taking a “from scratch” approach.

I think the use cases / test cases could be classified into three categories :-

  1. Buy side - the High frequency trader perspective
  2. Order router - the Broker algorithim perspective
  3. Exchange - the high volume trade matching engine perspective

Regards,
Mahesh

4

[ original email was from John Harris - john.harris@bondmart.com ]
Starting with a blank sheet is best. My initial list of elemental use cases:

Market-data-related operations

Request information
Receive information
Subscribe to information stream
Receive information stream
Cancel subscription to information stream
Receive acknowledgement of information-stream cancellation

Reference- and other static-data-related operations

Post information
Receive acknowledgement of information post
Request information
Receive information

Order- and trade-related operations

Place order
Amend order
Cancel order
Receive order-related reports
Settle trades*

I did not include any administrative functions. Any thoughts on whether those are in scope?

  • I’m sure “settle trades” is actually a complex, not elemental case, but I will have to decompose it later.

real life transaction traces are very valuable but hard to come by for obvious reasons. The next best thing is to synthesize traces using a statistical distribution gathered from real life data. This approach has lower fidelity and requires more work to verify the validity of the generated data. A third approach is to use public data as a basis and then “enrich” the data with synthesized data such as client ids, order ids etc.

However we do it, we will need trading firms and exchanges to verify the validity of the test data sets.

/Rolf

What could we use as the starting point for compiling usecases / testcases for HFT? Could we start with testcases in one of the appendices of FIXProtocol specifications. Or should we start with a blank sheet because HFT design is being done from scratch rather that incrementaly refining / enhancing existing Tag=Value^ FIX and many new / different usecases could be found by taking a “from scratch” approach.

I think the use cases / test cases could be classified into three categories :-

  1. Buy side - the High frequency trader perspective
  2. Order router - the Broker algorithim perspective
  3. Exchange - the high volume trade matching engine perspective

Regards,
Mahesh

5

Rolf,

You said “we will need trading firms and exchanges to verify the validity of the test data sets”. Would FPL member exchanges, brokers, buy side firms and others be able to provide us with their valid “public testcases” and any other relevant data. Then our WG can find set of usecases / testcases common to all and various stakeholders can validate this common set.

John,

I would say we should restrict “High Frequency trading protocol” to be just what its name stands for. So only

Order- and trade-related operations

Place order(s) - Single, List, Multileg, …
Amend order(s)
Cancel order(s)
Enquire status of Order(s)
All flavours of Execution reports
Receive other order-related reports
DKT and Execution acknowledgement messages

Market data is not included in my scope because FAST is already very well optimized for streaming market data, so no point trying to optimize **T for both trading and market data. If I understand correctly, the design of FAST was based on optimizing “Streaming market data” only and there was no attempt to optimize all phases of trading life cysle.

Reference- and other static-data-related operations is not included in my scope because these kind of DataSets are massive and trying to transmit them would choke a pipe optimized for carrying lean & mean very latency sensitive trading information.

Settlement related messages are not included because I have not heard of any millisecond fast settlements, in many cases settlements are T+3. Also I believe settlement has some manual processes for security / legal reasons (I am not sure of this).

Regards,
Mahesh

Starting with a blank sheet is best. My initial list of elemental use cases:

Market-data-related operations

Request information
Receive information
Subscribe to information stream
Receive information stream
Cancel subscription to information stream
Receive acknowledgement of information-stream cancellation

Reference- and other static-data-related operations

Post information
Receive acknowledgement of information post
Request information
Receive information

Order- and trade-related operations

Place order
Amend order
Cancel order
Receive order-related reports
Settle trades*

I did not include any administrative functions. Any thoughts on whether those are in scope?

  • I’m sure “settle trades” is actually a complex, not elemental case, but I will have to decompose it later.

real life transaction traces are very valuable but hard to come by for obvious reasons. The next best thing is to synthesize traces using a statistical distribution gathered from real life data. This approach has lower fidelity and requires more work to verify the validity of the generated data. A third approach is to use public data as a basis and then “enrich” the data with synthesized data such as client ids, order ids etc.

However we do it, we will need trading firms and exchanges to verify the validity of the test data sets.

/Rolf

What could we use as the starting point for compiling usecases / testcases for HFT? Could we start with testcases in one of the appendices of FIXProtocol specifications. Or should we start with a blank sheet because HFT design is being done from scratch rather that incrementaly refining / enhancing existing Tag=Value^ FIX and many new / different usecases could be found by taking a “from scratch” approach.

I think the use cases / test cases could be classified into three categories :-

  1. Buy side - the High frequency trader perspective
  2. Order router - the Broker algorithim perspective
  3. Exchange - the high volume trade matching engine perspective

Regards,
Mahesh

6

[ original email was from John Harris - john.harris@bondmart.com ]
Mahesh,

Trading is one phase or activity in portfolio or market-position management. Imagine that a trader has reached a satisfied state. He takes in new information. This information unsettles him. He evaluates whether he should act. He decides to act. He submits an order. He receives order-state information. If executed, he settles his obligation and returns, however briefly, to a satisfied state. Repeat process.

The trader may move from one activity to the next instantaneously. Latency matters in every phase. Even when the trade is made, there will typically be a brief interlude between execution by an exchange or dealer and novation by a clearinghouse. All parties want that interlude to be as brief as possible.

The demand for message reliability varies throughout the process. An execution report, for example, matters in three operational domains at once: order management, market data, and settlement.

What constitutes “high frequency” trading is ambiguous. On any given trade, latency may matter more to someone who trades once per year than to someone trading dozens of times per second. Trying to design a new protocol just for high-frequency traders is a waste of time.

It may be that FaST is perfect and no further improvement in market data protocols is possible. But just in case that isn’t true, I would counsel against ignoring market data or any other aspect of trading, this early in a protocol development process. I have yet to encounter perfection in my adult life, except for the ice cream sundaes at Eggers on Staten Island.

Finally, it isn’t necessary to transmit an entire set of reference or static data in messages requiring same. Most trade messages require such data, whether implicitly or explicitly.

Best,
John

Rolf,

You said “we will need trading firms and exchanges to verify the validity of the test data sets”. Would FPL member exchanges, brokers, buy side firms and others be able to provide us with their valid “public testcases” and any other relevant data. Then our WG can find set of usecases / testcases common to all and various stakeholders can validate this common set.

John,

I would say we should restrict “High Frequency trading protocol” to be just what its name stands for. So only

Order- and trade-related operations

Place order(s) - Single, List, Multileg, …
Amend order(s)
Cancel order(s)
Enquire status of Order(s)
All flavours of Execution reports
Receive other order-related reports
DKT and Execution acknowledgement messages

Market data is not included in my scope because FAST is already very well optimized for streaming market data, so no point trying to optimize **T for both trading and market data. If I understand correctly, the design of FAST was based on optimizing “Streaming market data” only and there was no attempt to optimize all phases of trading life cysle.

Reference- and other static-data-related operations is not included in my scope because these kind of DataSets are massive and trying to transmit them would choke a pipe optimized for carrying lean & mean very latency sensitive trading information.

Settlement related messages are not included because I have not heard of any millisecond fast settlements, in many cases settlements are T+3. Also I believe settlement has some manual processes for security / legal reasons (I am not sure of this).

Regards,
Mahesh

Starting with a blank sheet is best. My initial list of elemental use cases:

Market-data-related operations

Request information
Receive information
Subscribe to information stream
Receive information stream
Cancel subscription to information stream
Receive acknowledgement of information-stream cancellation

Reference- and other static-data-related operations

Post information
Receive acknowledgement of information post
Request information
Receive information

Order- and trade-related operations

Place order
Amend order
Cancel order
Receive order-related reports
Settle trades*

I did not include any administrative functions. Any thoughts on whether those are in scope?

  • I’m sure “settle trades” is actually a complex, not elemental case, but I will have to decompose it later.

real life transaction traces are very valuable but hard to come by for obvious reasons. The next best thing is to synthesize traces using a statistical distribution gathered from real life data. This approach has lower fidelity and requires more work to verify the validity of the generated data. A third approach is to use public data as a basis and then “enrich” the data with synthesized data such as client ids, order ids etc.

However we do it, we will need trading firms and exchanges to verify the validity of the test data sets.

/Rolf

What could we use as the starting point for compiling usecases / testcases for HFT? Could we start with testcases in one of the appendices of FIXProtocol specifications. Or should we start with a blank sheet because HFT design is being done from scratch rather that incrementaly refining / enhancing existing Tag=Value^ FIX and many new / different usecases could be found by taking a “from scratch” approach.

I think the use cases / test cases could be classified into three categories :-

  1. Buy side - the High frequency trader perspective
  2. Order router - the Broker algorithim perspective
  3. Exchange - the high volume trade matching engine perspective

Regards,
Mahesh

7

John,

Thanks for your feedback. I am renaming this branch thread since we have started discussing the scope of our working group in terms on the phases of trade life cycle expected to be worked on by our presently named “High Frequency Trading” working group.

Your comments are preceded with > and my comments are preceded with *** for clarity.

Trading is one phase or activity in portfolio or market-position management. Imagine that a trader has reached a satisfied state. He takes in new information. This information unsettles him. He evaluates whether he should act. He decides to act. He submits an order. He receives order-state information. If executed, he settles his obligation and returns, however briefly, to a satisfied state. Repeat process.

*** Each of the activities “reached a satisfied state”, “takes in new information”, “information unsettles him”, “evaluates whether he should act”, “decides to act”, “submits an order”, “receives order-state information” “settles his obligation” and “returns to a satisfied state” are seperate discrete steps in the trade lifecycle, the “however briefly” being a seperate aspect, each process takes a finite amount of time, its the band in which these amounts of times fall which would decide the high or low frequency of trade. If I am correct, an investor with a buylong-hold-sell investment strategy would not buy/sell as frequently as a commodities / options day trader who buys morning, sells forenoon, buys something else evening, sells in pre-open auction next morning… A 40 year old adult buying a house would spend much more time deciding where to invest his/her lifetime savings compared to a 20 year old buying his/her first car, the shortest time would be a small kid at ToysRFuss buying toys where the kid’s dad is paying with store value cards received as gifts in the kid’s birthday. As you had stated previously http://fixprotocol.org/discuss/read/4bc28170 , high and low frequency is subjective. But I do not think a single protocol would satisy the needs to all users.

An execution report, for example, matters in three operational domains at once: order management, market data, and settlement.

*** But from the Order submitter perspective, would not Order / position management and settlement be more important than market data ?

It may be that FaST is perfect and no further improvement in market data protocols is possible.

*** I never meant there can be no further improvement to Market Data delivery compared to FAST. I am open to further improvements in FAST. FIX Tag=Value^ covered all phases of trade life cycle. My view is that the future architecture would be diverging with Market Data over FAST over UDP, Trade Messaging over HFT over TCP and Settlement / post trade activities happening using FIXML over MQ.

I would like the scope of HFT working group in terms on the phases of trade life cycle expected to be covered to be included as a part of the HFT scope documentation.

Regards,
K. Mahesh

8

[…] My view is that the future architecture would be diverging with Market Data over FAST over UDP, Trade Messaging over HFT over TCP and Settlement / post trade activities happening using FIXML over MQ.
I beg to differ on the last bit “FIXML over MQ”. MQ is a commercial product from a vendor and should not be a target transport. Maybe you are not aware of an initiative for an open messaging middleware called Advanced Message Queuing Protocol (AMQP, www.amqp.org) which is very likely to gain significant traction this year as V1.0 is about to be finalized. Eurex is already using a pre-1.0 version with FIXML for the application layer in the area of risk management. CME and OCC still require their users to have MQ but this might change in the future.
Regards,
Hanno.

9

[ original email was from John Harris - john.harris@bondmart.com ]
Thanks, Mahesh. My replies are preceded with +++.

John,

Thanks for your feedback. I am renaming this branch thread since we have started discussing the scope of our working group in terms on the phases of trade life cycle expected to be worked on by our presently named “High Frequency Trading” working group.

Your comments are preceded with > and my comments are preceded with *** for clarity.

Trading is one phase or activity in portfolio or market-position management. Imagine that a trader has reached a satisfied state. He takes in new information. This information unsettles him. He evaluates whether he should act. He decides to act. He submits an order. He receives order-state information. If executed, he settles his obligation and returns, however briefly, to a satisfied state. Repeat process.

*** Each of the activities “reached a satisfied state”, “takes in new information”, “information unsettles him”, “evaluates whether he should act”, “decides to act”, “submits an order”, “receives order-state information” “settles his obligation” and “returns to a satisfied state” are seperate discrete steps in the trade lifecycle, the “however briefly” being a seperate aspect, each process takes a finite amount of time, its the band in which these amounts of times fall which would decide the high or low frequency of trade. If I am correct, an investor with a buylong-hold-sell investment strategy would not buy/sell as frequently as a commodities / options day trader who buys morning, sells forenoon, buys something else evening, sells in pre-open auction next morning… A 40 year old adult buying a house would spend much more time deciding where to invest his/her lifetime savings compared to a 20 year old buying his/her first car, the shortest time would be a small kid at ToysRFuss buying toys where the kid’s dad is paying with store value cards received as gifts in the kid’s birthday. As you had stated previously http://fixprotocol.org/discuss/read/4bc28170 , high and low frequency is subjective. But I do not think a single protocol would satisy the needs to all users.

+++ Full credit to you for Best Use of Analogy to Brighten the Day. Actually, I fully agree with you on your single-protocol statement, and the general thrust of your comment.

An execution report, for example, matters in three operational domains at once: order management, market data, and settlement.

*** But from the Order submitter perspective, would not Order / position management and settlement be more important than market data ?

+++ I would just say that the need for reliability of message delivery varies by trade phase. Really good reliability is okay for market data. Guaranteed delivery is best advised for post-trade messages.

It may be that FaST is perfect and no further improvement in market data protocols is possible.

*** I never meant there can be no further improvement to Market Data delivery compared to FAST. I am open to further improvements in FAST. FIX Tag=Value^ covered all phases of trade life cycle. My view is that the future architecture would be diverging with Market Data over FAST over UDP, Trade Messaging over HFT over TCP and Settlement / post trade activities happening using FIXML over MQ.

I would like the scope of HFT working group in terms on the phases of trade life cycle expected to be covered to be included as a part of the HFT scope documentation.

+++ Agreed.

Regards,
K. Mahesh