Imported from previous forum
All,
I’d like to start a discussion on how we create a performance baseline to work from when we analyse different alternatives in terms of encoding and decoding costs. As a starting point I think we should compare each new proposal with (at least):
- Binary fixed size messages
- FAST encoded messages
- Classic FIX tag-value messages
Test Data
Georges has used a very simple message for the examples in his proposal. We need to select our test message populations with care so we can be reasonably certain that we have relevant test data.
Implementation Complexity
Judging from the discussions way bach when we developed FAST, I would assume that we will have quite a few discussions on performance versus complexity. Any suggestions on how we should formulate complexity evaluation criteria would be of great interest.
Best,
Rolf
[ original email was from Philip Beevers - philip.beevers@royalblue.com ]
My thoughts were that we would get hold of some real production traffic for a typical session with a FIX-based MTF. We’d then build a test harness which takes the place of the MTF, and, given suitable orders/amends/cancels, would build corresponding acknowledgements, executions etc.
The logic of the test harness would essentially just be copying fields from incoming to outgoing message, but assuming this is done using suitable type conversions (i.e. int type fields are pulled out of incoming FIX messages as ints rather than strings) it will give a good impression of the cost of unpacking and rebuilding different types of message. The benchmark for each message type would be the time between egress and ingress on the sender - i.e. the client, rather than at the MTF itself. This measure would include processing time, and networking too.
The next level up would be to generalise further into business scenarios rather than specific messages (thus allowing comparisons on the speed of semantic changes), but to begin, using the format of captured production messages seems like a good baseline.