Imported from previous forum
[ original email was from Ryan Pierce (FPL Technical Director) - ryan.pierce@fixprotocol.org ]
Resend request optimizations
Application errors causing DKT
Separating flows over FIX connections by type
Network vendors, cost, per session charges and optimization
Sequencing and parallelism – processing one item before another
FIXT.1.1 possible reduced performance
Benchmarking / Performance Analysis – how to compare apples to apples
Order ack vs. IOC fill, IOC cancel, order cancel, order change, etc.
Clock skew
Use of packet sniffing to measure benchmarks as opposed to measuring in software
Persistence – best practices, how it affects benchmarks
Communication between FIX engine and OMS / Exchange engine
Using similar state machines (ClOrdID, OrderID) to eliminate costly translations in Gateway
In-process, out-of-process
Middleware, guaranteed message delivery
Encoding / decoding for internal representations
Networking – firewalls, proxies, etc.
Threading, parallelism
[ original email was from Clive Browning - clive.browning@rapidaddition.co.uk ]
Could you expand on what was meant by the Clock Skew item. Is that relating to processor clock speeds?
Many thanks
Clive
Resend request optimizations Application errors causing DKT
Separating flows over FIX connections by type Network vendors, cost, per
session charges and optimization Sequencing and parallelism –
processing one item before anotherFIXT.1.1 possible reduced performance
Benchmarking / Performance Analysis – how to compare apples to apples
Order ack vs. IOC fill, IOC cancel, order cancel, order change, etc.
Clock skew Use of packet sniffing to measure benchmarks as opposed to
measuring in softwarePersistence – best practices, how it affects benchmarks
Communication between FIX engine and OMS / Exchange engine Using similar
state machines (ClOrdID, OrderID) to eliminate costly translations in
Gateway In-process, out-of-process Middleware, guaranteed message
delivery Encoding / decoding for internal representationsNetworking – firewalls, proxies, etc.
Threading, parallelism
By clock skew I meant the fact that clocks on different servers are not going to be perfectly in sync. Thus if you are attempting to measure latency between a message being sent from one server to another, you cannot just look at local log file timestamps. Most common configurations for NTP cannot keep skew levels down to the sub 1-ms level that is needed, thus other approaches are needed.
Could you expand on what was meant by the Clock Skew item. Is that
relating to processor clock speeds?Many thanks
Clive
Resend request optimizations Application errors causing DKT
Separating flows over FIX connections by type Network vendors, cost,
per session charges and optimization Sequencing and parallelism –
processing one item before anotherFIXT.1.1 possible reduced performance
Benchmarking / Performance Analysis – how to compare apples to
apples Order ack vs. IOC fill, IOC cancel, order cancel, order change,
etc. Clock skew Use of packet sniffing to measure benchmarks as
opposed to measuring in softwarePersistence – best practices, how it affects benchmarks
Communication between FIX engine and OMS / Exchange engine Using
similar state machines (ClOrdID, OrderID) to eliminate costly
translations in Gateway In-process, out-of-process Middleware,
guaranteed message delivery Encoding / decoding for internal
representationsNetworking – firewalls, proxies, etc.
Threading, parallelism
Interesting topics. Each question Ryan caused a pain when I was designing our FIX engine.
Hopefully i finally found ways to solve all the problems mentioned.
Resend request optimizations Application errors causing DKT
Separating flows over FIX connections by type Network vendors, cost, per
session charges and optimization Sequencing and parallelism –
processing one item before anotherFIXT.1.1 possible reduced performance
Benchmarking / Performance Analysis – how to compare apples to apples
Order ack vs. IOC fill, IOC cancel, order cancel, order change, etc.
Clock skew Use of packet sniffing to measure benchmarks as opposed to
measuring in softwarePersistence – best practices, how it affects benchmarks
Communication between FIX engine and OMS / Exchange engine Using similar
state machines (ClOrdID, OrderID) to eliminate costly translations in
Gateway In-process, out-of-process Middleware, guaranteed message
delivery Encoding / decoding for internal representationsNetworking – firewalls, proxies, etc.
Threading, parallelism
I’m looking forward to reading your comments about how you solved the problems.
/Rolf
Interesting topics. Each question Ryan caused a pain when I was
designing our FIX engine. Hopefully i finally found ways to solve all
the problems mentioned.
Well, we publish the entire system and full documentation for unrestricted for evaluation purpose. You are welcome to give it a try.
I’m looking forward to reading your comments about how you solved
the problems./Rolf
Interesting topics. Each question Ryan caused a pain when I was
designing our FIX engine. Hopefully i finally found ways to solve all
the problems mentioned.
[ original email was from jim whitehead - jwhitehead@latentzero.com ]
Just to recap, I think there are potentially at least two areas of connectivity here
i) Direct Connectivity with an execution venue (Eg sell side --> exchange)
ii) Buyside --> Sellside
I work on the buyside and I believe its fair to say that most connectivity buyside systems ship with a third party FIX Engine so I’m going to be limited in how I can help out with regard to the optimisation of FIX Engines. I can however try and encourage troops from Fidessa to join the group.
My interest is definitely around best practise and optimisation of a system but unfortunately that kind of concept is difficult to benchmark and so I don’t know how comfortably it sits alongside some of the other subjects discussed, or whether it would be appropriate to be the subject of a seperate whitepaper.
Here are some ideas
i) As previously mentioned there is a lack of clarity on the best way of hitting a broker for several workflows. For example should equity cash, program, dma/sor and algo flow all go over one line. Theoretically, it should be quite simple for a broker or hub to be able route that, which is great as it keeps the costs down, but is that really best practise or should we be proposing to separate FIX connections out?
ii) A slightly obvious one, but if you’ve got a distributed system over several servers its quite surprising how many people don’t necessarily synch the server times acroos all of them to an atomic clock or the such like, which then makes troubleshooting via logfiles, very painful.
iii)The amount of messages that can be generated as a result of a resend request or as DKTs can flood a fix session and affect performance.
iv) For DMA flow, should it really be necessary to wait for a cancel or cancel/replace ack?
v) several non equity liquidity venues prefer FIX connections via the internet than over a lease line, encrypted using an open source proxy, which I personally have an issue with.
vi) FIX Engines often have a High Availibility mode, should we be looking to benchmark this kind of setup againast an out of the box setup
vii) middleware and OMS internal messaging can affect the perception of a FIX connections speed, switching on guaranteed message delivery while giving certain benefits and greatly impede performance.
Not sure where these fit in, but all input is appreciated
Thanks
Jim
[ original email was from jim whitehead - jwhitehead@latentzero.com ]
I’ve been keeping my colleagues within Fidessa (as opposed to FLZ) upto date with the points raised, they’ve given some feedback to pass on, again its operational rather than strictly developmental as its felt that by examining sample performance related screwups that have occurred in FIX systems it may be possible to identifty common ‘gotchas’
Two specific examples which were given were:
- How messages are decoded - FIX is a self describing message format requiring a parser to lift field value pairs out.
- Recovery - FIX recovery screwups are common in my experience. Replaying large numbers of messages that either side has already received.
HTH
Jim
Resend request optimizations Application errors causing DKT
Separating flows over FIX connections by type Network vendors, cost, per
session charges and optimization Sequencing and parallelism –
processing one item before anotherFIXT.1.1 possible reduced performance
Benchmarking / Performance Analysis – how to compare apples to apples
Order ack vs. IOC fill, IOC cancel, order cancel, order change, etc.
Clock skew Use of packet sniffing to measure benchmarks as opposed to
measuring in softwarePersistence – best practices, how it affects benchmarks
Communication between FIX engine and OMS / Exchange engine Using similar
state machines (ClOrdID, OrderID) to eliminate costly translations in
Gateway In-process, out-of-process Middleware, guaranteed message
delivery Encoding / decoding for internal representationsNetworking – firewalls, proxies, etc.
Threading, parallelism
[ original email was from Clive Browning - clive.browning@rapidaddition.co.uk ]
We see FIX as a ‘mostly’ self-describing message format. But the armoured tags (length-refers tags), repeating groups and encoded tags do require a data-dictionary to enable the messages to be parsed fully. This is one area where we get a performance hit on parsing.
We have also come across session recovery screw-ups, where engines connect, resend-request for messages, which then causes a subsequent disconnection, leading to another connection with another resend-request. Manual intervention is then required to reset both sides message sequence numbers. This is generally where one of the counter party engines resend request processing has not been fully tested against all scenarios.
I’ve been keeping my colleagues within Fidessa (as opposed to FLZ) upto
date with the points raised, they’ve given some feedback to pass on,
again its operational rather than strictly developmental as its felt
that by examining sample performance related screwups that have occurred
in FIX systems it may be possible to identifty common ‘gotchas’Two specific examples which were given were:
- How messages are decoded - FIX is a self describing message format
requiring a parser to lift field value pairs out.- Recovery - FIX recovery screwups are common in my experience.
Replaying large numbers of messages that either side has
already received.HTH Jim
Resend request optimizations Application errors causing DKT
Separating flows over FIX connections by type Network vendors, cost,
per session charges and optimization Sequencing and parallelism –
processing one item before anotherFIXT.1.1 possible reduced performance
Benchmarking / Performance Analysis – how to compare apples to
apples Order ack vs. IOC fill, IOC cancel, order cancel, order change,
etc. Clock skew Use of packet sniffing to measure benchmarks as
opposed to measuring in softwarePersistence – best practices, how it affects benchmarks
Communication between FIX engine and OMS / Exchange engine Using
similar state machines (ClOrdID, OrderID) to eliminate costly
translations in Gateway In-process, out-of-process Middleware,
guaranteed message delivery Encoding / decoding for internal
representationsNetworking – firewalls, proxies, etc.
Threading, parallelism
[ original email was from Ryan Pierce (FPL Technical Director) - ryan.pierce@fixprotocol.org ]
> We see FIX as a ‘mostly’ self-describing message format. But the
armoured tags (length-refers tags), repeating groups and encoded tags do
require a data-dictionary to enable the messages to be parsed fully.
This is one area where we get a performance hit on parsing.
FYI, there is a draft document that I worked on (a year ago?) to draft a new version of the FIXT transport (i.e. 1.2 or 2.0, I’m not sure what we decided) which addresses all of these, and enables parsing without any data dictionary at all. It used tag-some delimiter-length-equals-value-SOH for encoded tags, and delimiters other than SOH to define repeating groups.
This hasn’t really gone anywhere. We can’t push another FIXT version at this time, with SP2 coming out. But it would be useful to address this in the future, and I’m wondering if our WG can have some involvement with this.
Ryan,
Can I have a copy of the draft document that you are talking about in your post? Could I request you to email it to me at
or provide me with the web URL?
Regards,
K. Mahesh
We see FIX as a ‘mostly’ self-describing message format. But the
armoured tags (length-refers tags), repeating groups and encoded tags
do require a data-dictionary to enable the messages to be parsed
fully. This is one area where we get a performance hit on parsing.FYI, there is a draft document that I worked on (a year ago?) to draft
a new version of the FIXT transport (i.e. 1.2 or 2.0, I’m not sure
what we decided) which addresses all of these, and enables parsing
without any data dictionary at all. It used tag-some delimiter-length-equals-value-
SOH for encoded tags, and delimiters other than SOH to define
repeating groups.This hasn’t really gone anywhere. We can’t push another FIXT version at
this time, with SP2 coming out. But it would be useful to address this
in the future, and I’m wondering if our WG can have some involvement
with this.
[ original email was from jim whitehead - jwhitehead@latentzero.com ]
I meant to bring this up on the call yesterday but got dragged off into a production issue.
Possibly a new subject for the document.
Given that a lot of trading still takes place over 4.2 sessions and that a lot of those sessions will be over FIX Hubs (NYFIX,TOR etc) what is the appropriate way of signalling to a firm, using a 4.2 hub session, that their trading counterparty on the other side of the hub is up or down?
Just to clarify I’m referring to a way of doing this using a FIX message itself not ‘we’ve got a gui that you can monitor on your desktop to see what the status is’ type answers, at our shop we using trading session status messages (35=h) I wondered if anyone else had any thoughts around this.
Regards
Jim
Resend request optimizations Application errors causing DKT
Separating flows over FIX connections by type Network vendors, cost, per
session charges and optimization Sequencing and parallelism –
processing one item before anotherFIXT.1.1 possible reduced performance
Benchmarking / Performance Analysis – how to compare apples to apples
Order ack vs. IOC fill, IOC cancel, order cancel, order change, etc.
Clock skew Use of packet sniffing to measure benchmarks as opposed to
measuring in softwarePersistence – best practices, how it affects benchmarks
Communication between FIX engine and OMS / Exchange engine Using similar
state machines (ClOrdID, OrderID) to eliminate costly translations in
Gateway In-process, out-of-process Middleware, guaranteed message
delivery Encoding / decoding for internal representationsNetworking – firewalls, proxies, etc.
Threading, parallelism
Jim,
You are right. I have seen Trading Session Status [type ‘h’] used for this purpose. At the start of our connection to Hub, Trading Session Status is received for each counterparty connected thru the Hub. When the status of connection between a counterparty and Hub changes, a Trading Session Status is received by us for each such change in connection status.
Regards,
Mahesh
I meant to bring this up on the call yesterday but got dragged off into
a production issue.Possibly a new subject for the document.
Given that a lot of trading still takes place over 4.2 sessions and
that a lot of those sessions will be over FIX Hubs (NYFIX,TOR etc) what
is the appropriate way of signalling to a firm, using a 4.2 hub
session, that their trading counterparty on the other side of the hub
is up or down?Just to clarify I’m referring to a way of doing this using a FIX
message itself not ‘we’ve got a gui that you can monitor on your
desktop to see what the status is’ type answers, at our shop we using
trading session status messages (35=h) I wondered if anyone else had
any thoughts around this.Regards Jim
Resend request optimizations Application errors causing DKT
Separating flows over FIX connections by type Network vendors, cost,
per session charges and optimization Sequencing and parallelism –
processing one item before anotherFIXT.1.1 possible reduced performance
Benchmarking / Performance Analysis – how to compare apples to
apples Order ack vs. IOC fill, IOC cancel, order cancel, order change,
etc. Clock skew Use of packet sniffing to measure benchmarks as
opposed to measuring in softwarePersistence – best practices, how it affects benchmarks
Communication between FIX engine and OMS / Exchange engine Using
similar state machines (ClOrdID, OrderID) to eliminate costly
translations in Gateway In-process, out-of-process Middleware,
guaranteed message delivery Encoding / decoding for internal
representationsNetworking – firewalls, proxies, etc.
Threading, parallelism
[ original email was from Clive Browning - clive.browning@rapidaddition.co.uk ]
Hi Jim
We have seen the use of custom versions of the FIX.4.4 NetworkCounterpartySystemStatusRequest (BC) and NetworkCounterpartySystemStatusResponse (BD) messages retrofitted to FIX.4.2 hubs.
This may be an option if your hub can support custom messages in your 4.2 sessions. If that is possible then you have the advantage that you are using the appropriate message if the hub moves forward to FIX.4.4 or later at some point.
Clive
Jim,
You are right. I have seen Trading Session Status [type ‘h’] used for
this purpose. At the start of our connection to Hub, Trading Session
Status is received for each counterparty connected thru the Hub. When
the status of connection between a counterparty and Hub changes, a
Trading Session Status is received by us for each such change in
connection status.Regards, Mahesh
I meant to bring this up on the call yesterday but got dragged off
into a production issue.Possibly a new subject for the document.
Given that a lot of trading still takes place over 4.2 sessions and
that a lot of those sessions will be over FIX Hubs (NYFIX,TOR etc)
what is the appropriate way of signalling to a firm, using a 4.2 hub
session, that their trading counterparty on the other side of the hub
is up or down?Just to clarify I’m referring to a way of doing this using a FIX
message itself not ‘we’ve got a gui that you can monitor on your
desktop to see what the status is’ type answers, at our shop we using
trading session status messages (35=h) I wondered if anyone else had
any thoughts around this.Regards Jim
Resend request optimizations Application errors causing DKT
Separating flows over FIX connections by type Network vendors, cost,
per session charges and optimization Sequencing and parallelism –
processing one item before anotherFIXT.1.1 possible reduced performance
Benchmarking / Performance Analysis – how to compare apples to
apples Order ack vs. IOC fill, IOC cancel, order cancel, order
change, etc. Clock skew Use of packet sniffing to measure benchmarks
as opposed to measuring in softwarePersistence – best practices, how it affects benchmarks
Communication between FIX engine and OMS / Exchange engine Using
similar state machines (ClOrdID, OrderID) to eliminate costly
translations in Gateway In-process, out-of-process Middleware,
guaranteed message delivery Encoding / decoding for internal
representationsNetworking – firewalls, proxies, etc.
Threading, parallelism
[ original email was from jim whitehead - jwhitehead@latentzero.com ]
That’s a good point but I’d be wary of suggesting to a wider audience that best practice for a 42 session is to retrofit a 44 tag.
Hi Jim
We have seen the use of custom versions of the FIX.4.4
NetworkCounterpartySystemStatusRequest (BC) and
NetworkCounterpartySystemStatusResponse (BD) messages retrofitted to
FIX.4.2 hubs.This may be an option if your hub can support custom messages in your
4.2 sessions. If that is possible then you have the advantage that you
are using the appropriate message if the hub moves forward to FIX.4.4 or
later at some point.Clive
Jim,
You are right. I have seen Trading Session Status [type ‘h’] used for
this purpose. At the start of our connection to Hub, Trading Session
Status is received for each counterparty connected thru the Hub. When
the status of connection between a counterparty and Hub changes, a
Trading Session Status is received by us for each such change in
connection status.Regards, Mahesh
I meant to bring this up on the call yesterday but got dragged off
into a production issue.Possibly a new subject for the document.
Given that a lot of trading still takes place over 4.2 sessions and
that a lot of those sessions will be over FIX Hubs (NYFIX,TOR etc)
what is the appropriate way of signalling to a firm, using a 4.2 hub
session, that their trading counterparty on the other side of the
hub is up or down?Just to clarify I’m referring to a way of doing this using a FIX
message itself not ‘we’ve got a gui that you can monitor on your
desktop to see what the status is’ type answers, at our shop we
using trading session status messages (35=h) I wondered if anyone
else had any thoughts around this.Regards Jim
Resend request optimizations Application errors causing DKT
Separating flows over FIX connections by type Network vendors,
cost, per session charges and optimization Sequencing and
parallelism – processing one item before anotherFIXT.1.1 possible reduced performance
Benchmarking / Performance Analysis – how to compare apples to
apples Order ack vs. IOC fill, IOC cancel, order cancel, order
change, etc. Clock skew Use of packet sniffing to measure
benchmarks as opposed to measuring in softwarePersistence – best practices, how it affects benchmarks
Communication between FIX engine and OMS / Exchange engine Using
similar state machines (ClOrdID, OrderID) to eliminate costly
translations in Gateway In-process, out-of-process Middleware,
guaranteed message delivery Encoding / decoding for internal
representationsNetworking – firewalls, proxies, etc.
Threading, parallelism