Imported from previous forum
Hi,
Recently,my manager give me a task to test the Stability and performance of our FIX engine while I have a little experience. IS there someone can give me some advices, and which aspects should I pay attention to？
Thanks in advance
Lswallow
[ original email was from Jeffrey Croft - jeff.croft@plusmarketsgroup.com ]
A good place to start if you are going to test from the ground up would be the ‘FIX Session-level Test Cases and Expected Behaviours’ which you can find in the specification (vol 2). They were written some time ago and you might find that your engine does not pass all these tests but you will be able to understand your current session layer behaviour.
Not sure exactly what you mean by ‘stability’ but after the session layer I would start to build up a library of functional tests that validate the FIX messages that you are supporting. This is where your ROE will guide your testing. You will have conditional fields in each message so start by testing a minimum field set for each message and then build up variants supporting all fields you support upto fully-loaded messages. There are many variants of each message depending on which enumerations you support and comprehensive testing should test support for each combination.
Hope this helps but your question may have been more about non-functional test approach in which case please make that clear.
Jeff
Hi,
Recently,my manager give me a task to test the Stability and performance of our FIX engine while I have a little experience. IS there someone can give me some advices, and which aspects should I pay attention to？
Thanks in advance
Lswallow
Thank you for your advice Jeff.
I’m sorry I cann’t descrip the problem clearly. You are righ that my question is about non-functional test approach. By the way the word ‘stablity’ should be ‘stability’.For example,whether the connection between FIXengine and client can continue for 7 days.
Lswallow
A good place to start if you are going to test from the ground up would be the ‘FIX Session-level Test Cases and Expected Behaviours’ which you can find in the specification (vol 2). They were written some time ago and you might find that your engine does not pass all these tests but you will be able to understand your current session layer behaviour.
Not sure exactly what you mean by ‘stability’ but after the session layer I would start to build up a library of functional tests that validate the FIX messages that you are supporting. This is where your ROE will guide your testing. You will have conditional fields in each message so start by testing a minimum field set for each message and then build up variants supporting all fields you support upto fully-loaded messages. There are many variants of each message depending on which enumerations you support and comprehensive testing should test support for each combination.
Hope this helps but your question may have been more about non-functional test approach in which case please make that clear.
JeffHi,
Recently,my manager give me a task to test the Stability and performance of our FIX engine while I have a little experience. IS there someone can give me some advices, and which aspects should I pay attention to？
Thanks in advance
Lswallow
[ original email was from Jeffrey Croft - jeff.croft@plusmarketsgroup.com ]
Not sure there are many environments out there where you need a FIX session to stay up that long. Most operations are 5/7 with trading systems taken down at the weekend. Most systems would also logoff users at a scheduled time each day when the market closes or some some short time after.
Theoretically your FIX session should stay heartbeating forever. Performance is a little different and on a generic level your testing would stem from what sort of requirements you have, this must be the starting point.
Thank you for your advice Jeff.
I’m sorry I cann’t descrip the problem clearly. You are righ that my question is about non-functional test approach. By the way the word ‘stablity’ should be ‘stability’.For example,whether the connection between FIXengine and client can continue for 7 days.
LswallowA good place to start if you are going to test from the ground up would be the ‘FIX Session-level Test Cases and Expected Behaviours’ which you can find in the specification (vol 2). They were written some time ago and you might find that your engine does not pass all these tests but you will be able to understand your current session layer behaviour.
Not sure exactly what you mean by ‘stability’ but after the session layer I would start to build up a library of functional tests that validate the FIX messages that you are supporting. This is where your ROE will guide your testing. You will have conditional fields in each message so start by testing a minimum field set for each message and then build up variants supporting all fields you support upto fully-loaded messages. There are many variants of each message depending on which enumerations you support and comprehensive testing should test support for each combination.
Hope this helps but your question may have been more about non-functional test approach in which case please make that clear.
JeffHi,
Recently,my manager give me a task to test the Stability and performance of our FIX engine while I have a little experience. IS there someone can give me some advices, and which aspects should I pay attention to？
Thanks in advance
Lswallow