Imported from previous forum
Hi All,
Currently, we have 2 major problem areas in our FIX development:
- whenever we have change in our FIX application that may affects our buy/sell side FIX parties (e.g. as minor as correcting passing/not passing a tag value), besides internal testing, we need to invite a whole lot of FIX parties for regression test. The tasks of the regression test is so time consuming.
- Whether in production connection or certification testing on new connection, it is a headache to discover non-conformance area to FIX specification piece by piece, ending up impacting the production trade or timeline for go-live on new connection.
A year ago, we try to find an automated testing tool which can alleviate our burden on FIX testing. However, the result is not good:
Requirement #1
Validation of Conformance to FIX Standard (e.g. version 4.2)
a) Equipped with semantic based validation, e.g. correct values of orderStat, based on precedence
b) Be flexible enough to overrided/bypass the rule according to customization of own / our FIX parties’ application
c) Have a rich set of scenario for validation (at least should have all cases from FIX spec 4.2 Appendix D)
Evaluation Result for Requirement #1
The tools we tried:
- does not have built in test scenario for FIX 4.2 verification [e.g. semantic based validation, e.g. correct values of orderStat, based on precedence). We will need to create scenario as in FIX spec Appendix D to cover the whole scope to verify FIX conformance
- there are only limited number of validation profile by default
- unable to validate calculated tag in Execution Report e.g. AvgPx (6), CumQty (14), LeavesQty(151)
Requirement #2
Validation by Production Log
a) Via import of production FIX log, the tool will help to do some automatic regression test: i.e. check if our buy-side / sell-side FIX application still behave correctly (i.e. as in current production) after the change.
e.g. this may be done by replaying the FIX order from production log and comparing the execution report received against that in log.
Evaluation Result for Requirement #2
For the tool we tried, all the message (orders, execution report) should be in exactly the same sequence as in the log. That’s imply we have to carefully subdivide / extract units of scenario from the production logs. Otherwise, we can reproduce the response as in the log via our FIX application to arrive with the same execution report as in the log for comparsion.
To extract scenario, it would takes time to select the useful cases
Recently, we are revisiting to see if there is any better automated FIX testing tool in the market which can somehow achieve part/most of our problems.
So, I am making a post here just to see if there is any better tools that you guys/gals have tried and can be recommended.
Thanks very much!
Mike
FIX testing tools are listed at
you should be able to contact them and get a trial version which you could evaluate
Good response Mahesh - thanks. Mike if you are looking for direct responses and posts for product research or possible acquisition - may I suggest you contact the Products Discussion forum. Vendors are permitted to discuss in that venue.
But I think Mahesh’s suggestion is a good one.
FIX testing tools are listed at
http://fixprotocol.org/products/10
you should be able to contact them and get a trial version which you
could evaluate
Good response Mahesh - thanks. Mike if you are looking for direct
responses and posts for product research or possible acquisition - may I
suggest you contact the Products Discussion forum. Vendors are permitted
to discuss in that venue.But I think Mahesh’s suggestion is a good one.
FIX testing tools are listed at
http://fixprotocol.org/products/10
you should be able to contact them and get a trial version which you
could evaluate
Mike,
I would second Jim and Mahesh’s recommendations, and I would also suggest that you provide your email address in these kinds of posts so that vendors could engage you directly in response to your questions or concerns.
Cheers,
Russ Curry