Validation via Schematron

, ,
1

Imported from previous forum

2

[ original email was from Greg Malatestinic - greg.malatestinic@reuters.com ]
Has anyone had any experience using Schematron to validate FIXML messages? I am using Schematron in conjuction with XML Schema to validate algorithmic parameters of a message. But, I think it might be useful to expand this type of validation to the entire message. In other words, create a Schematron document that covers all the tags. Please chime in if you know of any traps/pitfalls regarding this method.
Thanks.
-Greg

3

[ original email was from Robert Stowsky - rstowsky@brookpath.com ]
Hi Greg,

I was a member of the ISDA/FpML Validation Working Group and also the FIXML Schema WG. For FpML we established a set of business validation rules and attempted to implement them in Schematron and CLIX. For our purposes, Schematron was not able to handle the complexity of some of the business rules for OTC Derivative Contracts, however CLIX was. Areas where we ran into trouble for FpML usually involved verifying intermediate payment date schedules given a start and end date.

This may not be an issue for FIXML as listed assets are well specified either by an exchange or the issuer. Determining that the proper elements and attributes are included for various messages should be straight forward. Actually verifying the data values for those elements, or attributes, may not be. Schematron seems to fall short if computation, or extrapolation, based on data is required.

Robert Stowsky

Co-Chair FIX GDC Technical Subcommittee

Has anyone had any experience using Schematron to validate FIXML
messages? I am using Schematron in conjuction with XML Schema to
validate algorithmic parameters of a message. But, I think it might be
useful to expand this type of validation to the entire message. In other
words, create a Schematron document that covers all the tags. Please
chime in if you know of any traps/pitfalls regarding this method.
Thanks. -Greg

4

Hi Greg,

I was a member of the ISDA/FpML Validation Working Group and also the
FIXML Schema WG. For FpML we established a set of business validation
rules and attempted to implement them in Schematron and CLIX. For our
purposes, Schematron was not able to handle the complexity of some of
the business rules for OTC Derivative Contracts, however CLIX was. Areas
where we ran into trouble for FpML usually involved verifying
intermediate payment date schedules given a start and end date.

This may not be an issue for FIXML as listed assets are well specified
either by an exchange or the issuer. Determining that the proper
elements and attributes are included for various messages should be
straight forward. Actually verifying the data values for those elements,
or attributes, may not be. Schematron seems to fall short if
computation, or extrapolation, based on data is required.

Robert Stowsky

Co-Chair FIX GDC Technical Subcommittee

Has anyone had any experience using Schematron to validate FIXML
messages? I am using Schematron in conjuction with XML Schema to
validate algorithmic parameters of a message. But, I think it might be
useful to expand this type of validation to the entire message. In
other words, create a Schematron document that covers all the tags.
Please chime in if you know of any traps/pitfalls regarding this
method. Thanks. -Greg

Greg,

As the current chair of the FpML Validation Working Group I can re-iterate what Robert has said. As part of our mandate we are looking into a adopting new rule language to codify our semantic validation rules. Schematron is a possibility but as Greg points out this will only cover about 90% of rules. NRL is another possibility and I’ve had conversations with Christian Netwich about this. Another possibility is OCL which we will be evaluating at some point in the future.

My personal belief is that none of these languages will ever be human readable and yet still expressive enough to define all rules one could dream up.

I’ve talked to some of the ISO20022 guys about this as well and we have agreed to share the approach we take. I hope we can include FIXML in this collaboration as well.

Cheers,

Simon Heinrich
Product Development Director, C24
Chair, FpML Validation Working Group

5

[ original email was from Greg Malatestinic - greg.malatestinic@reuters.com ]
Simon/Robert,

Thanks for the information about FpML validation. The FPL Algorithmic WG is currently evaluating schematron (and now also NRL). We are striving for a solution that has the widest acceptance from the OMS vendor community. So ease of implementation is critical for us.

I have found that validation with schematron is very easy to implement - just a couple of transforms - and does much of what we need. But, from what I’ve read so far, CLIX seems to have more capabilities than schematron. I’d like to take a look at that as well. Can someone direct me to a resource on CLIX describing how it might be implemented?

Thanks.

-Greg

6

Hi Greg,

I was a member of the ISDA/FpML Validation Working Group and also the
FIXML Schema WG. For FpML we established a set of business validation
rules and attempted to implement them in Schematron and CLIX. For our
purposes, Schematron was not able to handle the complexity of some of
the business rules for OTC Derivative Contracts, however CLIX was.
Areas where we ran into trouble for FpML usually involved verifying
intermediate payment date schedules given a start and end date.

This may not be an issue for FIXML as listed assets are well specified
either by an exchange or the issuer. Determining that the proper
elements and attributes are included for various messages should be
straight forward. Actually verifying the data values for those
elements, or attributes, may not be. Schematron seems to fall short if
computation, or extrapolation, based on data is required.

Robert Stowsky

Co-Chair FIX GDC Technical Subcommittee

Has anyone had any experience using Schematron to validate FIXML
messages? I am using Schematron in conjuction with XML Schema to
validate algorithmic parameters of a message. But, I think it might
be useful to expand this type of validation to the entire message.
In other words, create a Schematron document that covers all the
tags. Please chime in if you know of any traps/pitfalls regarding
this method. Thanks. -Greg

Greg,

As the current chair of the FpML Validation Working Group I can re-
iterate what Robert has said. As part of our mandate we are looking into
a adopting new rule language to codify our semantic validation rules.
Schematron is a possibility but as Greg points out this will only cover
about 90% of rules. NRL is another possibility and I’ve had
conversations with Christian Netwich about this. Another possibility is
OCL which we will be evaluating at some point in the future.

My personal belief is that none of these languages will ever be human
readable and yet still expressive enough to define all rules one
could dream up.

I’ve talked to some of the ISO20022 guys about this as well and we have
agreed to share the approach we take. I hope we can include FIXML in
this collaboration as well.

Cheers,

Simon Heinrich Product Development Director, C24 Chair, FpML Validation
Working Group

Simon,

We created NRL because OCL is not a business facing language. Whilst any machine processable language will not be poetic, NRL is very easily understood by business types.

Suggest you look at the examples Christian has created for FpML or the examples we have produced for the FIX Rep WG.

In either case the main issue with readability is the schema not the NRL grammar…

Should we consider combing the Algo and Rep WG work as we seem to be covering the same ground ?

Regards,

Rob

7

[ original email was from Greg Malatestinic - Greg.Malatestinic@reuters.com ]
Hi all,

The Algo WG is attemting to create a markup to describe the parameters of an algorithmic trading screen. OMS providers would code their systems so that they could import an XML instance and render their order entry screens based on the info in that instance. If this could be accomplished then the OMS provider would never need to create a new release just to support the latest algorithm. The ideal workflow would be:

  1. Algo provider creates an XML file that describes the parameters of an algorithm (parameter name, type, fix tag, , preferred GUI component, default value, …)

  2. XML file is sent to OMS providers (or directly to OMS users) where it is imported into their systems.

  3. Clients log onto their OMS - and Viola! The have a new algorithm to use. No upgrade was needed because the OMS renders the screen on-the-fly.

At this time we are researching how the values entered by the OMS user can be validated. We want to include validation instructions within the XML file. Schematron seems to work very well for this purpose. Using a Java Transformer object I can easily check the validity in only a few lines of code. But from what I’ve read, CLiX provides more capabilities than schematron. So I’d like to explore it in more detail.

Can anyone tell me how I can programmatically perform a CLiX validation? Does it follow the XSL tranform model as schematron does or does it require an API? And if I need an API what are my options?

Thanks for all the help so far.

Cheers.

-Greg

8

Has anyone had any experience using Schematron to validate FIXML
messages? I am using Schematron in conjuction with XML Schema to
validate algorithmic parameters of a message. But, I think it might be
useful to expand this type of validation to the entire message. In other
words, create a Schematron document that covers all the tags. Please
chime in if you know of any traps/pitfalls regarding this method.
Thanks. -Greg

I am working with Lisa Taikitsadaporn to create a set of example rules using the ‘Natural Rule Language’ This is an open source specification directly descended from Clix (created by Christian Nentwich and Rob James). It is more expressive than Clix and closer to English.

The language specification, user guide and a parser are available at http://nrl.sourceforge.net/.

If you would like to see the example rules created for the Repository WG then I’ll be happy to share these with you.

Greg - is there another WG working on this ? Or are you working on this alone ? If so I would suggest you ask Kevin to add you to the Rep WG.

  • Regards,
    Rob