Imported from previous forum
I want to use FIML as a repository and select only the mesages I require to build a schema without all the other stuff.
Has anyone tried this?
Reagrds
George
[ original email was from John Unwin - john@unwin.co.uk ]
George,
I do this for our routing system, so I create a high level schema that pulls in just the bits I want see below:
<xs:schema targetNamespace=“http://www.fixprotocol.org/FIXML-4-4” xmlns:xs=“http://www.w3.org/2001/XMLSchema” xmlns=“http://www.fixprotocol.org/FIXML-4-4” elementFormDefault=“qualified” attributeFormDefault=“unqualified”>
<xs:include schemaLocation=“fixml-quotation-impl-4-4.xsd”/>
<xs:include schemaLocation=“fixml-marketdata-impl-4-4.xsd”/>
<xs:include schemaLocation=“fixml-securitystatus-impl-4-4.xsd”/>
<xs:include schemaLocation=“fixml-order-impl-4-4.xsd”/>
</xs:schema>
I then run this against our data bindings package to produced C# and C++ classes that are used in the app.
Hope this helps
John
I want to use FIML as a repository and select only the mesages I require
to build a schema without all the other stuff.Has anyone tried this?
Reagrds George
[ original email was from Jim Northey - jnorthey@jandj.com ]
Thank you for the response with real usage as opposed to our sometimes speculative responses - it is greatly appreciated.
To further John’s point. We chose to add “wrapper” schema files if you will for life cycle sections (pretrade, trade, posttrade), categories (order, marketdata, position management, etc.), and messages to permit folks to only have to pull in what they need. We became very well aware of the cost of validation via schema during the process of creating the schema for optimized FIXML (4.4). We didn’t optimize validation (humor).
Taking that point a step further, some of the more active GTC folks (Global Technical Committee of FPL) have been mulling around and experiementing with the creation of a run time schema that would minimize abstractions and layers in hopes of improving (reducing) validation times. We would be interested in hearing back from folks if this is a valuable way to spend valuable time of our more active volunteers.
George,
<?xml version="1.0" encoding="UTF-8"?>
I do this for our routing system, so I create a high level schema that pulls in just the bits I want see below:
John,
Thanks for this, I’m equally concerned that by using the high level schema method we still pull in an awful of other lower level stuff. For example all 1800 components and all 18,000 field definitions.
I need to design an Oracle database schema to store the clients defined subset of FIXML and (for example) they only use about 30% of the instrument attributes. While I can currently handle storing the resulting XML automatically and fairly elegantly using metadata and about 200 lines of pl/sql.
I’m very aware that the published schema contains many more unused attributes and strictly speaking they would need to publish a schema with an errata of their acceptable subset - which for me, is unacceptable as it defeats the purpose of using a schema in the first place!
So I guess the issue becomes…
Do I build a tool to allow me to utilise FIXML as a repository and subset into a ‘clientFixml’ schema which keeps the overall structure but only contains the ‘stuff’ the client uses or design a database to hold the full XSD in all its glory?
Both options have their pro’s and con’s. and any suggestions are welcome.
Regards
George.
Looking around the Net it appears if I want to subset the main schema and only pull in only what I need than I’ll have to write some kind of interface myself.
Regards
George.
George,
<?xml version="1.0" encoding="UTF-8"?>
I do this for our routing system, so I create a high level schema that pulls in just the bits I want see below:
[ original email was from John Unwin - john@unwin.co.uk ]
I know what you mean if you use the raw schema you do produce a lot of data, on way we used to get a more reduced set was:
- create data binding classes based on the full schema
- output fix messages using the the databinding classes
- use a tool such as XML Spy to reverse engineer the fix messages (xml) to schemas (XSD)
You have to ensure that you output FIX messages with all the fields you support and of course you lose a lot of the type, enumerations and information that you get from the original - but you do get very condensed xsd files.
In our case we prefere to use the schema direct despite the size of the classes libs generated.
John,
Thanks for this, I’m equally concerned that by using the high level
schema method we still pull in an awful of other lower level stuff. For
example all 1800 components and all 18,000 field definitions.I need to design an Oracle database schema to store the clients defined
subset of FIXML and (for example) they only use about 30% of the
instrument attributes. While I can currently handle storing the
resulting XML automatically and fairly elegantly using metadata and
about 200 lines of pl/sql.I’m very aware that the published schema contains many more unused
attributes and strictly speaking they would need to publish a schema
with an errata of their acceptable subset - which for me, is
unacceptable as it defeats the purpose of using a schema in the
first place!So I guess the issue becomes…
Do I build a tool to allow me to utilise FIXML as a repository and
subset into a ‘clientFixml’ schema which keeps the overall structure but
only contains the ‘stuff’ the client uses or design a database to hold
the full XSD in all its glory?Both options have their pro’s and con’s. and any suggestions are
welcome.Regards George.
Looking around the Net it appears if I want to subset the main schema
and only pull in only what I need than I’ll have to write some kind of
interface myself.Regards George.
George, I do this for our routing system, so I create a high level
<?xml version="1.0" encoding="UTF-8"?>
schema that pulls in just the bits I want see below:
[ original email was from Clive Browning - clive@rapidaddition.com ]
Hi
We use the FIX repository to generate “run time schemas” as Jim Northey calls then. The FIX repository is the XML representation of the FIX message tables, data dictionary, and the source that the GTC generates the documentation, schema and FIXimate from.
We generate a run time schema that contains all the data types, components etc for each particular message we are interested in and then use the appropriate message schemas in a project.
You can also generate schemas for a specific product, such as order routing that only contains the messages, components etc for the relevant set of FIX messages.
We modified the sample application XSLT from the repository page to do this. The generic approach however is to iterate through the message contents xml writing the schema definition for a particular message, and whenever you encounter a component recursively iterate through that doing the same. If you add a trap to avoid writing a component twice you can generate a single schema for groups of messages.
Hope this helps
Clive Browning
Rapid Addition
www.rapidaddition.com
I know what you mean if you use the raw schema you do produce a lot of data, on way we used to get a more reduced set was:
- create data binding classes based on the full schema
- output fix messages using the the databinding classes
- use a tool such as XML Spy to reverse engineer the fix messages (xml)
to schemas (XSD)You have to ensure that you output FIX messages with all the fields you
support and of course you lose a lot of the type, enumerations and
information that you get from the original - but you do get very
condensed xsd files.In our case we prefere to use the schema direct despite the size of the
classes libs generated.John,
Thanks for this, I’m equally concerned that by using the high level
schema method we still pull in an awful of other lower level stuff.
For example all 1800 components and all 18,000 field definitions.I need to design an Oracle database schema to store the clients
defined subset of FIXML and (for example) they only use about 30% of
the instrument attributes. While I can currently handle storing the
resulting XML automatically and fairly elegantly using metadata and
about 200 lines of pl/sql.I’m very aware that the published schema contains many more unused
attributes and strictly speaking they would need to publish a schema
with an errata of their acceptable subset - which for me, is
unacceptable as it defeats the purpose of using a schema in the
first place!So I guess the issue becomes…
Do I build a tool to allow me to utilise FIXML as a repository and
subset into a ‘clientFixml’ schema which keeps the overall structure
but only contains the ‘stuff’ the client uses or design a database to
hold the full XSD in all its glory?Both options have their pro’s and con’s. and any suggestions are
welcome.Regards George.
Looking around the Net it appears if I want to subset the main schema
and only pull in only what I need than I’ll have to write some kind of
interface myself.Regards George.
George, I do this for our routing system, so I create a high level
<?xml version="1.0" encoding="UTF-8"?>
schema that pulls in just the bits I want see below:
Hmm,
Ok, from my perspective I can see the need for an easily operated mechanism to treat FIXML as a repository and generate specific sub-schemas containing one or more selected messages and selected atributes maintaining the added-value metadata such as iso*** etc…
From an Enterprise perspective we also need to be able to easily refactor for message/attribute changes and most importantly it needs to be easily reproduced without extensive XSD skills.
My current thoughts are to create a GUI holding the FIXML tree, where the user can select (drag/drop) the messages/nodes into another tree which drives a list of attributes for toggling on/off. Then outputting the refactored schema.
If I were to build such a thing am I barking up the wrong tree?
All opinions wellcome.
Regards
George
I know what you mean if you use the raw schema you do produce a lot of data, on way we used to get a more reduced set was:
- create data binding classes based on the full schema
- output fix messages using the the databinding classes
- use a tool such as XML Spy to reverse engineer the fix messages (xml)
to schemas (XSD)You have to ensure that you output FIX messages with all the fields you
support and of course you lose a lot of the type, enumerations and
information that you get from the original - but you do get very
condensed xsd files.In our case we prefere to use the schema direct despite the size of the
classes libs generated.John,
[ original email was from Jim Northey - jnorthey@jandj.com ]
George,
We have the FIX repository which is an XML representation of the specification and we have GUI tools that have been developed to maintain and modify the standard. These tools then generate schema, FIXimate, word docs, etc. Does this address your goals?
As one of the primary authors of the current schema design, I have to say that I would really not want us to base any work on using the schema as an input or part of our process. It is no secret that I have quite a bit of disdain XML Schema.
Actually the way Kevin designed the system - you can add your own XSLT files to transform the repository any way you like now.
FYI - I think we are looking for more input on a runtime schema Rapid Addition has there stuff - maybe we just need to review it and test it for some empirical evidence on the value of a runtime schema.
Jim N
Hmm,
Ok, from my perspective I can see the need for an easily operated
mechanism to treat FIXML as a repository and generate specific sub-
schemas containing one or more selected messages and selected atributes
maintaining the added-value metadata such as iso*** etc…From an Enterprise perspective we also need to be able to easily
refactor for message/attribute changes and most importantly it needs to
be easily reproduced without extensive XSD skills.My current thoughts are to create a GUI holding the FIXML tree, where
the user can select (drag/drop) the messages/nodes into another tree
which drives a list of attributes for toggling on/off. Then outputting
the refactored schema.If I were to build such a thing am I barking up the wrong tree?
All opinions wellcome.
Regards George
I know what you mean if you use the raw schema you do produce a lot of
data, on way we used to get a more reduced set was:
- create data binding classes based on the full schema
- output fix messages using the the databinding classes
- use a tool such as XML Spy to reverse engineer the fix messages
(xml) to schemas (XSD)You have to ensure that you output FIX messages with all the fields
you support and of course you lose a lot of the type, enumerations and
information that you get from the original - but you do get very
condensed xsd files.In our case we prefere to use the schema direct despite the size of
the classes libs generated.John,
Jim,
well, I composed a longer reply but the line dropped!
Please, don’t get me started on XML/dtd/xsd/xsl!
I do enterprise architecture stuff and having followed the gestation of FIX for some years, I can see loads of value in FIX/FIXML but mainly as a toolset/repository. As such it I think it needs to be a central FIX** repository of information, and as a repository it also needs tools for the average bod to address a wider audience.
I’ve done the internet bit (again) and I can’t still cn’t see what my client needs. In general, it seems to be, we’ve done x then y with product Z and we can get ‘fixml’ to work.
FIX needs to ne forged with real useage.
regards
George.
George,
We have the FIX repository which is an XML representation of the
specification and we have GUI tools that have been developed to maintain
and modify the standard. These tools then generate schema, FIXimate,
word docs, etc. Does this address your goals?As one of the primary authors of the current schema design, I have to
say that I would really not want us to base any work on using the schema
as an input or part of our process. It is no secret that I have quite a
bit of disdain XML Schema.Actually the way Kevin designed the system - you can add your own XSLT
files to transform the repository any way you like now.FYI - I think we are looking for more input on a runtime schema Rapid
Addition has there stuff - maybe we just need to review it and test it
for some empirical evidence on the value of a runtime schema.Jim N
Hmm,
Ok, from my perspective I can see the need for an easily operated
mechanism to treat FIXML as a repository and generate specific sub-
schemas containing one or more selected messages and selected
atributes maintaining the added-value metadata such as iso*** etc…From an Enterprise perspective we also need to be able to easily
refactor for message/attribute changes and most importantly it needs
to be easily reproduced without extensive XSD skills.My current thoughts are to create a GUI holding the FIXML tree, where
the user can select (drag/drop) the messages/nodes into another tree
which drives a list of attributes for toggling on/off. Then outputting
the refactored schema.If I were to build such a thing am I barking up the wrong tree?
All opinions wellcome.
Regards George
I know what you mean if you use the raw schema you do produce a lot
of data, on way we used to get a more reduced set was:
- create data binding classes based on the full schema
- output fix messages using the the databinding classes
- use a tool such as XML Spy to reverse engineer the fix messages
(xml) to schemas (XSD)You have to ensure that you output FIX messages with all the fields
you support and of course you lose a lot of the type, enumerations
and information that you get from the original - but you do get very
condensed xsd files.In our case we prefere to use the schema direct despite the size of
the classes libs generated.John,
Kevin,
I did advise the client to become a member of FPL etc. but they have declined…
"I think this allows you to generate all the stuff your trying to do. If it does not let us know, prefereably by getting involved and contributing to the Repository Working group. "
Hmm, fair comment, as a freelance consultant how would I do that?
I have looked at the sample repository and based on examining that, I’d suggest it needs some further work to become a more general tool, the minimum being modelled as primary/freign key realtionships, a couple of sample queries and possibly some some code modules as a functional library. The above would benefit from being created in an ANSI standard manner to allow it to be ported into a industrial strength RDBMS, say Oracle/SQl Server.
Having said that, as FIX is XML based I think an option is to consider whether it make sense to drop the public database repository and only provide the one standard schema as the repository. XML based tools can generate classes/messages etc. to meet application design requirements.
Regards
George
George
I agree with the sentiments expressed in the first half of your post,
FPL needs a repository, but not the second half, that the schema is
that document.The FIX repository, written in XML, is posted on the specification
page. You need to be a member of FPL to access the link. If you are
not a member of the FPL but logged into the site you should see the
documentation, and a cut down sample of the repository. Full
working etc just restricted to 4 application messages and the
session layer messages.This repository comprises 5 XML files.
MsgTypes - List messages, names, messagetypes etc Components - List
components, componenttypes etc Fields - List Tag numbers, fieldnames
etc Enums - List enumeration and description for each field if
appropriate MsgConts - Describes how Components and Fields are
assembled into messages.I think this allows you to generate all the stuff your trying to do. If
it does not let us know, prefereably by getting involved and
contributing to the Repository Working group.We uses these XML files as the source to generate the schema, FIXimate
and the message tables and data dictionary that you see in the Word
documentation. With the exception of the Word Documentation this is
performed by applying XSLT transforms to the XML documents, we are also
in the process of preparing an approach to version releases where all
new functionality is captured in a series of XSLT files that update on
version of the prepository to the next.Member firms of FPL use these files to generate runtime schemas similar
to the ones discussed on this thread and some of the FIX engine vendors
uses these files as the basis of the data dictionaries etc that allow
thier engines to validate thier FIX messages etc.The XML files include information on message catagory so if you need a
schema for just pretrade and trade you could compile one.I’d encourage you to use this resource rather than the schema as your
source as we’re likely to change both over the future years as FPL
incorporates conditional message validation and more business process
into it computer readable forms. We’ll be aiming to change the
repository to only introduce breakages where absolutely needed to
deliver benefit. The schema on the other hand will change to reflect the
needs of it’s user community, which we do not consider include people
using it an alternative to the repository. It may continue in it present
form, it may get expanded to incorporate conditional logic, business
rules and process, we may at some point in the future replace the xsd
with a Relax NG form.In terms of tools we are working on them but also encourage others to
develop tools as indeed a number have.Hope this helps and let me know if your still having problems finding
the relevant parts of the web site.Cheers Kevin Houstoun GTC CoChair Consultant to HSBC
Jim,
well, I composed a longer reply but the line dropped!
Please, don’t get me started on XML/dtd/xsd/xsl!
I do enterprise architecture stuff and having followed the gestation
of FIX for some years, I can see loads of value in FIX/FIXML but
mainly as a toolset/repository. As such it I think it needs to be a
central FIX** repository of information, and as a repository it also
needs tools for the average bod to address a wider audience.I’ve done the internet bit (again) and I can’t still cn’t see what my
client needs. In general, it seems to be, we’ve done x then y with
product Z and we can get ‘fixml’ to work.FIX needs to ne forged with real useage.
regards George.
George,
We have the FIX repository which is an XML representation of the
specification and we have GUI tools that have been developed to
maintain and modify the standard. These tools then generate schema,
FIXimate, word docs, etc. Does this address your goals?As one of the primary authors of the current schema design, I have
to say that I would really not want us to base any work on using the
schema as an input or part of our process. It is no secret that I
have quite a bit of disdain XML Schema.Actually the way Kevin designed the system - you can add your own
XSLT files to transform the repository any way you like now.FYI - I think we are looking for more input on a runtime schema
Rapid Addition has there stuff - maybe we just need to review it and
test it for some empirical evidence on the value of a runtime
schema.Jim N
Hmm,
Ok, from my perspective I can see the need for an easily operated
mechanism to treat FIXML as a repository and generate specific sub-
schemas containing one or more selected messages and selected
atributes maintaining the added-value metadata such as iso***
etc…From an Enterprise perspective we also need to be able to easily
refactor for message/attribute changes and most importantly it
needs to be easily reproduced without extensive XSD skills.My current thoughts are to create a GUI holding the FIXML tree,
where the user can select (drag/drop) the messages/nodes into
another tree which drives a list of attributes for toggling
on/off. Then outputting the refactored schema.If I were to build such a thing am I barking up the wrong tree?
All opinions wellcome.
Regards George
I know what you mean if you use the raw schema you do produce a
lot of data, on way we used to get a more reduced set was:
- create data binding classes based on the full schema
- output fix messages using the the databinding classes
- use a tool such as XML Spy to reverse engineer the fix
messages
(xml) to schemas (XSD)You have to ensure that you output FIX messages with all the
fields you support and of course you lose a lot of the type,
enumerations and information that you get from the original -
but you do get very condensed xsd files.In our case we prefere to use the schema direct despite the size
of the classes libs generated.John,
Hi Kevin,
Well here we are 3.5 years on and I’m back on the Fixml trail.
All to the good I hope.
Anyway the point of this blast from the past is that I never found a way to generate a database schema for one or more Fix nodes (or XSD in general) so I’ve had something in R&D for a while that shreds a XML schema (tested Fix) into an SQL database schema (optional pk/fk/ri and check constraints). 100% shred gives about 95% structure because of node reuse e.g. Pty/SubPty and I’ve never bothered with the last 5%.
Now having remembered your original comments I’m considering redeveloping the process against the original XML repository and wondering if it’s worth the effort delving deeply into XSLT. Any comments/suggestions/warning 
Regards
George
Regards
George
George
I agree with the sentiments expressed in the first half of your post,
FPL needs a repository, but not the second half, that the schema is
that document.The FIX repository, written in XML, is posted on the specification
page. You need to be a member of FPL to access the link. If you are
not a member of the FPL but logged into the site you should see the
documentation, and a cut down sample of the repository. Full
working etc just restricted to 4 application messages and the
session layer messages.This repository comprises 5 XML files.
MsgTypes - List messages, names, messagetypes etc Components - List
components, componenttypes etc Fields - List Tag numbers, fieldnames
etc Enums - List enumeration and description for each field if
appropriate MsgConts - Describes how Components and Fields are
assembled into messages.I think this allows you to generate all the stuff your trying to do. If
it does not let us know, prefereably by getting involved and
contributing to the Repository Working group.We uses these XML files as the source to generate the schema, FIXimate
and the message tables and data dictionary that you see in the Word
documentation. With the exception of the Word Documentation this is
performed by applying XSLT transforms to the XML documents, we are also
in the process of preparing an approach to version releases where all
new functionality is captured in a series of XSLT files that update on
version of the prepository to the next.Member firms of FPL use these files to generate runtime schemas similar
to the ones discussed on this thread and some of the FIX engine vendors
uses these files as the basis of the data dictionaries etc that allow
thier engines to validate thier FIX messages etc.The XML files include information on message catagory so if you need a
schema for just pretrade and trade you could compile one.I’d encourage you to use this resource rather than the schema as your
source as we’re likely to change both over the future years as FPL
incorporates conditional message validation and more business process
into it computer readable forms. We’ll be aiming to change the
repository to only introduce breakages where absolutely needed to
deliver benefit. The schema on the other hand will change to reflect the
needs of it’s user community, which we do not consider include people
using it an alternative to the repository. It may continue in it present
form, it may get expanded to incorporate conditional logic, business
rules and process, we may at some point in the future replace the xsd
with a Relax NG form.In terms of tools we are working on them but also encourage others to
develop tools as indeed a number have.Hope this helps and let me know if your still having problems finding
the relevant parts of the web site.Cheers Kevin Houstoun GTC CoChair Consultant to HSBC
Jim,
well, I composed a longer reply but the line dropped!
Please, don’t get me started on XML/dtd/xsd/xsl!
I do enterprise architecture stuff and having followed the gestation
of FIX for some years, I can see loads of value in FIX/FIXML but
mainly as a toolset/repository. As such it I think it needs to be a
central FIX** repository of information, and as a repository it also
needs tools for the average bod to address a wider audience.I’ve done the internet bit (again) and I can’t still cn’t see what my
client needs. In general, it seems to be, we’ve done x then y with
product Z and we can get ‘fixml’ to work.FIX needs to ne forged with real useage.
regards George.
George,
We have the FIX repository which is an XML representation of the
specification and we have GUI tools that have been developed to
maintain and modify the standard. These tools then generate schema,
FIXimate, word docs, etc. Does this address your goals?As one of the primary authors of the current schema design, I have
to say that I would really not want us to base any work on using the
schema as an input or part of our process. It is no secret that I
have quite a bit of disdain XML Schema.Actually the way Kevin designed the system - you can add your own
XSLT files to transform the repository any way you like now.FYI - I think we are looking for more input on a runtime schema
Rapid Addition has there stuff - maybe we just need to review it and
test it for some empirical evidence on the value of a runtime
schema.Jim N
Hmm,
Ok, from my perspective I can see the need for an easily operated
mechanism to treat FIXML as a repository and generate specific sub-
schemas containing one or more selected messages and selected
atributes maintaining the added-value metadata such as iso***
etc…From an Enterprise perspective we also need to be able to easily
refactor for message/attribute changes and most importantly it
needs to be easily reproduced without extensive XSD skills.My current thoughts are to create a GUI holding the FIXML tree,
where the user can select (drag/drop) the messages/nodes into
another tree which drives a list of attributes for toggling
on/off. Then outputting the refactored schema.If I were to build such a thing am I barking up the wrong tree?
All opinions wellcome.
Regards George
I know what you mean if you use the raw schema you do produce a
lot of data, on way we used to get a more reduced set was:
- create data binding classes based on the full schema
- output fix messages using the the databinding classes
- use a tool such as XML Spy to reverse engineer the fix
messages
(xml) to schemas (XSD)You have to ensure that you output FIX messages with all the
fields you support and of course you lose a lot of the type,
enumerations and information that you get from the original -
but you do get very condensed xsd files.In our case we prefere to use the schema direct despite the size
of the classes libs generated.John,