Generating XML dictionaries from recent EP release XML

1

It appears the structure of the recent EPs available is some kind of hybrid between a 2010 edition and a 2016 edition:

For example in the latest EP release FIXRepository_FIX.5.0SP2_EP240:

Unified/FixRepository.xml:

<fixRepository xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           copyright="Copyright (c) FIX Protocol Ltd. All Rights Reserved."
           xsi:noNamespaceSchemaLocation="FixRepository.xsd"
           generated="2017-12-29T02:40:54Z"
           edition="2010">

Note the edition states it is the 2010 version. This is the same for the Basic files, e.g.:

Basic/Messages.xml:

<Messages xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      copyright="Copyright (c) FIX Protocol Ltd. All Rights Reserved."
      edition="2010"
      version="FIX.5.0SP2"
      xsi:noNamespaceSchemaLocation="../../schema/Messages.xsd"
      generated="2017-12-25T05:18:39.12Z"
      latestEP="239">

Yet previously linked converters (e.g. https://code.google.com/archive/p/fix-repository-to-quickfix-xml and quickfixj-dictgenerator ) which are designed for the 2010 edition fail because they expect a MsgType.xml file that doesn’t exist (and isn’t the same format as the Messages.xml file)

The fix-orchestra project claims 2016 import ability and yet doesn’t seem to be able to understand any of the files provided as far as I can tell.

Using it on the Unified/FixRepository.xml file generates several errors for the root tag (it expects repository instead of fixRepository). When these are fixed it complains about a lack of code sets. Then in the writing phase the actual structure doesn’t seem to be parsed.

Has anyone used the fix-orchestra repository-quickfix conversion functionality succesfully on a recent EP release - if so how was this done.

2

The EP build process is only disseminating FIX Repository 2010 Edition currently. (Orchestra / 2016 Edition format dissemination is planned for the future.) Two formats are currently supplied, Basic Repository and Unified Repository. However, I don’t see a file named MsgType.xml in any EP package. Could this file come from a another source?

1 Like
3

Follow-up: I followed your link to the converter code and it states that it is based on 2008 Edition. Release notes for 2010 Edition say that file MsgType.xml was renamed to Messages.xml. The code needs to be updated to work with current EPs. (When Orchestra files are publicly released, you will be able to use converter to QuickFIX data dictionary format that already exists in GitHub.)

1 Like
4

Thanks Donald - also I can see you’ve done a lot of the fix-orchestra stuff too so thanks for that as well.

So right now, there’s no tool to convert 2010 format - given that fix-orchestra only has an XSD right now.

I guess the MsgType.xml was provided in the 2008 edition which was only used for FIX40->FIX50SP2.

It looks like fix-orchestra should ideally have an 2008, 2010 and 2016 repository. Unfortunately as the license for QuickFIXJ is not Apache 2 I guess it’s a clean-room re-write to implement even a 2008 converter.

I guess I’ll have to assess the level of work building a 2010 converter vs the net gain of a dictionary with the new tags.

Thanks for helping to clarify the situation.

5

I’ve successfully converted EP233 to a QuickFIX compatible XML DataDictionary via a 2-step process using the tools available at: GitHub

  • Using RepositoryXslTransformer, convert the 2010 version of EP233 to a 2016 version of EP233.
  • Using DataDictionaryGenerator, generate the QuickFIX compatible XML DataDictionaries using the 2016 converted Repository from above.
2 Likes
6

This approach did not work for me. I can convert to 2016 format. But when I send this to DDG, I get this exception. Any ideas?

java.lang.NumberFormatException: Zero length BigInteger
at java.math.BigInteger.(Unknown Source)
at java.math.BigInteger.(Unknown Source)
at com.sun.xml.internal.bind.DatatypeConverterImpl._parseInteger(Unknown Source)
at com.sun.xml.internal.bind.v2.model.impl.RuntimeBuiltinLeafInfoImpl$22.parse(Unknown Source)
at com.sun.xml.internal.bind.v2.model.impl.RuntimeBuiltinLeafInfoImpl$22.parse(Unknown Source)
at com.sun.xml.internal.bind.v2.runtime.reflect.TransducedAccessor$CompositeTransducedAccessorImpl.parse(Unknown Source)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.StructureLoader.startElement(Unknown Source)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallingContext._startElement(Unknown Source)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallingContext.startElement(Unknown Source)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.SAXConnector.startElement(Unknown Source)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(Unknown Source)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(Unknown Source)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(Unknown Source)
at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(Unknown Source)
at javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(Unknown Source)
at javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(Unknown Source)
at io.fixprotocol.orchestra.quickfix.DataDictionaryGenerator.unmarshal(DataDictionaryGenerator.java:187)
at io.fixprotocol.orchestra.quickfix.DataDictionaryGenerator.generate(DataDictionaryGenerator.java:88)

7

Which EP were you trying to convert @rbaxter1 - I got the routine to work for EP240 providing the Unified/FixRepository.xml file as the input.

If you debug into it you ought to be able to work out what element is causing the problem.

8

I am using the latestEP(240) from here: https://www.fixtrading.org/standards/fix-repository/ but it also fails with the file generated using the existing XslTransformerTest. I recently upgraded java to 9.0.4.

9

We are investigating an apparent flaw in the Repository. Three data fields are missing a corresponding Length field. In the translation to Orchestra format, an empty attribute is created, resulting in the numeric parse error.

The three fields are:

id=“42486” name=“LegPaymentStreamFormula” type="XMLData"
id=“42684” name=“PaymentStreamFormula” type="XMLData"
id=“42982” name=“UnderlyingPaymentStreamFormula” type=“XMLData”

If you are not using those fields, you can work around the problem by removing them and references to them while we determine a resolution.

10

@donaldmendelson bingo! that did the trick. thank you.

11

@donaldmendelson i also discovered this problem:

<field number="41237" name="NoStreamAssetAttributes" type="NUMINGROUP">
    <value enum="D" description="DAY"/>
</field>

Another observation about the data dictionary resulting from the conversion process described above is that none of the groups are decorated with “required” data. Is this expected? it’s easy enough to assume “N” is not available.

12

Thanks, 41237 does not look too good in FIXimate…we will “fix” that. NumInGroup fields never have a list of pre-defined valid values.