FIX Orchestra tags "Sort" and "Id" required or optional in Markdown

1

Hello community,

currently, I am writing a custom Markdown document according to the FIX Orchestra standard.
I used the Orchestra FIX Latest XML as a starting point. Now I would like to know whether I am supposed to keep the column “Sort” and “Id” to be mapped back to the XML attributes.
Please, could anybody advise?

Many thanks, Jörg

2

“sort” attribute is used to organize documentation of codes in a code set, along with the “group” attribute. This may or may not be important to your ROE. For an example of how this is used by FIXimate, see SecurityTypeCodeSet.

On the other hand, “id” attribute is used to uniquely identify every message and message element in an Orchestra repository, along with a humanly readable “name” attribute. md2orchestra is very tolerant of missing IDs in your markdown document. It will try to look up an ID in an Orchestra reference file from a supplied name. If it cannot find it in the reference file, such as for a user-defined element, it will generate a random ID for it. The pair of md2orchestra and orchestra2md will attempt to preserve IDs in round-trips. However, if you don’t keep the ID in your markdown, a new ID will be generated on every editing round-trip. Thus will appear to be a change when you don’t intend one. Therefore, I recommend preserving any generated IDs in markdown.

3

Thanks, @donaldmendelson.

A good compromise could be to have further options for orchestra2md to suppress these attributes for generating the documentation to be published, that is to be converted by pandoc.

That means to work with two Markdown documents.

4

The solution is to use simple scripts to work on the markdown (or also already on the XML file) content prior to handing it over to Pandoc. orchestra2md should not be burdened too much with additional usages of the markdown file.

My personal view is that markdown should be the master format for authoring and used as a source in both directions, i.e. a) to create an Orchestra XML file and b) to create an RoE document as docx/pdf file. Initially you need orchestra2md to create your first markdown file from an Orchestra XML file created by Log2Orchestra or Playlist. That is, if you are a FIX member… :wink:

1 Like
5

I use a workflow like that: define the technical ROE in Markdown and have a Makefile that runs the tablature tools and generates a PDF file using pandoc. The PDF file contains the technical ROE plus some other Markdown files containing notes and appendixes. The Makefile then creates a zip archive with the PDF, the XML and the source Markdown too, the latter to make it easy for users of the ROE to see diff between versions.

I don’t include the Sort columns in my Markdown file.

There are a few little issues with the tablature tools, some of which we hope to fix soon: if you do a roundtrip md -> xml -> md, then some odd artifacts get added, so I have to patch the Markdown before publishing, and the generated XML only seems to include Datatypes of the Fields, not Datatypes of the Codesets or the Union Types, so I patch that too.

1 Like
6

@mordav, what is your use case for doing roundtrips repeatedly? md2orchestra does lookups in the reference file to create the XML and hence makes additions that then show up when you use orchestra2md to get an “enhanced” markdown from the XML. That is intentionally different from the one you started with.

7

Hello @mordav,

I started using the tool chains as well. You are writing about some odd artifacts. What do you mean?
Personally, I triggered these issues on GitHub:

Maybe we could share our experiences.

Cheers, Jörg

8

@mordav, @hanno.klein: I anticipated that Tablature tools would be used iteratively to refine a ROE. Similarly, I may edit a Word document many times until I am satisfied with the final result.

My suggestion is to use orchestra2md with argument -a or --all to output all elements to preserve them in a round-trip. Then on the final pass, output elements selectively for presentation. By default, pedigree, FIXML attributes, and datatypes are suppressed, but you can turn them on individually with --pedigree, --fixml, and --datatypes arguments respectively.

9

@hanno.klein my reason for repeated roundtrips is partly that having a clean roundtrip gives me confidence that there are no gremlins in my file or the tools. But more concretely:

  • I want my Markdown ROE to be standalone and complete, containing not just Messages but all the associated Components, Groups, Fields and their Codesets - this last particularly important as I often reduce the Codeset to the smaller subset that is handled by my application
  • so I first create a Markdown file A, then run a roundtrip to pull in additional data from the reference file, to give me a new Markdown file A’ which I can distribute
  • I can then make further changes, and run further roundtrips to make sure everything is being converted successfully by the tablature tools
  • but A’ includes some unnecessary elements which I don’t want, so I have a small diff file that I apply using patch to remove them when generating my PDF and ZIP files for distribution
  • this is all in my Makefile, and running ‘make clean’ applies the reverse patch, leaving everything unchanged (clean to git status).

I never fully got to the bottom of what was happening to cause the extra elements to be added. It seems to differ depending on whether there is a preamble after the title and before the metadata table, but typically it’s things like

#### Actors

None

or

## Actors and Flows

None

I would have to run some tests to remind myself of all the variations I came across.

10

thank you @donaldmendelson I will experiment with these switches

11

Yes I would be happy to share experiences @jorgthonnes. The tablature tools have proved very good for us. The Markdown is a perfect format for the technical API part of an ROE, and is easily converted using Pandoc. I convert to PDF but I’ve seen people create HTML versions too.

david.morgan@itiviti.com

12

Just as an FYI: all major FIX Technical Standards are maintained in markdown from where a docx/pdf version is generated for download and an html version is generated for the online pages on the FIX website. We also use the same markdown source to generate the docx ISO layout for FIX Tag=Value and the FIX Session Layer standards (ISO 3531) for submission to ISO.

13

First, sorry to jump without understanding the topic too much.

I wanted to propose that the sort attribute in the codeType should actually be in an appinfo element. Specifically it tells the application (e.g. FIXIMATE) the order of the enumeration code.

Note that currently although orchestra defines <annotation>/<appinfo> the fix representation in orchestra does not have any <appinfo> element. So that is the first time to use it.

So the example for use would be:
instead of
<fixr:code name="Buy" id="4001" value="B" sort="1" added="FIX.2.7">

the FIX orchestra xml file should contain:
<fixr:code name="Buy" id="4001" value="B" added="FIX.2.7"> <fixr:annotation> <fixr:appinfo purpose=”sort”>1</fixr:appinfo>...

BTW I don’t think that the group attribute (of codeType) should be considered as appinfo that is because the group attribute sometimes contains interesting business terms for grouping (I consider the group like the category attribute of a message ).

Please consider the above as a proposal for a change

14

@yuvalcohen, I agree with your suggestion that an attribute that is specific to one application should be encoded as appinfo.

For your information, we already have another instance in the works. Unified Repository had an attribute to inform the FIXML generator that certain properties were to be ignored for the XML translation. In the future, it will be encoded in Orchestra as:

<fixr:appinfo purpose="FIXML">notReqXML</fixr:appinfo>

As you see, in the appinfo element, purpose attribute holds the name of the application. Also note that the XML schema allows extension of appinfo with other attributes. So in the case of FIXimate, it might be something like:

<fixr:appinfo purpose="FIXimate" sort="1"/>
2 Likes