Imported from previous forum
Hi,
The XML specs specify the UTF-8 and UTF-16 encodings as being normative. That is, any valid XML document MUST be in one of these two encodings. If a FIXML message is generated in UTF-16, would current software be able to handle it? Or does FIXML envisage only the UTF-8 encoding? Or even the subset of UTF-8: ASCII? What I mean to say is that if somebody wrote a parser assuming that the elements of FIXML will always be ASCII (NOTE I am not talking about the data, but name tags and elements, i.e., the FIXML protocol itself) would this be OK?
This is an important question and one that we overlooked initially due to lack of understanding of XML back in the early day. I have sent out some inquiries to some of our more expert practitioners and hope to post a response early this week on what we should mean when we say UTF-8 and UTF-16 are normative. Also, there is a proposal to expand FIXML to support FIX encoded fields, with the proposal that the data of the EncodedData fields be encoded as Base64.
I think for the foreseeable future (likely 5 years minimum - based upon my best guess), you can write your parser assuming that the FIXML element and attribute names will remain within the ASCII subset of UTF-8 encodings.
(Do you really mean you are writing a custom XML parser (not that this has not been done many times) or are you writing a parsing application using one of the widely used industry standard parsers?)
I hope this response will help you continue to move your application forward while we work to provide clearer direction on normative encodings for FIXML.
Hi,
The XML specs specify the UTF-8 and UTF-16 encodings as being normative. That is, any valid XML document MUST be in one of these two encodings. If a FIXML message is generated in UTF-16, would current software be able to handle it? Or does FIXML envisage only the UTF-8 encoding? Or even the subset of UTF-8: ASCII? What I mean to say is that if somebody wrote a parser assuming that the elements of FIXML will always be ASCII (NOTE I am not talking about the data, but name tags and elements, i.e., the FIXML protocol itself) would this be OK?
Actually, it’s like this. I have lodged a permanent residence application in Australia and have been applying for jobs. It seems that my 10+ years experience in Pakistan is a bit dubitable in the eyes of recruiters. So, I have decided to start an open source project to show off my development skills AND gain experience on Linux (I am core Windows). Since I am from a finance background, I saw experience with FIX as one of the desirables.
At the moment I am not decided on writing an XML parser, but I have taken a look at the available ones. The open source ones have one or the other limitations. You see, XML is a whole protocol stack, and as things stand currently, we have XML Core, XML Schema, XML Namespaces, XPath etc. etc. One thing depends on the other. FIXML already uses XML and XML Schema and there is no knowing what else they might use. At the same time, coding the FULL generality of XML is also complex. If you get a parser that caters to the FULL specification, it would be a behemoth that amongst other things would need to take into account the complex Unicode collation rules when matching element names (actually I am not VERY sure about this so correct me if I am wrong). A parser that can assume ASCII would be much more efficient. So, for FIXML, we need a parser that provides just enough functionality because more functionality can lead to a performance penalty.
Other than this, for real-time networked systems, one can imagine that data would be received piece by piece. So, in order to avoid waiting for the receive operation to complete, the parser should start incremental parsing up to the available data. Here again, we couldn’t really make a black-box use of an already implemented parser.
This is an important question and one that we overlooked initially due to lack of understanding of XML back in the early day. I have sent out some inquiries to some of our more expert practitioners and hope to post a response early this week on what we should mean when we say UTF-8 and UTF-16 are normative. Also, there is a proposal to expand FIXML to support FIX encoded fields, with the proposal that the data of the EncodedData fields be encoded as Base64.
I think for the foreseeable future (likely 5 years minimum - based upon my best guess), you can write your parser assuming that the FIXML element and attribute names will remain within the ASCII subset of UTF-8 encodings.
(Do you really mean you are writing a custom XML parser (not that this has not been done many times) or are you writing a parsing application using one of the widely used industry standard parsers?)
I hope this response will help you continue to move your application forward while we work to provide clearer direction on normative encodings for FIXML.
Hi,
The XML specs specify the UTF-8 and UTF-16 encodings as being normative. That is, any valid XML document MUST be in one of these two encodings. If a FIXML message is generated in UTF-16, would current software be able to handle it? Or does FIXML envisage only the UTF-8 encoding? Or even the subset of UTF-8: ASCII? What I mean to say is that if somebody wrote a parser assuming that the elements of FIXML will always be ASCII (NOTE I am not talking about the data, but name tags and elements, i.e., the FIXML protocol itself) would this be OK?