Representation of "data" in FIXML

, ,
1

Imported from previous forum

2

I understand the mapping between ASCII data in fields, and Shift_JIS (or other encoded) data in encoded fields (with encoding defined as per message encoding tag) and the Unicode strings placed into attributes in the FIXML documents.

However, what about other non character data, such as those fields which have a corresponding length field?

I think that if a FIX field contains “data” its content should be stored in the FIXML attribute using base64 encoding.
I say this as its the most obvious way to ensure all the bytes are preserved.
I’m not convinced it would be wise to attempt to convert the bytes to a String using any character encoding, even UTF-16.

But I can’t find any statement to this effect (or otherwise).

Comments…?

{{{ Andy

3

To convert binary data (or other non-printable-string data) to a printable string, some sort of encoding can be applied. NOTE: This always expands the size. For example, hex encoding produces a string that is 2 character for each byte (i.e. it doubles the size). Base64 encoding is a better choice because the output is 4 characters for every 3 bytes encoded. Yes, Base64 would be a good choice.

For example, play around with the following Perl script:

#!/usr/bin/perl
#
# Copyright (c) Simon Flannery
#
use strict;
my $bdata = "";
use MIME::Base64;
use Compress::Zlib;

while(<>) {
   if (m/^\s*[^\s]+\s*$/) {
      $bdata = $bdata . $_;
   }
}

# Decode the data into binary data
my $compressed = decode_base64($bdata);
# Inflate the binary data into the original data
my $human      = uncompress($compressed);

print $human;