CME FastFix Decoder

, ,
1

Imported from previous forum

2

Guys,

Has anyone used the FastFix Decoder available from the cme. http://www.cmegroup.com/globex/resources/fix-fast-decoder-agreement.html

If so, how would you rate it’s performance?

It decodes and then generates the fix strings. Our initial tests show that it takes 1.7ms to do this for each message. I presuming I should be aiming for much faster times than this?

Many Thanks.

  • Lee.
3

Lee,

We tested CME’s FixFAST decoder and found that the performance was good. Decoding 1M messages took an average of 35 microseconds per messages on a standard Dell dual core 2GHz desktop.

Franz

Guys,

Has anyone used the FastFix Decoder available from the cme. http://www.cmegroup.com/globex/resources/fix-fast-decoder-
agreement.html

If so, how would you rate it’s performance?

It decodes and then generates the fix strings. Our initial tests show
that it takes 1.7ms to do this for each message. I presuming I should be
aiming for much faster times than this?

Many Thanks.

  • Lee.
4

Franz,

Did that time include recreating the Fix string?

  • Lee.

Lee,

We tested CME’s FixFAST decoder and found that the performance was good.
Decoding 1M messages took an average of 35 microseconds per messages on
a standard Dell dual core 2GHz desktop.

Franz

Guys,

Has anyone used the FastFix Decoder available from the cme. http://www.cmegroup.com/globex/resources/fix-fast-decoder-
agreement.html

If so, how would you rate it’s performance?

It decodes and then generates the fix strings. Our initial tests show
that it takes 1.7ms to do this for each message. I presuming I should
be aiming for much faster times than this?

Many Thanks.

  • Lee.
5

Lee,

No, the time noted was only for decoding the message into CME’s tree structure, we did not query the structure for data or convert it to a Fix string.

Franz

Franz,

Did that time include recreating the Fix string?

  • Lee.

Lee,

We tested CME’s FixFAST decoder and found that the performance was
good. Decoding 1M messages took an average of 35 microseconds per
messages on a standard Dell dual core 2GHz desktop.

Franz

Guys,

Has anyone used the FastFix Decoder available from the cme. http://www.cmegroup.com/globex/resources/fix-fast-decoder-
agreement.html

If so, how would you rate it’s performance?

It decodes and then generates the fix strings. Our initial tests
show that it takes 1.7ms to do this for each message. I presuming I
should be aiming for much faster times than this?

Many Thanks.

  • Lee.
6

Franz,

Used the performance timer and am now seeing it takes 192 microseconds to decode and recreate the fix string. More like it.

Did you have any issues with using the decoder on any messages passed to it? I’m wondering if it supports the full spec.

Thanks.

  • Lee.

Lee,

No, the time noted was only for decoding the message into CME’s tree
structure, we did not query the structure for data or convert it to a
Fix string.

Franz

Franz,

Did that time include recreating the Fix string?

  • Lee.

Lee,

We tested CME’s FixFAST decoder and found that the performance was
good. Decoding 1M messages took an average of 35 microseconds per
messages on a standard Dell dual core 2GHz desktop.

Franz

Guys,

Has anyone used the FastFix Decoder available from the cme. http://www.cmegroup.com/globex/resources/fix-fast-decoder-
agreement.html

If so, how would you rate it’s performance?

It decodes and then generates the fix strings. Our initial tests
show that it takes 1.7ms to do this for each message. I presuming
I should be aiming for much faster times than this?

Many Thanks.

  • Lee.
7

Lee,

We only tested the decoder on CME messages. We are not currently using the CME decoder; we just wanted to compare times to our decoder. I would be surprised if it did not support the full spec.

Franz

Franz,

Used the performance timer and am now seeing it takes 192 microseconds
to decode and recreate the fix string. More like it.

Did you have any issues with using the decoder on any messages passed to
it? I’m wondering if it supports the full spec.

Thanks.

  • Lee.

Lee,

No, the time noted was only for decoding the message into CME’s tree
structure, we did not query the structure for data or convert it to a
Fix string.

Franz

Franz,

Did that time include recreating the Fix string?

  • Lee.

Lee,

We tested CME’s FixFAST decoder and found that the performance was
good. Decoding 1M messages took an average of 35 microseconds per
messages on a standard Dell dual core 2GHz desktop.

Franz

Guys,

Has anyone used the FastFix Decoder available from the cme. http://www.cmegroup.com/globex/resources/fix-fast-decoder-
agreement.html

If so, how would you rate it’s performance?

It decodes and then generates the fix strings. Our initial tests
show that it takes 1.7ms to do this for each message. I
presuming I should be aiming for much faster times than this?

Many Thanks.

  • Lee.
8

Franz,

Decoding incremental messages with the CME decoder is taking ~300 microseconds. This still seems high in comparison with the figures you mention. The cme decoder also generates the fix strings, so I believe this to be taking a lot of the time. How did you disable this?

Thanks.

  • Lee.

Lee,

We only tested the decoder on CME messages. We are not currently using
the CME decoder; we just wanted to compare times to our decoder. I would
be surprised if it did not support the full spec.

Franz

Franz,

Used the performance timer and am now seeing it takes 192 microseconds
to decode and recreate the fix string. More like it.

Did you have any issues with using the decoder on any messages passed
to it? I’m wondering if it supports the full spec.

Thanks.

  • Lee.

Lee,

No, the time noted was only for decoding the message into CME’s tree
structure, we did not query the structure for data or convert it to
a Fix string.

Franz

Franz,

Did that time include recreating the Fix string?

  • Lee.

Lee,

We tested CME’s FixFAST decoder and found that the performance
was good. Decoding 1M messages took an average of 35
microseconds per messages on a standard Dell dual core 2GHz
desktop.

Franz

Guys,

Has anyone used the FastFix Decoder available from the cme.
http://www.cmegroup.com/globex/resources/fix-fast-decoder-
agreement.html

If so, how would you rate it’s performance?

It decodes and then generates the fix strings. Our initial
tests show that it takes 1.7ms to do this for each message. I
presuming I should be aiming for much faster times than this?

Many Thanks.

  • Lee.
9

Lee,

I did not disable generating the fix strings. I simply grabbed some raw data off the wire and ran a series of messages through their decoder timing how long it took for FastTemplateIdDecoderStrategy.decode() to run.

Franz

Franz,

Decoding incremental messages with the CME decoder is taking ~300
microseconds. This still seems high in comparison with the figures you
mention. The cme decoder also generates the fix strings, so I believe
this to be taking a lot of the time. How did you disable this?

Thanks.

  • Lee.

Lee,

We only tested the decoder on CME messages. We are not currently using
the CME decoder; we just wanted to compare times to our decoder. I
would be surprised if it did not support the full spec.

Franz

Franz,

Used the performance timer and am now seeing it takes 192
microseconds to decode and recreate the fix string. More like it.

Did you have any issues with using the decoder on any messages
passed to it? I’m wondering if it supports the full spec.

Thanks.

  • Lee.

Lee,

No, the time noted was only for decoding the message into CME’s
tree structure, we did not query the structure for data or convert
it to a Fix string.

Franz

Franz,

Did that time include recreating the Fix string?

  • Lee.

Lee,

We tested CME’s FixFAST decoder and found that the performance
was good. Decoding 1M messages took an average of 35
microseconds per messages on a standard Dell dual core 2GHz
desktop.

Franz

Guys,

Has anyone used the FastFix Decoder available from the cme.
http://www.cmegroup.com/globex/resources/fix-fast-decoder-
agreement.html

If so, how would you rate it’s performance?

It decodes and then generates the fix strings. Our initial
tests show that it takes 1.7ms to do this for each message.
I presuming I should be aiming for much faster times than
this?

Many Thanks.

  • Lee.
10

Franz,

I’m timing the same way, seeing how long decode() takes. However, this call includes the generation of the fix string. I do not have a FastTemplateIdDecoderStrategy in this code though, so looks like you may have a different version :confused: Or you’ve renamed it :slight_smile:

Thanks.

Lee,

I did not disable generating the fix strings. I simply grabbed some
raw data off the wire and ran a series of messages through their
decoder timing how long it took for
FastTemplateIdDecoderStrategy.decode() to run.

Franz

Franz,

Decoding incremental messages with the CME decoder is taking ~300
microseconds. This still seems high in comparison with the figures you
mention. The cme decoder also generates the fix strings, so I believe
this to be taking a lot of the time. How did you disable this?

Thanks.

  • Lee.

Lee,

We only tested the decoder on CME messages. We are not currently
using the CME decoder; we just wanted to compare times to our
decoder. I would be surprised if it did not support the full spec.

Franz

Franz,

Used the performance timer and am now seeing it takes 192
microseconds to decode and recreate the fix string. More like it.

Did you have any issues with using the decoder on any messages
passed to it? I’m wondering if it supports the full spec.

Thanks.

  • Lee.

Lee,

No, the time noted was only for decoding the message into CME’s
tree structure, we did not query the structure for data or
convert it to a Fix string.

Franz

Franz,

Did that time include recreating the Fix string?

  • Lee.

Lee,

We tested CME’s FixFAST decoder and found that the
performance was good. Decoding 1M messages took an average
of 35 microseconds per messages on a standard Dell dual core
2GHz desktop.

Franz

Guys,

Has anyone used the FastFix Decoder available from the
cme. http://www.cmegroup.com/globex/resources/fix-fast-
decoder- agreement.html

If so, how would you rate it’s performance?

It decodes and then generates the fix strings. Our initial
tests show that it takes 1.7ms to do this for each
message. I presuming I should be aiming for much faster
times than this?

Many Thanks.

  • Lee.
11

Hi,

We have been using CME implementation and its test cases for validating our own implementation. In general we found the CME implementation good and compliant. But, you should be aware that it resets its dictionary on every message (as per CME requirement). So if you want to use it for other (non-CME) messages you need to fix the code.

We have our own Java implementation for FAST which decodes about 150-200K CME messages/sec (incremental refresh with 5 entries). The output of the decoding process is a Java Bean.

Dell Dual Core using JRE1.5 -server -mx256m -ms200m

Thanks

Krishnan

Guys,

Has anyone used the FastFix Decoder available from the cme. http://www.cmegroup.com/globex/resources/fix-fast-decoder-
agreement.html

If so, how would you rate it’s performance?

It decodes and then generates the fix strings. Our initial tests show
that it takes 1.7ms to do this for each message. I presuming I should be
aiming for much faster times than this?

Many Thanks.

  • Lee.
12

Krishnan

150-200K msg/s? Is that just decoding to raw values, or creating the fix strings aswell?

Using the cme decoder to decode and create the fix strings on a cme increment message takes ~300 microseconds. This doesn’t seem fast enough to me.

Many thanks.

  • Lee.

Hi,

We have been using CME implementation and its test cases for validating
our own implementation. In general we found the CME implementation good
and compliant. But, you should be aware that it resets its dictionary on
every message (as per CME requirement). So if you want to use it for
other (non-CME) messages you need to fix the code.

We have our own Java implementation for FAST which decodes about 150-
200K CME messages/sec (incremental refresh with 5 entries). The output
of the decoding process is a Java Bean.

Dell Dual Core using JRE1.5 -server -mx256m -ms200m

Thanks

Krishnan

Guys,

Has anyone used the FastFix Decoder available from the cme. http://www.cmegroup.com/globex/resources/fix-fast-decoder-
agreement.html

If so, how would you rate it’s performance?

It decodes and then generates the fix strings. Our initial tests show
that it takes 1.7ms to do this for each message. I presuming I should
be aiming for much faster times than this?

Many Thanks.

  • Lee.
13

Hi,

150-200K messages/sec or ~6 micro seconds/message for decoding the message to a Java object (basically decoding raw values only). We don’t convert it to FIX.

Someone else in this thread have mentioned that CME decoder gives about 35 microseconds /message for decoding to raw values (no FIX conversion) so 6 micros is not bad.

Regards,
Krishnan

Krishnan

150-200K msg/s? Is that just decoding to raw values, or creating the fix
strings aswell?

Using the cme decoder to decode and create the fix strings on a cme
increment message takes ~300 microseconds. This doesn’t seem fast
enough to me.

Many thanks.

  • Lee.

Hi,

We have been using CME implementation and its test cases for
validating our own implementation. In general we found the CME
implementation good and compliant. But, you should be aware that it
resets its dictionary on every message (as per CME requirement). So
if you want to use it for other (non-CME) messages you need to fix
the code.

We have our own Java implementation for FAST which decodes about 150-
200K CME messages/sec (incremental refresh with 5 entries). The output
of the decoding process is a Java Bean.

Dell Dual Core using JRE1.5 -server -mx256m -ms200m

Thanks

Krishnan

Guys,

Has anyone used the FastFix Decoder available from the cme. http://www.cmegroup.com/globex/resources/fix-fast-decoder-
agreement.html

If so, how would you rate it’s performance?

It decodes and then generates the fix strings. Our initial tests
show that it takes 1.7ms to do this for each message. I presuming I
should be aiming for much faster times than this?

Many Thanks.

  • Lee.
14

Pantor I believe have one of the fastest implementation, at about 3M CME msg / sec, decode only. My own implementation currently does abt 750k CME msg / sec, decode only, on my Thinkpad T60 (core 2 duo, 2 Ghz). For my implementation, I don’t create the FIX string, just into an internal data structure.

Just FYI.

Krishnan

150-200K msg/s? Is that just decoding to raw values, or creating the fix
strings aswell?

Using the cme decoder to decode and create the fix strings on a cme
increment message takes ~300 microseconds. This doesn’t seem fast
enough to me.

Many thanks.

  • Lee.
15

Hi,

I am aware of Pantor’s numbers; the one I had quoted was in Java. We have another in C++ which decodes about 650K/sec. I think the number reported by Pantor (3M) is fantastic.

Since we are comparing performance it is better understand what is included in the decoding process.

a) What is the output of the decoding process. Is it a independent bean like object which can be handed over to the application? Does it have any dependency on the decoder or dictionary?

b) CME has few fields which represent data and time (encoded as integer). Do you convert the integer/string value to date ?

Thanks,

Krishnan

Pantor I believe have one of the fastest implementation, at about 3M CME
msg / sec, decode only. My own implementation currently does abt 750k
CME msg / sec, decode only, on my Thinkpad T60 (core 2 duo, 2 Ghz). For
my implementation, I don’t create the FIX string, just into an internal
data structure.

Just FYI.

Krishnan

150-200K msg/s? Is that just decoding to raw values, or creating the
fix strings aswell?

Using the cme decoder to decode and create the fix strings on a cme
increment message takes ~300 microseconds. This doesn’t seem fast
enough to me.

Many thanks.

  • Lee.
16

For my own testing, the result just an object with no dependencies, with all the necessary date / price fields converted.

Actually I am not sure if my test is comparable, since it also does a few more things, like looking through a list of “interested” products, does some basic field validation / scrubbing, date fields, price fields (based on a band of previous prices), for those “interested” products (which, take for instance interest rate futs, is most of the products).

Well, it is just more appropriate for our own internal needs, basically it is an “update” ready to be applied to global tick plant, or published out.

Hi,

I am aware of Pantor’s numbers; the one I had quoted was in Java. We
have another in C++ which decodes about 650K/sec. I think the number
reported by Pantor (3M) is fantastic.

Since we are comparing performance it is better understand what is
included in the decoding process.

a) What is the output of the decoding process. Is it a independent bean
like object which can be handed over to the application? Does it have
any dependency on the decoder or dictionary?

b) CME has few fields which represent data and time (encoded as
integer). Do you convert the integer/string value to date ?

Thanks,

Krishnan

Pantor I believe have one of the fastest implementation, at about 3M
CME msg / sec, decode only. My own implementation currently does abt
750k CME msg / sec, decode only, on my Thinkpad T60 (core 2 duo, 2
Ghz). For my implementation, I don’t create the FIX string, just into
an internal data structure.

Just FYI.

Krishnan

150-200K msg/s? Is that just decoding to raw values, or creating the
fix strings aswell?

Using the cme decoder to decode and create the fix strings on a cme
increment message takes ~300 microseconds. This doesn’t seem fast
enough to me.

Many thanks.

  • Lee.
17

Hi All,

Does anyone of you have a figure of how many CME MDP(an older format) msg can be decoded / sec. I would like to compare how efficiency between them. I just wonder if it takes less time in decoding MDP than decoding FIX/FAST.

Thanks,
Alex Leung

Pantor I believe have one of the fastest implementation, at about 3M CME
msg / sec, decode only. My own implementation currently does abt 750k
CME msg / sec, decode only, on my Thinkpad T60 (core 2 duo, 2 Ghz). For
my implementation, I don’t create the FIX string, just into an internal
data structure.

Just FYI.

Krishnan

150-200K msg/s? Is that just decoding to raw values, or creating the
fix strings aswell?

Using the cme decoder to decode and create the fix strings on a cme
increment message takes ~300 microseconds. This doesn’t seem fast
enough to me.

Many thanks.

  • Lee.
18

Alex, if we are just talking abt CME, CME published some statistics during their POC (proof of concept) stage. I can pull out some of their presentations if you want to have a look.

But obviously every proprietary system’s implementation maybe different, for parsing RLC under MDP and FIX/FAST. For my own implementation, there is no doubt that FIX/FAST is faster (now closer to 1M msg/sec), vs old RLC over MDP (slow due to mostly the ZLIB decompression).

Hi All,

Does anyone of you have a figure of how many CME MDP(an older format)
msg can be decoded / sec. I would like to compare how efficiency
between them. I just wonder if it takes less time in decoding MDP than
decoding FIX/FAST.

Thanks, Alex Leung

19

Guys,

Has anyone used the FastFix Decoder available from the cme. http://www.cmegroup.com/globex/resources/fix-fast-decoder-
agreement.html

If so, how would you rate it’s performance?

It decodes and then generates the fix strings. Our initial tests show
that it takes 1.7ms to do this for each message. I presuming I should be
aiming for much faster times than this?

Many Thanks.

  • Lee.

Taking this question a little further, has anyone used the C++ Decoder?

-jd-

20

Guys,

Has anyone used the FastFix Decoder available from the cme. http://www.cmegroup.com/globex/resources/fix-fast-decoder-
agreement.html

If so, how would you rate it’s performance?

It decodes and then generates the fix strings. Our initial tests show
that it takes 1.7ms to do this for each message. I presuming I should
be aiming for much faster times than this?

Many Thanks.

  • Lee.

Taking this question a little further, has anyone used the C++ Decoder?

-jd-
Hello!
I have been evaluating the C++ CME decoder, and found many limitations and bugs.
First it only implements a subset of FAST 1.1 (e.g., no Group Field Instruction); then it’s unable to build codecs from XML containing TemplateRef tags; then it’s been designed to run exactly once on a buffer containing one or more FAST messages – that is many statics, and various states not easily resetables. It is not thread safe at all.
Finally it’s got some bugs, causing it to core dump on some of CME’s own feeds (such as snapshots) (due to incorect PMAP handling).

All this considered, this piece of code is indeed very useful as a “bootstrap” into coding a decoder, but by no mean something useful in production.

I’d recommend either going with a off the shelf product, or develop one from scratch.

Hope this helps,

JS