Imported from previous forum
Is anyone decoding and parsing CME FIX/FAST feed?
We have written our own FAST decoder, and we notice grossly out of sequence values for RptSeq(83).
Is anyone parsing CMEs FIX/FAST feed using their own FAST decoder?
Franz,
if you get the other fields correctly, then you most probably have your template almost correct. Could be that you have specified an operator for the RptSeq field that doesn’t match the CME spec.
I suggest you check with the CME support people.
Best,
Rolf
Is anyone decoding and parsing CME FIX/FAST feed? We have written
our own FAST decoder, and we notice grossly out of sequence values
for RptSeq(83). Is anyone parsing CMEs FIX/FAST feed using their own
FAST decoder?
Is anyone decoding and parsing CME FIX/FAST feed? We have written
our own FAST decoder, and we notice grossly out of sequence values
for RptSeq(83). Is anyone parsing CMEs FIX/FAST feed using their own
FAST decoder?
Yes, I have a system that went into production with CME FIX/FAST last week. There are some problems in the certification environment, so it seems, that requires market data recovery. But so far, that is not the case with the production feed. Cheers.
I believe I read the spec incorrectly. I would just like to confirm that when decoding Delta field operations I should always decode the value as an Int64, regardless of the field type. However, the result value after the Delta operation has been applied must conform the the field type.
Is that correct?
Is anyone decoding and parsing CME FIX/FAST feed? We have written
our own FAST decoder, and we notice grossly out of sequence values
for RptSeq(83). Is anyone parsing CMEs FIX/FAST feed using their own
FAST decoder?
Is anyone decoding and parsing CME FIX/FAST feed? We have written
our own FAST decoder, and we notice grossly out of sequence values
for RptSeq(83). Is anyone parsing CMEs FIX/FAST feed using their own
FAST decoder?
I am also trying to write a decoder for CME FIX/FAST feed in C language.
The values for RptSeq(83) in the decoded messages are in sequence. But I have another problem.
When decoding the repeating groups, the decoder first extracts a Pmap from data. But in the Pmap, there is not enough number of bits to cover all the fields in the template which require Pmap bits.
Any help would be appreciated. Thanks a lot.
FAST PMAPs are supposed to be terminated with infinite number of 0’s (zeroes).
So as you decode the PMAP (from left to right) you can then assume that every other field is 0.
From the Spec, section 10.5 Presence Map:
“A presence map is represented as a stop bit encoded entity. Logically a presence map has an infinite suffix of zeroes. This makes it possible to truncate a presence map that ends in a sequence where the bits are all zero. The length of the remaining part must be a multiple of seven.”
Is anyone decoding and parsing CME FIX/FAST feed? We have written our
own FAST decoder, and we notice grossly out of sequence values for
RptSeq(83). Is anyone parsing CMEs FIX/FAST feed using their own FAST
decoder?I am also trying to write a decoder for CME FIX/FAST feed in C language.
The values for RptSeq(83) in the decoded messages are in sequence. But I
have another problem.When decoding the repeating groups, the decoder first extracts a Pmap
from data. But in the Pmap, there is not enough number of bits to cover
all the fields in the template which require Pmap bits.Any help would be appreciated. Thanks a lot.
Is anyone decoding and parsing CME FIX/FAST feed? We have written
our own FAST decoder, and we notice grossly out of sequence values
for RptSeq(83). Is anyone parsing CMEs FIX/FAST feed using their own
FAST decoder?
Just a question for you on performance. Have you tried measure the performance with respect to decoding CME FIX/FAST messages? We have a C++ decoder and to decode each message into field name/value pairs, we could only reach ~100,000msg/sec on average
Is anyone decoding and parsing CME FIX/FAST feed? We have written our
own FAST decoder, and we notice grossly out of sequence values for
RptSeq(83). Is anyone parsing CMEs FIX/FAST feed using their own FAST
decoder?Just a question for you on performance. Have you tried measure the
performance with respect to decoding CME FIX/FAST messages? We have a
C++ decoder and to decode each message into field name/value pairs, we
could only reach ~100,000msg/sec on average
100k msg / sec seems very low. I just whipped up a quick perf test stub, it contains abt 80-100 diff msgs (some with long-ish seqs), and randomly chose one to encode / decode, for 1/3/5M messages. It uses a simple single thread test app, On my Thinkpad T60, Core 2 Duo T2500 (2 Ghz), it can do abt 500k msgs / sec (avg between 9 sec 670 ms and abt 9 sec 980 ms for 5M msg iterations), (compile flag “-O2”), or abt 2 microsec per msg. I believe Rolf @ Pantor has done some perf testing that they can do encode / decode in the 100ns range.
My parser is written in C, but for modern compilers the performance diff between C and C++ should be negligible.
Is anyone decoding and parsing CME FIX/FAST feed? We have written our
own FAST decoder, and we notice grossly out of sequence values for
RptSeq(83). Is anyone parsing CMEs FIX/FAST feed using their own FAST
decoder?Just a question for you on performance. Have you tried measure the
performance with respect to decoding CME FIX/FAST messages? We have a
C++ decoder and to decode each message into field name/value pairs, we
could only reach ~100,000msg/sec on average
Pantor’s description of their commercial version:
http://www.pantor.com/software.html
On a per-session basis, the performance is roughly 100k msg / sec, scaled linearly with number of sessions.
Is anyone decoding and parsing CME FIX/FAST feed? We have written
our own FAST decoder, and we notice grossly out of sequence values
for RptSeq(83). Is anyone parsing CMEs FIX/FAST feed using their own
FAST decoder?Just a question for you on performance. Have you tried measure the
performance with respect to decoding CME FIX/FAST messages? We have a
C++ decoder and to decode each message into field name/value pairs, we
could only reach ~100,000msg/sec on averagePantor’s description of their commercial version:
On a per-session basis, the performance is roughly 100k msg / sec,
scaled linearly with number of sessions.
The performance level that you quote from our home page includes conversion of large messages into FIX text format, where a benchmark of the CME feed would be more relevant to decode into an internal format.
Testing on a beta of our next FVM (FAST Virtual Machine) release, a 45 nm 2.83 Ghz XEON decodes >3 million CME FIX/FAST msgs/s. FVM takes the template XML file as command line input, i.e. it not necessary to recompile and redeploy a new decoder version to handle a template change. FVM also supports receiving templates dynamically, over SCP, resulting in the same decoder performance. By plugging in statically tuned decoder logic for a specific template, FVM can be made to run faster. As a comparison, our static Presto decoder was demonstrated to decode >10 million OPRA or ARCA msgs/s, resulting in the 100 ns per message latency quoted in an earlier message.
Is anyone decoding and parsing CME FIX/FAST feed? We have written
our own FAST decoder, and we notice grossly out of sequence values
for RptSeq(83). Is anyone parsing CMEs FIX/FAST feed using their
own FAST decoder?Just a question for you on performance. Have you tried measure the
performance with respect to decoding CME FIX/FAST messages? We have
a C++ decoder and to decode each message into field name/value
pairs, we could only reach ~100,000msg/sec on averagePantor’s description of their commercial version:
On a per-session basis, the performance is roughly 100k msg / sec,
scaled linearly with number of sessions.The performance level that you quote from our home page includes
conversion of large messages into FIX text format, where a benchmark of
the CME feed would be more relevant to decode into an internal format.Testing on a beta of our next FVM (FAST Virtual Machine) release, a 45
nm 2.83 Ghz XEON decodes >3 million CME FIX/FAST msgs/s. FVM takes the
template XML file as command line input, i.e. it not necessary to
recompile and redeploy a new decoder version to handle a template
change. FVM also supports receiving templates dynamically, over SCP,
resulting in the same decoder performance. By plugging in statically
tuned decoder logic for a specific template, FVM can be made to run
faster. As a comparison, our static Presto decoder was demonstrated to
decode >10 million OPRA or ARCA msgs/s, resulting in the 100 ns per
message latency quoted in an earlier message.
I am curious how performance is measured in your tests. What is the configuration of the application. i.e., are the msgs received over SCP (or TCP), how many sessions are there in the test application, etc. I am just curious. Thanks
Is anyone decoding and parsing CME FIX/FAST feed? We have
written our own FAST decoder, and we notice grossly out of
sequence values for RptSeq(83). Is anyone parsing CMEs FIX/FAST
feed using their own FAST decoder?Just a question for you on performance. Have you tried measure the
performance with respect to decoding CME FIX/FAST messages? We
have a C++ decoder and to decode each message into field
name/value pairs, we could only reach ~100,000msg/sec on averagePantor’s description of their commercial version:
pantor.com - pantor Resources and Information.
On a per-session basis, the performance is roughly 100k msg / sec,
scaled linearly with number of sessions.The performance level that you quote from our home page includes
conversion of large messages into FIX text format, where a
benchmark of the CME feed would be more relevant to decode into an
internal format.Testing on a beta of our next FVM (FAST Virtual Machine) release, a 45
nm 2.83 Ghz XEON decodes >3 million CME FIX/FAST msgs/s. FVM takes the
template XML file as command line input, i.e. it not necessary to
recompile and redeploy a new decoder version to handle a template
change. FVM also supports receiving templates dynamically, over SCP,
resulting in the same decoder performance. By plugging in statically
tuned decoder logic for a specific template, FVM can be made to run
faster. As a comparison, our static Presto decoder was demonstrated to
decode >10 million OPRA or ARCA msgs/s, resulting in the 100 ns per
message latency quoted in an earlier message.I am curious how performance is measured in your tests. What is the
configuration of the application. i.e., are the msgs received over SCP
(or TCP), how many sessions are there in the test application, etc. I am
just curious. Thanks
The messages were recorded from the interactive test feed, CME Equity Futures, channel 7 into a file sometime around December 11, 2007. The file contains 350,000 messages. A single thread / a single core was used to decode the file, but multiple cores could be used to share the load from multiple channels to scale further.
Is anyone decoding and parsing CME FIX/FAST feed? We have
written our own FAST decoder, and we notice grossly out of
sequence values for RptSeq(83). Is anyone parsing CMEs
FIX/FAST feed using their own FAST decoder?Just a question for you on performance. Have you tried measure
the performance with respect to decoding CME FIX/FAST messages?
We have a C++ decoder and to decode each message into field
name/value pairs, we could only reach ~100,000msg/sec on averagePantor’s description of their commercial version:
On a per-session basis, the performance is roughly 100k msg / sec,
scaled linearly with number of sessions.The performance level that you quote from our home page includes
conversion of large messages into FIX text format, where a benchmark
of the CME feed would be more relevant to decode into an internal
format.Testing on a beta of our next FVM (FAST Virtual Machine) release, a
45 nm 2.83 Ghz XEON decodes >3 million CME FIX/FAST msgs/s. FVM
takes the template XML file as command line input, i.e. it not
necessary to recompile and redeploy a new decoder version to handle
a template change. FVM also supports receiving templates
dynamically, over SCP, resulting in the same decoder performance. By
plugging in statically tuned decoder logic for a specific template,
FVM can be made to run faster. As a comparison, our static Presto
decoder was demonstrated to decode >10 million OPRA or ARCA msgs/s,
resulting in the 100 ns per message latency quoted in an earlier
message.I am curious how performance is measured in your tests. What is the
configuration of the application. i.e., are the msgs received over SCP
(or TCP), how many sessions are there in the test application, etc. I
am just curious. ThanksThe messages were recorded from the interactive test feed, CME Equity
Futures, channel 7 into a file sometime around December 11, 2007. The
file contains 350,000 messages. A single thread / a single core was used
to decode the file, but multiple cores could be used to share the load
from multiple channels to scale further.
Thanks everyone for the information. I think my own measurement wasn’t done efficiently (e.g. I meaured the time it takes to decode every message) plus the box I was running was old too. However, from the numbers I gathered above, it looks like there is room for improvement, so I will try to profile the library and make it faster.
Thanks again
Is anyone decoding and parsing CME FIX/FAST feed? We have written
our own FAST decoder, and we notice grossly out of sequence values
for RptSeq(83). Is anyone parsing CMEs FIX/FAST feed using their own
FAST decoder?
I am currently decoding the certification feed and am having a few problems. Its always worth calling or emailing the CSET helpdesk as they are very helpful.
I hope the certification feed isnt sending out incorrect information as this will really slow me down.
Hi,
I have a open Fast.net library.I want to read market data using open Fast. I don’t know how do I connect to market data provider using open fast library. please can anyone guide me.if possible send sample example for it.
Thanks
Karthik.
please read the recent thread started by your colleague Rama krishna Boyina
Best,
Rolf
Hi,
I have a open Fast.net library.I want to read market data using open Fast. I don’t know how do I connect to market data provider using open fast library. please can anyone guide me.if possible send sample example for it.Thanks
Karthik.