Proposals from the High Performance Working Group (SBE, GPB, ASN.1)

, , ,
1

Imported from previous forum

2

Three major approaches to the efficient encoding of FIX that have been developed and are being proposed by the High Performance Working Group:-

http://www.fixprotocol.org/encodings

  1. Simple Binary Encoding (SBE) -

Simple Binary Encoding produces fast and compact encodings of FIX messages. SBE provides different characteristics than other binary encodings. It is optimized for low latency. This encoding complements the existing only binary FAST (FIX Adapted for STreaming market data) encoding with a focus on reducing bandwidth utilization for market data. In addition, the encoding is also defined and controlled within FPL only in contrast to the other binary encodings proposals to encode FIX with Google Protocol Buffers and ASN.1.

http://www.fixprotocol.org/encodings#Simple_Binary_Encoding

  1. Google Protocol Buffers (GPB) -

Protocol buffers are Google’s language neutral, platform neutral, extensible mechanism for serializing structured data.

http://www.fixprotocol.org/encodings#Google_Protocol_Buffers_Mapping

  1. Abstract Syntax Notation 1 (ASN.1) -

ASN.1 is a family of International Standards for the definition and encoding of messages, jointly developed and published by the International Organization for Standardization and the International Telecommunication Union. ASN is a standard and notation that describes rules and structures for representing, encoding, transmitting, and decoding data in telecommunications and computer networking.

http://www.fixprotocol.org/encodings#ASN.1_Mapping

I am surprised that FPL Program Office / FIX Protocol Ltd. fpl@fixprotocol.org has NOT announced these proposals in this discussion forum. I am sorry to see these discussions are taking place in the “General Q/A” discussion forum instead of this forum. This “High Performance Interface WG” (previously “High Frequency trading” forum) was specifically created to discuss these.

Regards,
Mahesh

3

Mahesh,

for the past year, most of the discussions within the HPWG have taken place outside of the FPL discussion forums. It was when the proposals were published that it was decided to move back to the FPL forums.

Best,
Rolf

…
I am surprised that FPL Program Office / FIX Protocol Ltd. fpl@fixprotocol.org has NOT announced these proposals in this discussion forum. I am sorry to see these discussions are taking place in the “General Q/A” discussion forum instead of this forum. This “High Performance Interface WG” (previously “High Frequency trading” forum) was specifically created to discuss these.

Regards,
Mahesh

4

Mahesh/Rolf, are any of the approaches being favored over the others?  I was trying to understand if all three approaches were going to be supported, or if one will ultimately prevail.  I realize there are pros/cons to each, so didn't know if there was a single approach that was gaining momentum.

Thanks for any thoughts.

Regards,

-Brian Schneider

5

Brian,

all three proposals will be supported as far as I know, but I would guess that SBE will be favored if it gets traction as it is the only encoding that is under FPL control. The three initiatives are different in nature; the ASN proposal is mature and highly efficient, the GPB proposal is basically a set of rules for automatic template construction based on the FIX repository, and SBE is more of a new encoding although not entirely a green-field exercise.

I initially hoped that Google would participate and add some missing but critical pieces to GPB, but they (Google) told us  that it wasn't going to happen and if they would make any changes it would be to remove GPB features, not add new ones.

The SBE effort will likely gain traction as more designers and implementers begin to review and implement. I believe there is much more to do before the SBE spec is ready for prime time, but CME is putting its weight behind SBE, both by funding its development and by being an early adopter. CME's new market data protocol is roughly based on the pre-release SBE spec if I have understood correctly.

Fred Malabre and Hanno Klein are running the working group so they would be the guys to talk to to get more info.

Hope this helps,

Rolf

6

Mahesh, just a quick note on your comment that ASN cannot perform best for a single need.

Alessandro, who is heading the ASN encoding subgroup, has demonstrated that their proposal has very high performance. I suggest you review their test results.

My view is that ASN can pretty much do anything that you would like to do in SBE. They just do it within a different framwork.

Re SBE controlled by FPL - I believe this will be the top argument for not re-using an external encoding. The drawback is that FPL needs to make sure that the SBE encoding is complete, well-defined, verified, documented and that there is support available, possibly also good open-source implementations for the major programming environments. Time will tell.

We have an earlier example in FAST - it was widely adopted very quickly, but it also suffered from underspecification and dialects were created early on.

/Rolf

7

How to use the SBE tools to generate the C++ code

8

SBE is currently only a specification that has been published here: http://www.fixtradingcommunity.org/pg/structure/tech-specs/simple-binary-encoding. What “SBE tools” are you referring to? Do you have a link?

9

Here is the link for the SBE open source.

10

Hi Klein,Thank you for your reply，Here is the link for the SBE tools：

“The tool will accept options using the system property option format as follows:
java [-Doption=value] -jar sbe.jar <message-declarations-file.xml>”

I use this command：
java -sbe.target.language=Cpp98 -jar sbe.jar my.xml
but Without success
use command “java -jar sbe.jar my.xml“ is success，but defaults to Java，I need C++ code

11

I see that the open source on github supposedly also supports Cpp98 and CSharp. FIX Trading Community is not providing the open source, i.e. please use the github website to get in touch with those that have provided the software. I am not familiar with it, maybe others can help.

12

Hello,

You guys should consider some of the common formats for transport as there are existing software to decode these message types:

Simple Binary Encoding
Google Protocol Buffers
Google FlatBuffers
Apache Thrift
Apache Avro
MessagePack
Cap’n Proto

Some would outperform other but API simplicity can be sometimes the deciding factor when latency is not the biggest deciding factor. Also users have a choice.

Suminda