Frame Length

, ,
1

Imported from previous forum

2

[ original email was from Greg Orsini - go@cameronsystems.com ]
I would like to ‘frame’ the argument around including a frame length in SCP to encapsulate FAST messages sent over a stream.

There are several factors which weigh for and against its inclusion and it is important to document these as well as our final decision. To that end, I would like to start a list of pros and cons for “Frame Length”, highlighting safety and simplicity vs. cost.

Pros

  1. Analogous to current FIX philosophy starting FIX messages with tag 8 and 9 (version and length). This would allow the same separation of reader and parser that classic FIX allows. A state driven reader can accomplish the same thing, without doubt, but it is simpler to not require this.

  2. Easier to implement a MAX SIZE limitation to a message if you know how big the incoming message is in advance. There are good reasons to implement a limit. Given a 4 MB message (not really uncommon), one would see the frame length and make allocations as necessary. If the frame length were 1GB, you might decide not to allocate for it, but issue an alert instead and skip the message.

If, however, there is no frame length, trying to receive a 1GB message could prove quite taxing as one tries to allocate more and more storage to accommodate the message until finally giving up (perhaps because exceeds available memory or a configured limit) and freeing all that storage. In this way, we have let a remote process needlessly affect our performance. Mischievous trading partners could use this behavior to their advantage.

  1. Security. Buffer overrun vulnerability discussion and attacks are the rage these days. Creating a MAX SIZE could be important for applications that can import unbounded message data into memory. Overrunning a buffer would be more likely if an attacker did not have to declare a size from the start.

Cons

  1. Redundant Data. With a message payload size of 100 bytes and IP overhead of 42 bytes, the overhead of SCP frame length is 1/(42+1+100) = 1/143 => ~0.7% (adapted shamelessly from one of Rolf’s earlier postings).

  2. Cannot start transmission of packet until length is known. (Not relevant for FAST/FIX, however, as the size and checksum must already be computed.)

The cost of a frame length is small and it simplifies a number important safety and programming issues. While not strictly necessary, it is simpler. Simpler means more people will get it right.

Greg.

3

Regarding “easier to implement a max size limitation”, I would agree that the reader should do bounds checking. Is it really hard to do without the frame length though?