Best Java API for FAST

kapilawitharana_x · September 11, 2012, 3:50am

Imported from previous forum

kapilawitharana_x · September 11, 2012, 3:50am

Hi All,

What is the BEST (in-terms of performance/data load) Java API/Lib for FAST (commercial/non-commercial)?

Really appreciate if you can share your experience.

Thanks;

fplpo_x · September 11, 2012, 4:52am

[ original email was from janaka Sooriyaarachchi - janakas@millenniumit.com ]
OpenFAST is better (I can’t say this is the best).

http://www.openfast.org/

Hi All,

What is the BEST (in-terms of performance/data load) Java API/Lib for FAST (commercial/non-commercial)?

Really appreciate if you can share your experience.

Thanks;

kapilawitharana_x · September 11, 2012, 9:34am

I have used OpenFast but it does not decode message when data load is high. That’s why I thought to use other FAST framework or API.

If you can let me know any other good Java API for FAST, it will be really helpful.

OpenFAST is better (I can’t say this is the best).

http://www.openfast.org/

Hi All,

What is the BEST (in-terms of performance/data load) Java API/Lib for FAST (commercial/non-commercial)?

Really appreciate if you can share your experience.

Thanks;

fplpo_x · September 11, 2012, 10:16am

[ original email was from janaka Sooriyaarachchi - janakas@millenniumit.com ]
How is message rate of your data feed and what is the protocol (tcp/udp) ?

I don’t think data rate will impact on the functionality of OpenFast library.

I have used OpenFast but it does not decode message when data load is high. That’s why I thought to use other FAST framework or API.

If you can let me know any other good Java API for FAST, it will be really helpful.

OpenFAST is better (I can’t say this is the best).

http://www.openfast.org/

Hi All,

What is the BEST (in-terms of performance/data load) Java API/Lib for FAST (commercial/non-commercial)?

Really appreciate if you can share your experience.

Thanks;

kapilawitharana_x · September 12, 2012, 6:06am

Thanks a lot Janaka for replying.

Actually, we get thousands messages within few seconds on TCP channel.

I have copied my classes below.

This is my message reader class
…

public class MessageReader {

private final MessageInputStream messageIn;
protected MessageBlockReaderFactory messageBlockReaderFactory;



public MessageReader(InputStream inputStream, TemplateStore templateStore, int readOffset) {

	messageIn = new MessageInputStream(inputStream);
	messageIn.setTemplateRegistry(templateStore.getTemplateRegistry());
	messageBlockReaderFactory = new MessageBlockReaderFactory(readOffset);
	MessageBlockReader msgBlockReader = messageBlockReaderFactory.create();
	messageIn.setBlockReader(msgBlockReader);
	messageIn.addMessageHandler(templateStore.getTemplateRegistry().get("Reset"), RESET_HANDLER);
	messageIn.reset();

}

public synchronized Message readMessage() {

	Message m = messageIn.readMessage();

	return m;

}

public void reset() {
	messageIn.reset();
}

public void close() {
	messageIn.close();
}

public int getRemainingBytes() {
	try {
		return messageIn.getUnderlyingStream().available();
	} catch (IOException e) {
	}
	return -1;
}

private static final MessageHandler RESET_HANDLER = new MessageHandler() {
	@Override
	public void handleMessage(Message readMessage, Context context, Coder coder) {
		coder.reset();
	}
};

}

…

This is how I read messages

…

final MessageReader reader = new MessageReader(bufferedInputStream, templateStore, 1);

	new Thread() {
		public void run() {
			while (true) {

				try {
					
					Message message = reader.readMessage();

					if (message == null) {
						break;
					}


					handleMessage(message);


				} catch (FastException exception) {
					exception.printStackTrace();
					
				}
			}
		}
	}.start();

…

Error messages we received when reading messages on TCP channel.

…

org.openfast.error.FastException: An error occurred while decoding SecurityDefinition
at org.openfast.template.MessageTemplate.decode(MessageTemplate.java:133)
at org.openfast.codec.FastDecoder.readMessage(FastDecoder.java:82)
at org.openfast.MessageInputStream.readMessage(MessageInputStream.java:65)
Caused by: org.openfast.error.FastException: Error occurred while decoding Scalar [name=NoSecurityAltID, operator=none, type=uInt32, dictionary=SecurityDefinition]
at org.openfast.template.Scalar.decode(Scalar.java:246)
at org.openfast.template.Sequence.decode(Sequence.java:200)
at org.openfast.template.Group.decodeFieldValues(Group.java:293)
at org.openfast.template.MessageTemplate.decode(MessageTemplate.java:126)
… 3 more
Caused by: org.openfast.error.FastException: The value 3486042861337320496 is out of range for type uInt32
at org.openfast.error.ErrorCode.throwException(ErrorCode.java:45)
at org.openfast.error.ErrorHandler$1.error(ErrorHandler.java:26)
at org.openfast.Global.handleError(Global.java:41)
at org.openfast.template.type.IntegerType.validateValue(IntegerType.java:88)
at org.openfast.template.Scalar.validateDecodedValueIsCorrectForType(Scalar.java:260)
at org.openfast.template.Scalar.decode(Scalar.java:240)
… 6 more

…

It would be really great if you can help me to resolve this issue as I have already spent 3 weeks to solve this issue.

Thanks,

How is message rate of your data feed and what is the protocol (tcp/udp) ?

I don’t think data rate will impact on the functionality of OpenFast library.

I have used OpenFast but it does not decode message when data load is high. That’s why I thought to use other FAST framework or API.

If you can let me know any other good Java API for FAST, it will be really helpful.

OpenFAST is better (I can’t say this is the best).

http://www.openfast.org/

Hi All,

What is the BEST (in-terms of performance/data load) Java API/Lib for FAST (commercial/non-commercial)?

Really appreciate if you can share your experience.

Thanks;

fplpo_x · September 12, 2012, 6:32am

[ original email was from janaka Sooriyaarachchi - janakas@millenniumit.com ]
As per the error message this is not a issue which related to data rate.
This relates to message decoding and template miss-matching.

So Check the following points,

XML template file is comply with the target Data Source. If you try to decode FAST Message with incorrect Template you will get this Error. Consider the versions.
Check whether FAST message contains a Preamble data. Because some data providers add Non-FAST fix Length Headers to Message. You need to remove those data before decode the message.
Check that Data feed contains a Message length field at the begining of the message. Open FAST dose not work with such Data Feeds. You need to decode Message length manually(you can use field decoding functions in OPEN FAST library for that task) and try to decode only the message segment.

Thanks a lot Janaka for replying.

Actually, we get thousands messages within few seconds on TCP channel.

I have copied my classes below.

This is my message reader class
…

public class MessageReader {

private final MessageInputStream messageIn;
protected MessageBlockReaderFactory messageBlockReaderFactory;

public MessageReader(InputStream inputStream, TemplateStore templateStore, int readOffset) {
  messageIn = new MessageInputStream(inputStream);
  messageIn.setTemplateRegistry(templateStore.getTemplateRegistry());
  messageBlockReaderFactory = new MessageBlockReaderFactory(readOffset);
  MessageBlockReader msgBlockReader = messageBlockReaderFactory.create();
  messageIn.setBlockReader(msgBlockReader);
  messageIn.addMessageHandler(templateStore.getTemplateRegistry().get("Reset"), RESET_HANDLER);
  messageIn.reset();
}

public synchronized Message readMessage() {
  Message m = messageIn.readMessage();

  return m;
}

public void reset() {
messageIn.reset();
}

public void close() {
messageIn.close();
}

public int getRemainingBytes() {
try {
return messageIn.getUnderlyingStream().available();
} catch (IOException e) {
}
return -1;
}

private static final MessageHandler RESET_HANDLER = new MessageHandler() {
@Override
public void handleMessage(Message readMessage, Context context, Coder coder) {
coder.reset();
}
};

}

…

This is how I read messages

…

final MessageReader reader = new MessageReader(bufferedInputStream, templateStore, 1);
  new Thread() {
  	public void run() {
  		while (true) {

  			try {
  				
  				Message message = reader.readMessage();

  				if (message == null) {
  					break;
  				}


  				handleMessage(message);


  			} catch (FastException exception) {
  				exception.printStackTrace();
  				
  			}
  		}
  	}
  }.start();
…

Error messages we received when reading messages on TCP channel.

…

org.openfast.error.FastException: An error occurred while decoding SecurityDefinition
at org.openfast.template.MessageTemplate.decode(MessageTemplate.java:133)
at org.openfast.codec.FastDecoder.readMessage(FastDecoder.java:82)
at org.openfast.MessageInputStream.readMessage(MessageInputStream.java:65)
Caused by: org.openfast.error.FastException: Error occurred while decoding Scalar [name=NoSecurityAltID, operator=none, type=uInt32, dictionary=SecurityDefinition]
at org.openfast.template.Scalar.decode(Scalar.java:246)
at org.openfast.template.Sequence.decode(Sequence.java:200)
at org.openfast.template.Group.decodeFieldValues(Group.java:293)
at org.openfast.template.MessageTemplate.decode(MessageTemplate.java:126)
… 3 more
Caused by: org.openfast.error.FastException: The value 3486042861337320496 is out of range for type uInt32
at org.openfast.error.ErrorCode.throwException(ErrorCode.java:45)
at org.openfast.error.ErrorHandler$1.error(ErrorHandler.java:26)
at org.openfast.Global.handleError(Global.java:41)
at org.openfast.template.type.IntegerType.validateValue(IntegerType.java:88)
at org.openfast.template.Scalar.validateDecodedValueIsCorrectForType(Scalar.java:260)
at org.openfast.template.Scalar.decode(Scalar.java:240)
… 6 more

…

It would be really great if you can help me to resolve this issue as I have already spent 3 weeks to solve this issue.

Thanks,

How is message rate of your data feed and what is the protocol (tcp/udp) ?

I don’t think data rate will impact on the functionality of OpenFast library.

I have used OpenFast but it does not decode message when data load is high. That’s why I thought to use other FAST framework or API.

If you can let me know any other good Java API for FAST, it will be really helpful.

OpenFAST is better (I can’t say this is the best).

http://www.openfast.org/

Hi All,

What is the BEST (in-terms of performance/data load) Java API/Lib for FAST (commercial/non-commercial)?

Really appreciate if you can share your experience.

Thanks;

kapilawitharana_x · September 12, 2012, 7:12am

Thank you very much Janaka.

I checked XML Templates, both are identical.

Could you please let me know how to archive 2nd and 3rd points, as I’m new to OpenFast? It would be really great if you share some resources to understand this.

Thanks a lot again.

As per the error message this is not a issue which related to data rate.
This relates to message decoding and template miss-matching.

So Check the following points,

XML template file is comply with the target Data Source. If you try to decode FAST Message with incorrect Template you will get this Error. Consider the versions.

Check whether FAST message contains a Preamble data. Because some data providers add Non-FAST fix Length Headers to Message. You need to remove those data before decode the message.

Check that Data feed contains a Message length field at the begining of the message. Open FAST dose not work with such Data Feeds. You need to decode Message length manually(you can use field decoding functions in OPEN FAST library for that task) and try to decode only the message segment.
Thanks a lot Janaka for replying.

Actually, we get thousands messages within few seconds on TCP channel.

I have copied my classes below.

This is my message reader class
…

public class MessageReader {
private final MessageInputStream messageIn;
protected MessageBlockReaderFactory messageBlockReaderFactory;



public MessageReader(InputStream inputStream, TemplateStore templateStore, int readOffset) {

  messageIn = new MessageInputStream(inputStream);
  messageIn.setTemplateRegistry(templateStore.getTemplateRegistry());
  messageBlockReaderFactory = new MessageBlockReaderFactory(readOffset);
  MessageBlockReader msgBlockReader = messageBlockReaderFactory.create();
  messageIn.setBlockReader(msgBlockReader);
  messageIn.addMessageHandler(templateStore.getTemplateRegistry().get("Reset"), RESET_HANDLER);
  messageIn.reset();

}

public synchronized Message readMessage() {

  Message m = messageIn.readMessage();

  return m;

}

public void reset() {
  messageIn.reset();
}

public void close() {
  messageIn.close();
}

public int getRemainingBytes() {
  try {
  	return messageIn.getUnderlyingStream().available();
  } catch (IOException e) {
  }
  return -1;
}

private static final MessageHandler RESET_HANDLER = new MessageHandler() {
  @Override
  public void handleMessage(Message readMessage, Context context, Coder coder) {
  	coder.reset();
  }
};
}

…

This is how I read messages

…

final MessageReader reader = new MessageReader(bufferedInputStream, templateStore, 1);
  new Thread() {
  	public void run() {
  		while (true) {

  			try {
  				
  				Message message = reader.readMessage();

  				if (message == null) {
  					break;
  				}


  				handleMessage(message);


  			} catch (FastException exception) {
  				exception.printStackTrace();
  				
  			}
  		}
  	}
  }.start();
…

Error messages we received when reading messages on TCP channel.

…

org.openfast.error.FastException: An error occurred while decoding SecurityDefinition
at org.openfast.template.MessageTemplate.decode(MessageTemplate.java:133)
at org.openfast.codec.FastDecoder.readMessage(FastDecoder.java:82)
at org.openfast.MessageInputStream.readMessage(MessageInputStream.java:65)
Caused by: org.openfast.error.FastException: Error occurred while decoding Scalar [name=NoSecurityAltID, operator=none, type=uInt32, dictionary=SecurityDefinition]
at org.openfast.template.Scalar.decode(Scalar.java:246)
at org.openfast.template.Sequence.decode(Sequence.java:200)
at org.openfast.template.Group.decodeFieldValues(Group.java:293)
at org.openfast.template.MessageTemplate.decode(MessageTemplate.java:126)
… 3 more
Caused by: org.openfast.error.FastException: The value 3486042861337320496 is out of range for type uInt32
at org.openfast.error.ErrorCode.throwException(ErrorCode.java:45)
at org.openfast.error.ErrorHandler$1.error(ErrorHandler.java:26)
at org.openfast.Global.handleError(Global.java:41)
at org.openfast.template.type.IntegerType.validateValue(IntegerType.java:88)
at org.openfast.template.Scalar.validateDecodedValueIsCorrectForType(Scalar.java:260)
at org.openfast.template.Scalar.decode(Scalar.java:240)
… 6 more

…

It would be really great if you can help me to resolve this issue as I have already spent 3 weeks to solve this issue.

Thanks,

How is message rate of your data feed and what is the protocol (tcp/udp) ?

I don’t think data rate will impact on the functionality of OpenFast library.

I have used OpenFast but it does not decode message when data load is high. That’s why I thought to use other FAST framework or API.

If you can let me know any other good Java API for FAST, it will be really helpful.

OpenFAST is better (I can’t say this is the best).

http://www.openfast.org/

Hi All,

What is the BEST (in-terms of performance/data load) Java API/Lib for FAST (commercial/non-commercial)?

Really appreciate if you can share your experience.

Thanks;

kapilawitharana_x · September 12, 2012, 7:27am

Just want to highlight one thing.

I’m not getting this error every time. Sometime, my application works without any error, but sometime we get this decoding error after processing few messages.

Also, I have skipped first bit (message length) before decoding.

…

return new MessageBlockReader() {
@Override
public void messageRead(InputStream in, Message message) {
}

		@Override
		public boolean readBlock(InputStream in) {
			try {
				in.skip(offset);
				return true;
			} catch (IOException e) {
				e.printStackTrace();
				return false;
			}
		}

		@Override
		public String toString() {
			return "(default block reader, offset=" + offset + ")";
		}
	};

…

Do you see any issues on above code?

Thanks;

Thank you very much Janaka.

I checked XML Templates, both are identical.

Could you please let me know how to archive 2nd and 3rd points, as I’m new to OpenFast? It would be really great if you share some resources to understand this.

Thanks a lot again.
As per the error message this is not a issue which related to data rate.
This relates to message decoding and template miss-matching.

So Check the following points,

XML template file is comply with the target Data Source. If you try to decode FAST Message with incorrect Template you will get this Error. Consider the versions.

Check whether FAST message contains a Preamble data. Because some data providers add Non-FAST fix Length Headers to Message. You need to remove those data before decode the message.

Check that Data feed contains a Message length field at the begining of the message. Open FAST dose not work with such Data Feeds. You need to decode Message length manually(you can use field decoding functions in OPEN FAST library for that task) and try to decode only the message segment.
Thanks a lot Janaka for replying.

Actually, we get thousands messages within few seconds on TCP channel.

I have copied my classes below.

This is my message reader class
…

public class MessageReader {
private final MessageInputStream messageIn;
protected MessageBlockReaderFactory messageBlockReaderFactory;



public MessageReader(InputStream inputStream, TemplateStore templateStore, int readOffset) {

  messageIn = new MessageInputStream(inputStream);
  messageIn.setTemplateRegistry(templateStore.getTemplateRegistry());
  messageBlockReaderFactory = new MessageBlockReaderFactory(readOffset);
  MessageBlockReader msgBlockReader = messageBlockReaderFactory.create();
  messageIn.setBlockReader(msgBlockReader);
  messageIn.addMessageHandler(templateStore.getTemplateRegistry().get("Reset"), RESET_HANDLER);
  messageIn.reset();

}

public synchronized Message readMessage() {

  Message m = messageIn.readMessage();

  return m;

}

public void reset() {
  messageIn.reset();
}

public void close() {
  messageIn.close();
}

public int getRemainingBytes() {
  try {
  	return messageIn.getUnderlyingStream().available();
  } catch (IOException e) {
  }
  return -1;
}

private static final MessageHandler RESET_HANDLER = new MessageHandler() {
  @Override
  public void handleMessage(Message readMessage, Context context, Coder coder) {
  	coder.reset();
  }
};
}

…

This is how I read messages

…

final MessageReader reader = new MessageReader(bufferedInputStream, templateStore, 1);
  new Thread() {
  	public void run() {
  		while (true) {

  			try {
  				
  				Message message = reader.readMessage();

  				if (message == null) {
  					break;
  				}


  				handleMessage(message);


  			} catch (FastException exception) {
  				exception.printStackTrace();
  				
  			}
  		}
  	}
  }.start();
…

Error messages we received when reading messages on TCP channel.

…

org.openfast.error.FastException: An error occurred while decoding SecurityDefinition
at org.openfast.template.MessageTemplate.decode(MessageTemplate.java:133)
at org.openfast.codec.FastDecoder.readMessage(FastDecoder.java:82)
at org.openfast.MessageInputStream.readMessage(MessageInputStream.java:65)
Caused by: org.openfast.error.FastException: Error occurred while decoding Scalar [name=NoSecurityAltID, operator=none, type=uInt32, dictionary=SecurityDefinition]
at org.openfast.template.Scalar.decode(Scalar.java:246)
at org.openfast.template.Sequence.decode(Sequence.java:200)
at org.openfast.template.Group.decodeFieldValues(Group.java:293)
at org.openfast.template.MessageTemplate.decode(MessageTemplate.java:126)
… 3 more
Caused by: org.openfast.error.FastException: The value 3486042861337320496 is out of range for type uInt32
at org.openfast.error.ErrorCode.throwException(ErrorCode.java:45)
at org.openfast.error.ErrorHandler$1.error(ErrorHandler.java:26)
at org.openfast.Global.handleError(Global.java:41)
at org.openfast.template.type.IntegerType.validateValue(IntegerType.java:88)
at org.openfast.template.Scalar.validateDecodedValueIsCorrectForType(Scalar.java:260)
at org.openfast.template.Scalar.decode(Scalar.java:240)
… 6 more

…

It would be really great if you can help me to resolve this issue as I have already spent 3 weeks to solve this issue.

Thanks,

How is message rate of your data feed and what is the protocol (tcp/udp) ?

I don’t think data rate will impact on the functionality of OpenFast library.

I have used OpenFast but it does not decode message when data load is high. That’s why I thought to use other FAST framework or API.

If you can let me know any other good Java API for FAST, it will be really helpful.

OpenFAST is better (I can’t say this is the best).

http://www.openfast.org/

Hi All,

What is the BEST (in-terms of performance/data load) Java API/Lib for FAST (commercial/non-commercial)?

Really appreciate if you can share your experience.

Thanks;

fplpo_x · September 12, 2012, 7:39am

[ original email was from janaka Sooriyaarachchi - janakas@millenniumit.com ]
Do not skip one byte always it may vary as per the length of the message. That length field is encoded as a unsigned integer. If message length > 128 then it length field consist with 2 bytes.

You can use unsigned field decoding functions in OPEN FAST and get the length of message length. Then skip those bytes and decode the message.

Field decoding code samples may find via google.

Just want to highlight one thing.

I’m not getting this error every time. Sometime, my application works without any error, but sometime we get this decoding error after processing few messages.

Also, I have skipped first bit (message length) before decoding.

…

return new MessageBlockReader() {
@Override
public void messageRead(InputStream in, Message message) {
}
  	@Override
  	public boolean readBlock(InputStream in) {
  		try {
  			in.skip(offset);
  			return true;
  		} catch (IOException e) {
  			e.printStackTrace();
  			return false;
  		}
  	}

  	@Override
  	public String toString() {
  		return "(default block reader, offset=" + offset + ")";
  	}
  };
…

Do you see any issues on above code?

Thanks;
Thank you very much Janaka.

I checked XML Templates, both are identical.

Could you please let me know how to archive 2nd and 3rd points, as I’m new to OpenFast? It would be really great if you share some resources to understand this.

Thanks a lot again.
As per the error message this is not a issue which related to data rate.
This relates to message decoding and template miss-matching.

So Check the following points,

XML template file is comply with the target Data Source. If you try to decode FAST Message with incorrect Template you will get this Error. Consider the versions.

Check whether FAST message contains a Preamble data. Because some data providers add Non-FAST fix Length Headers to Message. You need to remove those data before decode the message.

Check that Data feed contains a Message length field at the begining of the message. Open FAST dose not work with such Data Feeds. You need to decode Message length manually(you can use field decoding functions in OPEN FAST library for that task) and try to decode only the message segment.
Thanks a lot Janaka for replying.

Actually, we get thousands messages within few seconds on TCP channel.

I have copied my classes below.

This is my message reader class
…

public class MessageReader {
private final MessageInputStream messageIn;
protected MessageBlockReaderFactory messageBlockReaderFactory;



public MessageReader(InputStream inputStream, TemplateStore templateStore, int readOffset) {

  messageIn = new MessageInputStream(inputStream);
  messageIn.setTemplateRegistry(templateStore.getTemplateRegistry());
  messageBlockReaderFactory = new MessageBlockReaderFactory(readOffset);
  MessageBlockReader msgBlockReader = messageBlockReaderFactory.create();
  messageIn.setBlockReader(msgBlockReader);
  messageIn.addMessageHandler(templateStore.getTemplateRegistry().get("Reset"), RESET_HANDLER);
  messageIn.reset();

}

public synchronized Message readMessage() {

  Message m = messageIn.readMessage();

  return m;

}

public void reset() {
  messageIn.reset();
}

public void close() {
  messageIn.close();
}

public int getRemainingBytes() {
  try {
  	return messageIn.getUnderlyingStream().available();
  } catch (IOException e) {
  }
  return -1;
}

private static final MessageHandler RESET_HANDLER = new MessageHandler() {
  @Override
  public void handleMessage(Message readMessage, Context context, Coder coder) {
  	coder.reset();
  }
};
}

…

This is how I read messages

…

final MessageReader reader = new MessageReader(bufferedInputStream, templateStore, 1);
  new Thread() {
  	public void run() {
  		while (true) {

  			try {
  				
  				Message message = reader.readMessage();

  				if (message == null) {
  					break;
  				}


  				handleMessage(message);


  			} catch (FastException exception) {
  				exception.printStackTrace();
  				
  			}
  		}
  	}
  }.start();
…

Error messages we received when reading messages on TCP channel.

…

org.openfast.error.FastException: An error occurred while decoding SecurityDefinition
at org.openfast.template.MessageTemplate.decode(MessageTemplate.java:133)
at org.openfast.codec.FastDecoder.readMessage(FastDecoder.java:82)
at org.openfast.MessageInputStream.readMessage(MessageInputStream.java:65)
Caused by: org.openfast.error.FastException: Error occurred while decoding Scalar [name=NoSecurityAltID, operator=none, type=uInt32, dictionary=SecurityDefinition]
at org.openfast.template.Scalar.decode(Scalar.java:246)
at org.openfast.template.Sequence.decode(Sequence.java:200)
at org.openfast.template.Group.decodeFieldValues(Group.java:293)
at org.openfast.template.MessageTemplate.decode(MessageTemplate.java:126)
… 3 more
Caused by: org.openfast.error.FastException: The value 3486042861337320496 is out of range for type uInt32
at org.openfast.error.ErrorCode.throwException(ErrorCode.java:45)
at org.openfast.error.ErrorHandler$1.error(ErrorHandler.java:26)
at org.openfast.Global.handleError(Global.java:41)
at org.openfast.template.type.IntegerType.validateValue(IntegerType.java:88)
at org.openfast.template.Scalar.validateDecodedValueIsCorrectForType(Scalar.java:260)
at org.openfast.template.Scalar.decode(Scalar.java:240)
… 6 more

…

It would be really great if you can help me to resolve this issue as I have already spent 3 weeks to solve this issue.

Thanks,

How is message rate of your data feed and what is the protocol (tcp/udp) ?

I don’t think data rate will impact on the functionality of OpenFast library.

I have used OpenFast but it does not decode message when data load is high. That’s why I thought to use other FAST framework or API.

If you can let me know any other good Java API for FAST, it will be really helpful.

OpenFAST is better (I can’t say this is the best).

http://www.openfast.org/

Hi All,

What is the BEST (in-terms of performance/data load) Java API/Lib for FAST (commercial/non-commercial)?

Really appreciate if you can share your experience.

Thanks;

kapilawitharana_x · September 12, 2012, 7:53am

Do i need to write my own decoder?

Do not skip one byte always it may vary as per the length of the message. That length field is encoded as a unsigned integer. If message length > 128 then it length field consist with 2 bytes.

You can use unsigned field decoding functions in OPEN FAST and get the length of message length. Then skip those bytes and decode the message.

Field decoding code samples may find via google.
Just want to highlight one thing.

I’m not getting this error every time. Sometime, my application works without any error, but sometime we get this decoding error after processing few messages.

Also, I have skipped first bit (message length) before decoding.

…

return new MessageBlockReader() {
@Override
public void messageRead(InputStream in, Message message) {
}
  	@Override
  	public boolean readBlock(InputStream in) {
  		try {
  			in.skip(offset);
  			return true;
  		} catch (IOException e) {
  			e.printStackTrace();
  			return false;
  		}
  	}

  	@Override
  	public String toString() {
  		return "(default block reader, offset=" + offset + ")";
  	}
  };
…

Do you see any issues on above code?

Thanks;
Thank you very much Janaka.

I checked XML Templates, both are identical.

Could you please let me know how to archive 2nd and 3rd points, as I’m new to OpenFast? It would be really great if you share some resources to understand this.

Thanks a lot again.
As per the error message this is not a issue which related to data rate.
This relates to message decoding and template miss-matching.

So Check the following points,

XML template file is comply with the target Data Source. If you try to decode FAST Message with incorrect Template you will get this Error. Consider the versions.

Check whether FAST message contains a Preamble data. Because some data providers add Non-FAST fix Length Headers to Message. You need to remove those data before decode the message.

Check that Data feed contains a Message length field at the begining of the message. Open FAST dose not work with such Data Feeds. You need to decode Message length manually(you can use field decoding functions in OPEN FAST library for that task) and try to decode only the message segment.
Thanks a lot Janaka for replying.

Actually, we get thousands messages within few seconds on TCP channel.

I have copied my classes below.

This is my message reader class
…

public class MessageReader {
private final MessageInputStream messageIn;
protected MessageBlockReaderFactory messageBlockReaderFactory;



public MessageReader(InputStream inputStream, TemplateStore templateStore, int readOffset) {

  messageIn = new MessageInputStream(inputStream);
  messageIn.setTemplateRegistry(templateStore.getTemplateRegistry());
  messageBlockReaderFactory = new MessageBlockReaderFactory(readOffset);
  MessageBlockReader msgBlockReader = messageBlockReaderFactory.create();
  messageIn.setBlockReader(msgBlockReader);
  messageIn.addMessageHandler(templateStore.getTemplateRegistry().get("Reset"), RESET_HANDLER);
  messageIn.reset();

}

public synchronized Message readMessage() {

  Message m = messageIn.readMessage();

  return m;

}

public void reset() {
  messageIn.reset();
}

public void close() {
  messageIn.close();
}

public int getRemainingBytes() {
  try {
  	return messageIn.getUnderlyingStream().available();
  } catch (IOException e) {
  }
  return -1;
}

private static final MessageHandler RESET_HANDLER = new MessageHandler() {
  @Override
  public void handleMessage(Message readMessage, Context context, Coder coder) {
  	coder.reset();
  }
};
}

…

This is how I read messages

…

final MessageReader reader = new MessageReader(bufferedInputStream, templateStore, 1);
  new Thread() {
  	public void run() {
  		while (true) {

  			try {
  				
  				Message message = reader.readMessage();

  				if (message == null) {
  					break;
  				}


  				handleMessage(message);


  			} catch (FastException exception) {
  				exception.printStackTrace();
  				
  			}
  		}
  	}
  }.start();
…

Error messages we received when reading messages on TCP channel.

…

org.openfast.error.FastException: An error occurred while decoding SecurityDefinition
at org.openfast.template.MessageTemplate.decode(MessageTemplate.java:133)
at org.openfast.codec.FastDecoder.readMessage(FastDecoder.java:82)
at org.openfast.MessageInputStream.readMessage(MessageInputStream.java:65)
Caused by: org.openfast.error.FastException: Error occurred while decoding Scalar [name=NoSecurityAltID, operator=none, type=uInt32, dictionary=SecurityDefinition]
at org.openfast.template.Scalar.decode(Scalar.java:246)
at org.openfast.template.Sequence.decode(Sequence.java:200)
at org.openfast.template.Group.decodeFieldValues(Group.java:293)
at org.openfast.template.MessageTemplate.decode(MessageTemplate.java:126)
… 3 more
Caused by: org.openfast.error.FastException: The value 3486042861337320496 is out of range for type uInt32
at org.openfast.error.ErrorCode.throwException(ErrorCode.java:45)
at org.openfast.error.ErrorHandler$1.error(ErrorHandler.java:26)
at org.openfast.Global.handleError(Global.java:41)
at org.openfast.template.type.IntegerType.validateValue(IntegerType.java:88)
at org.openfast.template.Scalar.validateDecodedValueIsCorrectForType(Scalar.java:260)
at org.openfast.template.Scalar.decode(Scalar.java:240)
… 6 more

…

It would be really great if you can help me to resolve this issue as I have already spent 3 weeks to solve this issue.

Thanks,

How is message rate of your data feed and what is the protocol (tcp/udp) ?

I don’t think data rate will impact on the functionality of OpenFast library.

I have used OpenFast but it does not decode message when data load is high. That’s why I thought to use other FAST framework or API.

If you can let me know any other good Java API for FAST, it will be really helpful.

OpenFAST is better (I can’t say this is the best).

http://www.openfast.org/

Hi All,

What is the BEST (in-terms of performance/data load) Java API/Lib for FAST (commercial/non-commercial)?

Really appreciate if you can share your experience.

Thanks;

fplpo_x · September 12, 2012, 8:02am

[ original email was from janaka Sooriyaarachchi - janakas@millenniumit.com ]
Decoding function available in library. You can use it.

Do i need to write my own decoder?
Do not skip one byte always it may vary as per the length of the message. That length field is encoded as a unsigned integer. If message length > 128 then it length field consist with 2 bytes.

You can use unsigned field decoding functions in OPEN FAST and get the length of message length. Then skip those bytes and decode the message.

Field decoding code samples may find via google.
Just want to highlight one thing.

I’m not getting this error every time. Sometime, my application works without any error, but sometime we get this decoding error after processing few messages.

Also, I have skipped first bit (message length) before decoding.

…

return new MessageBlockReader() {
@Override
public void messageRead(InputStream in, Message message) {
}
  	@Override
  	public boolean readBlock(InputStream in) {
  		try {
  			in.skip(offset);
  			return true;
  		} catch (IOException e) {
  			e.printStackTrace();
  			return false;
  		}
  	}

  	@Override
  	public String toString() {
  		return "(default block reader, offset=" + offset + ")";
  	}
  };
…

Do you see any issues on above code?

Thanks;
Thank you very much Janaka.

I checked XML Templates, both are identical.

Could you please let me know how to archive 2nd and 3rd points, as I’m new to OpenFast? It would be really great if you share some resources to understand this.

Thanks a lot again.
As per the error message this is not a issue which related to data rate.
This relates to message decoding and template miss-matching.

So Check the following points,

XML template file is comply with the target Data Source. If you try to decode FAST Message with incorrect Template you will get this Error. Consider the versions.

Check whether FAST message contains a Preamble data. Because some data providers add Non-FAST fix Length Headers to Message. You need to remove those data before decode the message.

Check that Data feed contains a Message length field at the begining of the message. Open FAST dose not work with such Data Feeds. You need to decode Message length manually(you can use field decoding functions in OPEN FAST library for that task) and try to decode only the message segment.
Thanks a lot Janaka for replying.

Actually, we get thousands messages within few seconds on TCP channel.

I have copied my classes below.

This is my message reader class
…

public class MessageReader {
private final MessageInputStream messageIn;
protected MessageBlockReaderFactory messageBlockReaderFactory;



public MessageReader(InputStream inputStream, TemplateStore templateStore, int readOffset) {

  messageIn = new MessageInputStream(inputStream);
  messageIn.setTemplateRegistry(templateStore.getTemplateRegistry());
  messageBlockReaderFactory = new MessageBlockReaderFactory(readOffset);
  MessageBlockReader msgBlockReader = messageBlockReaderFactory.create();
  messageIn.setBlockReader(msgBlockReader);
  messageIn.addMessageHandler(templateStore.getTemplateRegistry().get("Reset"), RESET_HANDLER);
  messageIn.reset();

}

public synchronized Message readMessage() {

  Message m = messageIn.readMessage();

  return m;

}

public void reset() {
  messageIn.reset();
}

public void close() {
  messageIn.close();
}

public int getRemainingBytes() {
  try {
  	return messageIn.getUnderlyingStream().available();
  } catch (IOException e) {
  }
  return -1;
}

private static final MessageHandler RESET_HANDLER = new MessageHandler() {
  @Override
  public void handleMessage(Message readMessage, Context context, Coder coder) {
  	coder.reset();
  }
};
}

…

This is how I read messages

…

final MessageReader reader = new MessageReader(bufferedInputStream, templateStore, 1);
  new Thread() {
  	public void run() {
  		while (true) {

  			try {
  				
  				Message message = reader.readMessage();

  				if (message == null) {
  					break;
  				}


  				handleMessage(message);


  			} catch (FastException exception) {
  				exception.printStackTrace();
  				
  			}
  		}
  	}
  }.start();
…

Error messages we received when reading messages on TCP channel.

…

org.openfast.error.FastException: An error occurred while decoding SecurityDefinition
at org.openfast.template.MessageTemplate.decode(MessageTemplate.java:133)
at org.openfast.codec.FastDecoder.readMessage(FastDecoder.java:82)
at org.openfast.MessageInputStream.readMessage(MessageInputStream.java:65)
Caused by: org.openfast.error.FastException: Error occurred while decoding Scalar [name=NoSecurityAltID, operator=none, type=uInt32, dictionary=SecurityDefinition]
at org.openfast.template.Scalar.decode(Scalar.java:246)
at org.openfast.template.Sequence.decode(Sequence.java:200)
at org.openfast.template.Group.decodeFieldValues(Group.java:293)
at org.openfast.template.MessageTemplate.decode(MessageTemplate.java:126)
… 3 more
Caused by: org.openfast.error.FastException: The value 3486042861337320496 is out of range for type uInt32
at org.openfast.error.ErrorCode.throwException(ErrorCode.java:45)
at org.openfast.error.ErrorHandler$1.error(ErrorHandler.java:26)
at org.openfast.Global.handleError(Global.java:41)
at org.openfast.template.type.IntegerType.validateValue(IntegerType.java:88)
at org.openfast.template.Scalar.validateDecodedValueIsCorrectForType(Scalar.java:260)
at org.openfast.template.Scalar.decode(Scalar.java:240)
… 6 more

…

It would be really great if you can help me to resolve this issue as I have already spent 3 weeks to solve this issue.

Thanks,

How is message rate of your data feed and what is the protocol (tcp/udp) ?

I don’t think data rate will impact on the functionality of OpenFast library.

I have used OpenFast but it does not decode message when data load is high. That’s why I thought to use other FAST framework or API.

If you can let me know any other good Java API for FAST, it will be really helpful.

OpenFAST is better (I can’t say this is the best).

http://www.openfast.org/

Hi All,

What is the BEST (in-terms of performance/data load) Java API/Lib for FAST (commercial/non-commercial)?

Really appreciate if you can share your experience.

Thanks;

kapilawitharana_x · September 18, 2012, 4:10am

Finally I was able to decode messages successfully.

Thank you very much Janaka for your support and guidance extended to resolve this issue.

Thanks again!

Decoding function available in library. You can use it.
Do i need to write my own decoder?
Do not skip one byte always it may vary as per the length of the message. That length field is encoded as a unsigned integer. If message length > 128 then it length field consist with 2 bytes.

You can use unsigned field decoding functions in OPEN FAST and get the length of message length. Then skip those bytes and decode the message.

Field decoding code samples may find via google.
Just want to highlight one thing.

I’m not getting this error every time. Sometime, my application works without any error, but sometime we get this decoding error after processing few messages.

Also, I have skipped first bit (message length) before decoding.

…

return new MessageBlockReader() {
@Override
public void messageRead(InputStream in, Message message) {
}
  	@Override
  	public boolean readBlock(InputStream in) {
  		try {
  			in.skip(offset);
  			return true;
  		} catch (IOException e) {
  			e.printStackTrace();
  			return false;
  		}
  	}

  	@Override
  	public String toString() {
  		return "(default block reader, offset=" + offset + ")";
  	}
  };
…

Do you see any issues on above code?

Thanks;
Thank you very much Janaka.

I checked XML Templates, both are identical.

Could you please let me know how to archive 2nd and 3rd points, as I’m new to OpenFast? It would be really great if you share some resources to understand this.

Thanks a lot again.
As per the error message this is not a issue which related to data rate.
This relates to message decoding and template miss-matching.

So Check the following points,

XML template file is comply with the target Data Source. If you try to decode FAST Message with incorrect Template you will get this Error. Consider the versions.

Check whether FAST message contains a Preamble data. Because some data providers add Non-FAST fix Length Headers to Message. You need to remove those data before decode the message.

Check that Data feed contains a Message length field at the begining of the message. Open FAST dose not work with such Data Feeds. You need to decode Message length manually(you can use field decoding functions in OPEN FAST library for that task) and try to decode only the message segment.
Thanks a lot Janaka for replying.

Actually, we get thousands messages within few seconds on TCP channel.

I have copied my classes below.

This is my message reader class
…

public class MessageReader {
private final MessageInputStream messageIn;
protected MessageBlockReaderFactory messageBlockReaderFactory;



public MessageReader(InputStream inputStream, TemplateStore templateStore, int readOffset) {

  messageIn = new MessageInputStream(inputStream);
  messageIn.setTemplateRegistry(templateStore.getTemplateRegistry());
  messageBlockReaderFactory = new MessageBlockReaderFactory(readOffset);
  MessageBlockReader msgBlockReader = messageBlockReaderFactory.create();
  messageIn.setBlockReader(msgBlockReader);
  messageIn.addMessageHandler(templateStore.getTemplateRegistry().get("Reset"), RESET_HANDLER);
  messageIn.reset();

}

public synchronized Message readMessage() {

  Message m = messageIn.readMessage();

  return m;

}

public void reset() {
  messageIn.reset();
}

public void close() {
  messageIn.close();
}

public int getRemainingBytes() {
  try {
  	return messageIn.getUnderlyingStream().available();
  } catch (IOException e) {
  }
  return -1;
}

private static final MessageHandler RESET_HANDLER = new MessageHandler() {
  @Override
  public void handleMessage(Message readMessage, Context context, Coder coder) {
  	coder.reset();
  }
};
}

…

This is how I read messages

…

final MessageReader reader = new MessageReader(bufferedInputStream, templateStore, 1);
  new Thread() {
  	public void run() {
  		while (true) {

  			try {
  				
  				Message message = reader.readMessage();

  				if (message == null) {
  					break;
  				}


  				handleMessage(message);


  			} catch (FastException exception) {
  				exception.printStackTrace();
  				
  			}
  		}
  	}
  }.start();
…

Error messages we received when reading messages on TCP channel.

…

org.openfast.error.FastException: An error occurred while decoding SecurityDefinition
at org.openfast.template.MessageTemplate.decode(MessageTemplate.java:133)
at org.openfast.codec.FastDecoder.readMessage(FastDecoder.java:82)
at org.openfast.MessageInputStream.readMessage(MessageInputStream.java:65)
Caused by: org.openfast.error.FastException: Error occurred while decoding Scalar [name=NoSecurityAltID, operator=none, type=uInt32, dictionary=SecurityDefinition]
at org.openfast.template.Scalar.decode(Scalar.java:246)
at org.openfast.template.Sequence.decode(Sequence.java:200)
at org.openfast.template.Group.decodeFieldValues(Group.java:293)
at org.openfast.template.MessageTemplate.decode(MessageTemplate.java:126)
… 3 more
Caused by: org.openfast.error.FastException: The value 3486042861337320496 is out of range for type uInt32
at org.openfast.error.ErrorCode.throwException(ErrorCode.java:45)
at org.openfast.error.ErrorHandler$1.error(ErrorHandler.java:26)
at org.openfast.Global.handleError(Global.java:41)
at org.openfast.template.type.IntegerType.validateValue(IntegerType.java:88)
at org.openfast.template.Scalar.validateDecodedValueIsCorrectForType(Scalar.java:260)
at org.openfast.template.Scalar.decode(Scalar.java:240)
… 6 more

…

It would be really great if you can help me to resolve this issue as I have already spent 3 weeks to solve this issue.

Thanks,

How is message rate of your data feed and what is the protocol (tcp/udp) ?

I don’t think data rate will impact on the functionality of OpenFast library.

I have used OpenFast but it does not decode message when data load is high. That’s why I thought to use other FAST framework or API.

If you can let me know any other good Java API for FAST, it will be really helpful.

OpenFAST is better (I can’t say this is the best).

http://www.openfast.org/

Hi All,

What is the BEST (in-terms of performance/data load) Java API/Lib for FAST (commercial/non-commercial)?

Really appreciate if you can share your experience.

Thanks;

Topic		Replies	Views
FAST decoding error can anyone help me.? imported , fast-protocol , xid_169567	6	192	January 9, 2012
CME FIX/FAST decoder imported , fast-protocol , xid_169483	16	248	September 16, 2011
FAST protocole with EUREX EBS imported , fast-protocol , xid_169645	26	66	July 23, 2008
FAST too complex? imported , fast-protocol , xid_169710	35	74	January 17, 2009
FAST compression and CPU utilization imported , fast-protocol , xid_169707	13	58	January 3, 2009

Best Java API for FAST

Related topics