Imported from previous forum
Hi, regarding Integers and Overlong Property.
The spec states that an integer is overlong if the entity value still represents the same integer after removing seven or more of the most significant bits.
Then it continues saying that for signed integers, there are some situations where the seven most significant bits of the entity value must all be zeroes, and adds an example:
For example, if the value to encode is 64, which has the binary representation 01000000, the stop bit encoding must be 0x00 0xC0. If we did not have the extra seven leading zero bits, the most significant bit, which is also the sign bit, would be one, and thus the encoding would incorrectly represent -64.
The spec clearly states that an “Integer” is overlong if 7 or more bits are zero, it does not say an “Unsigned Integer”.
So what is the proper way of checking for overlong? Do I assume its eight (8) or more bits?
Unless I am missing something the spec should be corrected as well.
[ original email was from Darshan Khedekar - darshan.khedekar.ext@deutsche-boerse.com ]
Hi
Consider the following example on page 33 of the FAST spec.
Int32 Example – Mandatory Positive Number with sign-bit extension
Input Value Native Hex/Binary FAST Hex/Binary
8193 0x20 0x01 0x00 0x40 0x81
00100000 00000001 0(0)00000 01000000 10000001
Sign bits indicated by ()
Sign bit extension necessary to specify sign (italics)
Remember Sign bit for the int32/64 is the 7th bit (adjacent to the Stop bit) of the first byte in the stream.
Now if you would have used only 2 bytes the encoding would have been
01000000 10000001
Here the problem is that we have used up the sign bit for the numeric value of the field and hence the dilemma.
In order to clearly convey the sign of the field I need to send the above 3 bytes where the additional bytes sign bit conveys the sign.
Hope I am clear.
Regards
Darshan
Hi, regarding Integers and Overlong Property.
The spec states that an integer is overlong if the entity value still
represents the same integer after removing seven or more of the most
significant bits.Then it continues saying that for signed integers, there are some
situations where the seven most significant bits of the entity value
must all be zeroes, and adds an example:For example, if the value to encode is 64, which has the binary
representation 01000000, the stop bit encoding must be 0x00 0xC0. If we
did not have the extra seven leading zero bits, the most significant
bit, which is also the sign bit, would be one, and thus the encoding
would incorrectly represent -64.The spec clearly states that an “Integer” is overlong if 7 or more bits
are zero, it does not say an “Unsigned Integer”.So what is the proper way of checking for overlong? Do I assume its
eight (8) or more bits?Unless I am missing something the spec should be corrected as well.
So you are saying that in this example, I actually only have 6 0’s? That I should not take the sign bit into consideration?
Hi
Consider the following example on page 33 of the FAST spec.
Int32 Example – Mandatory Positive Number with sign-bit extension
Input Value Native Hex/Binary FAST Hex/Binary 8193 0x20 0x01 0x00 0x40
0x81 00100000 00000001 0(0)00000 01000000 10000001Sign bits indicated by () Sign bit extension necessary to specify
sign (italics)Remember Sign bit for the int32/64 is the 7th bit (adjacent to the Stop
bit) of the first byte in the stream. Now if you would have used only 2
bytes the encoding would have been 01000000 10000001Here the problem is that we have used up the sign bit for the numeric
value of the field and hence the dilemma. In order to clearly convey the
sign of the field I need to send the above 3 bytes where the additional
bytes sign bit conveys the sign.Hope I am clear.
Regards Darshan
Hi, regarding Integers and Overlong Property.
The spec states that an integer is overlong if the entity value still
represents the same integer after removing seven or more of the most
significant bits.Then it continues saying that for signed integers, there are some
situations where the seven most significant bits of the entity value
must all be zeroes, and adds an example:For example, if the value to encode is 64, which has the binary
representation 01000000, the stop bit encoding must be 0x00 0xC0. If
we did not have the extra seven leading zero bits, the most
significant bit, which is also the sign bit, would be one, and thus
the encoding would incorrectly represent -64.The spec clearly states that an “Integer” is overlong if 7 or more
bits are zero, it does not say an “Unsigned Integer”.So what is the proper way of checking for overlong? Do I assume its
eight (8) or more bits?Unless I am missing something the spec should be corrected as well.
[ original email was from Mikael Brännström - m.brannstrom@ngm.se ]
Hi!
Yes, the spec saysthat an integer is overlong if the entity value still
represents the same integer after removing seven or more of the most
significant bits. Note that nothing is said that these bits must be zeroes.
For negative signed integer values this means that the seven bits are all ones. For example
the value -1 could be encoded as 0x7F 0x8F which is overlong since 0x8F also represents the same value.
The value 64 example you mentioned addresses that the sign bit should not be accidently changed/removed. When encoding signed integers you can think of reserving one bit for the sign. This means that the most significant byte can only carry 6 bits (plus 1 stop bit and 1 sign bit) while the other bytes can carry 7 bits. The value 64 which is (0)100 0000 need 7 value bits (the sign bit in parentesis) which can be translated into 2 bytes for encoding, i.e. 0x00 0xC0.
I hope this helps
Regards
Mikael
Hi, regarding Integers and Overlong Property.
The spec states that an integer is overlong if the entity value still
represents the same integer after removing seven or more of the most
significant bits.Then it continues saying that for signed integers, there are some
situations where the seven most significant bits of the entity value
must all be zeroes, and adds an example:For example, if the value to encode is 64, which has the binary
representation 01000000, the stop bit encoding must be 0x00 0xC0. If we
did not have the extra seven leading zero bits, the most significant
bit, which is also the sign bit, would be one, and thus the encoding
would incorrectly represent -64.The spec clearly states that an “Integer” is overlong if 7 or more bits
are zero, it does not say an “Unsigned Integer”.So what is the proper way of checking for overlong? Do I assume its
eight (8) or more bits?Unless I am missing something the spec should be corrected as well.
Ok, so signed integers always have 6 bits of data only on the first byte… Therefore the check has to be for a sequence of seven 1’s if the number is negative (6 bits from the first byte + 1 bit from second byte) and a sequence of 7+ zeroes following the same concept if the number is not a negative, but the type is signed.
And if it is unsigned, just need to check if the first byte == 0 right?
Hi!
Yes, the spec saysthat an integer is overlong if the entity value
still represents the same integer after removing seven or more of the
most significant bits. Note that nothing is said that these bits must
be zeroes.For negative signed integer values this means that the seven bits are
all ones. For example the value -1 could be encoded as 0x7F 0x8F which
is overlong since 0x8F also represents the same value.The value 64 example you mentioned addresses that the sign bit should
not be accidently changed/removed. When encoding signed integers you can
think of reserving one bit for the sign. This means that the most
significant byte can only carry 6 bits (plus 1 stop bit and 1 sign bit)
while the other bytes can carry 7 bits. The value 64 which is (0)100
0000 need 7 value bits (the sign bit in parentesis) which can be
translated into 2 bytes for encoding, i.e. 0x00 0xC0.I hope this helps
Regards Mikael
Hi, regarding Integers and Overlong Property.
The spec states that an integer is overlong if the entity value still
represents the same integer after removing seven or more of the most
significant bits.Then it continues saying that for signed integers, there are some
situations where the seven most significant bits of the entity value
must all be zeroes, and adds an example:For example, if the value to encode is 64, which has the binary
representation 01000000, the stop bit encoding must be 0x00 0xC0. If
we did not have the extra seven leading zero bits, the most
significant bit, which is also the sign bit, would be one, and thus
the encoding would incorrectly represent -64.The spec clearly states that an “Integer” is overlong if 7 or more
bits are zero, it does not say an “Unsigned Integer”.So what is the proper way of checking for overlong? Do I assume its
eight (8) or more bits?Unless I am missing something the spec should be corrected as well.
[ original email was from Mikael Brännström - m.brannstrom@ngm.se ]
I think the following patterns should cover all possible cases of overlong integers. The stop bit is in parentesis.
Overlong negative signed integer:
(0)111 1111, (*)1** ****, …
Overlong positive signed integer:
(0)000 0000, (*)0** ****, …
Overlong unsigned integer:
(0)000 0000, ()** ****, …
Note that the overlong property is only a reportable error, which means that you do not have to detect it when decoding.
Those are pretty much the patterns I am using now, and I agree. I put everything within a #ifdef as a special case if we want to debug something from the market… as a client, we don’t care if it is overlong or not being that the value will still be correct.
The only issue we care about if the value is overlong or not is for PMAP overlong, because that might break the decoding.
Thanks!
Fernando
I think the following patterns should cover all possible cases of
overlong integers. The stop bit is in parentesis.Overlong negative signed integer: ()111 1111, (*)1** ****, …
Overlong positive signed integer: ()000 0000, (*)0** ****, …
Overlong unsigned integer: ()000 0000, ()** ****, …
Note that the overlong property is only a reportable error, which
means that you do not have to detect it when decoding.