Imported from previous forum
[ original email was from Andrew Wilson - ]
4.1 Version Changes ID’s
Our implimentation will require less work to change the id data types to char than to add the extra code to handle the new ids to the many apps that use the engine.
Is this the case with others implimentations?
If this is so maybe its easier all round to just change the data type. For instance with IOIs we have a program that displays these and keys not unsprisingly on IOIId. It assumes this is unique but that is all. To change the engine so that we potentailly put blanks in IOIId and have to put the IOIId into IOICharId means we have to change the next level of our architecture. Each step up we propogate out the changes forces more work for the people who build system based on FIX.
Plus this will be confusing and we will inevitably get people wrongly using the Id that remain present but not supported. These systems will then break when the unsupported parts of implimentation are not longer generated. This sort of soft compatablility is very nasty.
NB Some vendors already use char for all ids so we’ve already made changes to accomodate this in the past. These vendors would then have no changes to make.
In summary, we are concerned about repeating information in two different, and one redundant, formats. This will inevitablely lead to confusion.
Andrew Wilson
Andrew.A.Wilson@flemings.com
[ original email was from Mark Hinman - ]
The problem with with changing the data types instead of creating a new data fields is maintaining multiple simultaneous FIX connections across different versions of the spec. Global search and replace sounds easy but it kill backwards compatiblity with other FIX connections. We currently are supporting concurrent connections to FIX 2.7, 3.0 and 4.0. and we intend to continue full backwards compatiblity.
Mark Hinman
Ease Technologies, Inc.
mhinman@easetech.com
> 4.1 Version Changes ID’s
>
> Our implimentation will require less work to change the id data types to char than to add the extra code to handle the new ids to the many apps that use the engine.
>
> Is this the case with others implimentations?
>
> If this is so maybe its easier all round to just change the data type. For instance with IOIs we have a program that displays these and keys not unsprisingly on IOIId. It assumes this is unique but that is all. To change the engine so that we potentailly put blanks in IOIId and have to put the IOIId into IOICharId means we have to change the next level of our architecture. Each step up we propogate out the changes forces more work for the people who build system based on FIX.
>
> Plus this will be confusing and we will inevitably get people wrongly using the Id that remain present but not supported. These systems will then break when the unsupported parts of implimentation are not longer generated. This sort of soft compatablility is very nasty.
>
> NB Some vendors already use char for all ids so we’ve already made changes to accomodate this in the past. These vendors would then have no changes to make.
>
> In summary, we are concerned about repeating information in two different, and one redundant, formats. This will inevitablely lead to confusion.
>
> Andrew Wilson
> Andrew.A.Wilson@flemings.com
>
[ original email was from Andrew Schorr - ]
Mark Hinman wrote:
> The problem with with changing the data types instead of creating a new data fields is maintaining multiple simultaneous FIX connections across different versions of the spec. Global search and replace sounds easy but it kill backwards compatiblity with other FIX connections. We currently are supporting concurrent connections to FIX 2.7, 3.0 and 4.0. and we intend to continue full backwards compatiblity.
Andrew Wilson wrote:
> >
> > Our implimentation will require less work to change the id data types to char than to add the extra code to handle the new ids to the many apps that use the engine.
> >
I agree with Andrew (and not Mark). We currently support
FIX 3.0 and 4.0 concurrently, and 4.1 will be much easier
to support concurrently if we just generalize the types of
the ID fields from int to char. I could understand the need
for new fields if the new type conflicted with the old one.
But, in this case, the new type is more general; it is a
superset of the old type. Thus, if everyone generalizes
his/her code to treat these fields as type char, then the
new code will work with all versions. Why isn’t that
the simplest approach to take? If we create new fields,
then we will still have to write the code to support
handling these fields as type char, but we will also have
to add a layer of indirection that will allow us to map
the field to the appropriate implementation field based
on the version number. Just generalizing the type will
enable us to avoid that layer of indirection.
So please just generalize the data type to char. Don’t
bother to add all those new fields.
Andrew Schorr
Daiwa Securities America Inc.
schorr@ead.dsa.com
We’ve had some good discussion on this topic. Note that we cannot change the data type for existing, released versions of the spec. It is imperative from a compatibility standpoint that one does not send alphabetic characters for ExecID in 2.7, 3.0, or 4.0. We must also recognize that people will begin implementing such changes at different times.
> Mark Hinman wrote:
>
> > The problem with with changing the data types instead of creating a new data fields is maintaining multiple simultaneous FIX connections across different versions of the spec. Global search and replace sounds easy but it kill backwards compatiblity with other FIX connections. We currently are supporting concurrent connections to FIX 2.7, 3.0 and 4.0. and we intend to continue full backwards compatiblity.
>
> Andrew Wilson wrote:
> > >
> > > Our implimentation will require less work to change the id data types to char than to add the extra code to handle the new ids to the many apps that use the engine.
> > >
>
> I agree with Andrew (and not Mark). We currently support
> FIX 3.0 and 4.0 concurrently, and 4.1 will be much easier
> to support concurrently if we just generalize the types of
> the ID fields from int to char. I could understand the need
> for new fields if the new type conflicted with the old one.
> But, in this case, the new type is more general; it is a
> superset of the old type. Thus, if everyone generalizes
> his/her code to treat these fields as type char, then the
> new code will work with all versions. Why isn’t that
> the simplest approach to take? If we create new fields,
> then we will still have to write the code to support
> handling these fields as type char, but we will also have
> to add a layer of indirection that will allow us to map
> the field to the appropriate implementation field based
> on the version number. Just generalizing the type will
> enable us to avoid that layer of indirection.
>
> So please just generalize the data type to char. Don’t
> bother to add all those new fields.
>
> Andrew Schorr
> Daiwa Securities America Inc.
> schorr@ead.dsa.com
>
>
[ original email was from Val Melamed - ]
I agree with Mark and Scott on this topic. It is not that simple to just change the type and to generalize for the previous versions. Even if it is guarateed that no site will ever send alphanumeric IOIRefId to FIX.2.7, 3.0, 4.0 sites, here is what "just changing the type" would mean for us:
- change of data base column types;
- change of data base stored procedures;
- loosing some perfomance in the searches (char[n] vs. int) both in the DB and in the code (ok, I can leave with this…
); - many changes of the back end and front end code;
- upgrading lots of cusmers.
Currently only we have installed over 30 servers talking FIX 2.7, 3.0, 4.0 and respectively 30*N client programs.
I really don’t know what exactly requires the change of the types of all these fields, but I am strongly opposed to it, unless someone out there proves that there is a very good compeling reason. In such a case, I would agree with Mark and Scott – let’s have new fields.
Please, guys, think about the fact that the most of the FIX sites are still talking 3.0. Putnam is still using 2.7…
Val
Ease Technolgies
[ original email was from Bob Lamoureux - ]
Wait a minute… why would you have to change your database structure ?? If you dont support 4.1 then you dont have to change a thing. Once you support 4.1 then your database structure must change with it. The IDs will be in a different column at the minimum. You have the choice as to which customers you upgrade and when. I maintain my position that changing the field types is just as much work as adding new fields which have to be mapped to new database columns (anyway). Please trust me on this… From version 2.0 to 2.7 we changed datatypes all the time and both methods have their associated work. The advantage then of not adding new fields is to keep the namespace small (i.e less tags)
Bob Lamoureux
> I agree with Mark and Scott on this topic. It is not that simple to just change the type and to generalize for the previous versions. Even if it is guarateed that no site will ever send alphanumeric IOIRefId to FIX.2.7, 3.0, 4.0 sites, here is what “just changing the type” would mean for us:
> 1) change of data base column types;
> 2) change of data base stored procedures;
> 3) loosing some perfomance in the searches (char[n] vs. int) both in the DB and in the code (ok, I can leave with this… 
);
> 4) many changes of the back end and front end code;
> 5) upgrading lots of cusmers.
> Currently only we have installed over 30 servers talking FIX 2.7, 3.0, 4.0 and respectively 30*N client programs.
>
> I really don’t know what exactly requires the change of the types of all these fields, but I am strongly opposed to it, unless someone out there proves that there is a very good compeling reason. In such a case, I would agree with Mark and Scott – let’s have new fields.
>
> Please, guys, think about the fact that the most of the FIX sites are still talking 3.0. Putnam is still using 2.7…
>
> Val
> Ease Technolgies
>
[ original email was from Val Melamed - ]
> Wait a minute… why would you have to change your database structure ??
Because the type of the ID which I am supposed to refer to later in the session, changed. I need to keep this ID somewhere. Well, it happens to be an int field in one of my tables now and it will need to move to another table with another type plus the required converting for the existing data…
>If you dont support 4.1 then you dont have to change a thing. Once you support 4.1 then your database structure must change with it.
It must change if the spec changes. And I think it doesn’t need to. When a spec changes the people who are changing it should think about the installed base, about backwards compatibility, etc. Fundamental changes as basic as changing type of data should be very carefully considered and if there is no compeling reason to do it they should not chage it.
>The IDs will be in a different column at the minimum. You have the choice as to which customers you upgrade and when. I maintain my position that changing the field types is just as much work as adding new fields which have to be mapped to new database columns (anyway). Please trust me on this…
Actualy my position is that neither of both is neccessary. Change like this means that the responsibility for mapping IDs is moved from the owner of the ID to the recipient. This doesn’t sound right to me especially when so many things would get worse:
1)backwards compatibility
2)performance
3)software/hardware portability
>From version 2.0 to 2.7 we changed datatypes all the time and both methods have their associated work.
I can understand changing the spec in its early phase with just a few players but now when you have so many companies exchanging FIX messages one should be much more carefull.
>The advantage then of not adding new fields is to keep the namespace small (i.e less tags)
>
> Bob Lamoureux
>
I do agree with yor remark in the other discussion though, that the protocol should be tailord with usability and functionality in mind not with MINE or YOURS implementation. This is why I am suggesting that ALL the data which is site specific (e.g. ExcecID, OrderID, etc.) to be hidden behind int magic cookies. Doesn’t this sound right? When you open a file the operating system gives you a magic cookie, not the object which has all the information about the file. I would like to reitarate that alphanumeric information can be nonportable or language specific and this can create serious problems.
Val
Ease Technologies
[ original email was from Bob Lamoureux - ]
OK I’m either on drugs, on Mars or both !! What backwards compatability ??? The way I understand it, these field changes are only in effect for FIX.4.1 and beyond. Therefore, there should be zero changes necessary if you do not plan to support 4.1 for the foreseeable future. In other words, the current INT fields in 2.7, 3.0 and 4.0 WILL REMAIN INT fields. So if your receive something other than INT in those versions, you may freely and briskly REJECT. So the only code change that I see you MIGHT have to make, is to batten down you syntax checker, but I am sure everyone who has been following this thread is doing field validation anyway. Maybe that’s why this discussion has been so heated… changing the data types of 4.0, 3.0 and 2.7 is absolutely a NO-NO but that is not what I interpreted this proposed change to be… Am I wrong ??
As far as 2.0 to 2.7 being EASIER to manage because the installed base was small is a bit off base; it was irrelevant as to the number of users of FIX engines out there. If it didn’t make sense for 2 players then it didn’t make sense for 200. We didn’t make changes and say " no problem, we’re only scr*wing ourselves so it’s OK !!!" …