Some you have probably seen a post from last Tuesday entitled Floating Point Fun. If you have not read this I would recommend going back and reading it before continuing. In this post I discuss some of the interesting things that can happen when dealing with floating point math in C#, it is important to note that these items did not happen in version 1.x of the framework.
The root of these problems is that when in a register the floating point is treated with a different precision than when it is being held in memory. As such you can run into cases where you are comparing a Float32 or a Float64 against an 80 bit register based float. These equality comparisons (or conversions to other types such as an integer) can obviously fail due to the difference in precision.
After tracing through the generated assembly, I found a great reference on the subject at David Notario's Blog. David correctly points out that this is not a CLR/JIT issue, in fact changes like this were eluded to in the CLR spec (there is a quote from the ECMA spec on his blog) or here http://dotnet.di.unipi.it/EcmaSpec/PartitionI/cont11.html#_Toc527182172
There was some documentation on this breaking change in 2.0. Here is the listing from the breaking changes documentation
In the CLR model, we assert that arguments, locals, and return values (things which you can't tell their size) can be expanded at will. When you say float, it means anything greater than or equal to a float. So we can sometimes give you what you asked for 'or better'. When we do this, we can spill 'extra' data, almost like a 'you only asked for 15 precision points, but congratulations! We gave you 18!'. If someone expected the floating point precision to always remain the exactly the same, they could be affected. In order to faciliate performance improvements and better scenarios, the CLR may rewrite (as in this case) parts of the register. For example, things that used to truncate because of spilling, no longer do. We make these kinds of changes all the time. We believe this is an appropriate change, and it is even called out specifically in the CLI specification, as something which can, and will occur with different iterations:
What makes these changes particularly nasty is that you are forced to second guess how the JIT works in order to provide consistent results. In my previous post I used the example of
|
float f = 97.09f;
f = (f * 100f);
int tmp = (int)f;
Console.WriteLine(tmp); |
This code will work in either debug or release mode when a debugger is attached, having the debugger attached will disable the JIT optimizations that cause the problem. It does as I describe in the previous post fail when run without the debugger. If we wanted it to work all of the time we would need to write it in the form.
|
float f = 97.09f;
f = (float)(f * 100f);
int tmp = (int)f;
Console.WriteLine(tmp); |
The explicit cast to a float forces it to be narrowed back to a float32, without the narrowing it will actually be in a register as an 80 bit float. As such we end with a predictable behavior of always producing the correct result of 9709.
The problem I have with this behavior is that it is a leaky abstraction. In order to have our code work properly (and to be efficient) we need to know exactly how the compiler and the JIT intends to optimize our code. This introduces a logical problem though as by its very definition we do not know how the JIT will optimize our code. The JIT very well could place this into a register at some times and not at others or the JIT run on a different platform could offer a different behavior than the JIT we tested with.
This becomes especially nasty when dealing constants, consider the following code.
|
float f = 97.09f;
f = (f * 100f);
bool test = f == 97.09f * 100f;
Console.WriteLine(test); |
What is the value of test? The abstraction leaks for both the compiler and the JIT. To start with, are the floats actually being calculated at runtime or is the compiler smart enough to realize that they are constants? In this particular case the C# compiler generates instructions for the first floating point operation but recognizes that the second is a constant value and as such pre-computes the value. These types of scenarios are exactly the type of thing that compilers look for when optimizing.
If the compiler did not recognize the constant expression this might work as both of the calculations would have been done with their result being saved in a register, at that point we would actually have to look at how the JIT handled this case. Both of these items may change based upon environment.
The CLR has basically left the choice to the language as to how it wants to handle these cases. Visual C++ has handled this by providing compiler switches. The link is also interesting as it deals with how the switches apply as well to optimizations that occur within the compiler that can cause further issues. C# does not have many such optimizations at this point but it is only a matter of time before they get introduced.
I would therefore propose that C# should be given switches as well (similar to those available for C++) which could allow for the automatic narrowing of floating point values.
It is often brought up that C# does it the way it does it for performance reasons; it is obviously faster to leave values in registers when possible as opposed to narrowing them. The only way consistent way of doing it is through the use of the narrowing. From all of the studies I have seen, C# is primarily used for business applications where consistency (and reduction of programmer thought) is the primary goal and quite often run-time speed is sacrificed in order to better meet these goals (think abstractions). If an argument can be made for C++ to have an option of a precise switch, I would imagine a better argument can in fact be made for C#.
Based upon this I would also propose that the default behavior of the compiler should be to support consistent operations (/fp:precise in C++).
This switch would not eliminate people from writing code that was dependent upon how the compiler/JIT treated things; it would however force the programmer to make a conscious decision by setting the switch that they were assuming the risks associated with the performance gains. VC++ by default runs with /fp:precise so I would not think it a large jump to make the C# compiler consistent.
As a note for the people I am sure will say, “don't do this .. use a precision range or round instead“. These are simply examples ... I am fine with using these solutions (in fact I normally use range checks). The problem is that code like this crops up regularly and it creates a very subtle problem (that did not exist in 1.x). That and there are times when you actually want (validly) to do an equivalence test on two floating point numbers that should have a consistent value (i.e. results of the same calculation). If these operations are to be disallowed, that is fine as well .. but lets completely disallow them and have the compiler generate an warning/error in the circumstance.
This issue is known by very few, if you agree with the concepts here I ask you to either leave a comment below or to link to the post on your blog. Hopefully getting this knowledge more into the mainstream will both reduce the number of bugs caused by this subtlety and bring more focus on it by those with the power to change it.