Floating-point values can look a lot like Real Arithmetic, but there are a lot of tricky details at the fringes. The accuracy of floating-point results is one of them, and so is its verification.
The mathematically correct result of a floating-point calculation can often not be represented in a computer’s floating-point representation. Dividing five by three, for example, does not fit in a finite binary representation. You can expect a rounded value, but can you count on always getting a correctly rounded value that corresponds to the mathematically precise result?
Naturally, you could look at the C language definition for guidance. However, you might well be disappointed, because the language specification states that:
The accuracy of the floating-point operations (+, -, *, /) and of the library functions […] is implementation-defined. […]
The implementation may state that the accuracy is unknown.
This does not make it easy to work with floating-point computations. Not only can a C programmer not rely on anything, it also makes it almost impossible (for us in the testing business) to verify any floating-point result.
There are historically good reasons for this lack of precision. For example, it allows for many different floating-point implementations to support C. Remember Cray computers of old? They were famously fast but notoriously less precise than one would wish for.
However, all is not lost. Since C99, implementations can declare to be compliant with the IEC 60559 standard (aka IEEE 754), which says that the result of +, -, *, / and the square root function must be correctly rounded from its mathematically precise result. That is quite a leap forward, and more importantly, many of today’s floating-point implementations come very close to it, from supercomputers down to small embedded cores.
Yet even IEC 60559 puts no bounds on the accuracy of the math library beyond square root. That is why, in our next update of SuperTest, we have taken matters into our own hands. We believe math library accuracy is important to developers and they therefore need some evidence about how well their library performs.
SuperTest has always checked for accuracy, but the limits were somewhat loose and not so well defined. That is now changing. In the next update, by default, the maximum acceptable error is set to four ULP (Units in Last Place). ULP is a relative measure and refers to the smallest interval between two floating-point numbers. With perfect accuracy, the maximum error possible is half a ULP, because you always have to round the mathematical result to the nearest floating-point value.
To get an accuracy of four ULP for all math library functions across the board is amazing, and it is actually achieved by many library implementations today. Of course, the value ‘four’ is a parameter in SuperTest that can be set to another value by the user. If your embedded implementation favors speed over accuracy, like a Cray, you now have that choice.
For more observations regarding floating-point, see our white paper Learning from Math Library Testing for C.
Dr. Marcel Beemster, CTO