Compare Errors in Microsoft Maths

[johnggold]

I have a library function RoundDouble() which takes a double value and rounds to 2DP. This is mostly use for display and other applications where trimming is required, but I have used it to compare values with strange results.

I use Visual Studio 10 Professional to build solutions.

The test :

if( RoundDouble(a) > RoundDouble(b) )

frequently returns TRUE, even when both values are identical BOTH rounded AND before rounding. I have of course used debugging to test the values being compared.

If I use :

if( ( RoundDouble(a) - RoundDouble(b) ) > (double) 0.00001 )


This works without fail on the same data. Based on the number of DPs displayed when debugging, this test should not make a difference.

Is there something I can do that will remove this anomaly?

[LB]

No, there is nothing you can do to fix that - that is an inherent property of floating point math. If you need exact precision with your calculations, don't use a floating point type.

[johnggold]

I wish I could use that argument with our application software - sorry - it just makes mistakes!!! (lol)

Based on what you said, the RoundDouble() function uses longs

long result;

if( val >= (double) 0 )
{
result = (long) ( ( val * (double) 100 ) + (double) 0.5 );
}
else
{
result = (long) ( ( val * (double) 100 ) - (double) 0.5 );
}

return ( (double) result / (double) 100 );

I assume that if RoundDouble was changed to RoundDoubleToLong() to return long, this would fix the problem.

The above code would change to returning result.

[LB]

Casting away the decimal places is definitely not ideal. There are existing C and C++ libraries for exact math with decimal types, for example: https://gmplib.org/