1e9 vs 1e-9

In the context of rounding to the ninth digit, I found operating with values 1e9 or 1e-9 to return different results:

1
2
3
double x=500.00000000000006;
x=floor(x/1e-9+0.5)*1e-9;  // result: x=500.00000000000006
x=floor(x*1e9+0.5)/1e9;    // result: x=500.00000000000000 


What is the difference in using 1e9 and 1e-9? To my understanding the precision in representation as a floating point number (exponent and fraction) differs only in the sign of the exponent. Algebraic, both expressions are equivalent.

Any light on that is appreciated.
Thanks in advance.
What is the difference in using 1e9 and 1e-9? To my understanding the precision in representation as a floating point number (exponent and fraction) differs only in the sign of the exponent


This is typical rounding error.

doubles have 54 bits of mantissa/precision. Therefore they have an effective range of 254 = 4503599627370496 possible values (before exponent). That's roughly 15 digits of precision.


500.00000000000006 requires 17 digits of precision. Therefore operations involving that number are very likely to have rounding error.



If you want to know specifically why it's rounding, then the only thing I can suggest is to examine the binary representation of the double during each step in this computation. Since that is too much work for me, I didn't bother with it for my reply.
Last edited on
I kind of thought in that direction.
But what makes me wonder, that the operation succeeds using 1e9 to scale and fails using 1e-9. Rounding seems to work properly up to the point scaling the result back.
I guess you are right, to investigate further, I should look into the binary representation.
In C++ the 'e' number is base 10. But in the binary representation, the exponent is base 2. So 1e9 and 1e-9 are likely going to be represented very differently. This can lead to discrepancies like you're seeing.
I guess binary representation is the key to this question. I marked the topic as solved. I am still curious and might follow the binary memory some time. Meanwhile thank you for your thoughts
Whole numbers can be exactly represented as doubles if their binary representation does not exceed mantissa capacity. 10^9 fits perfectly.

On the other hand only fractions which can be represented as a sum of different exponents of 2 (and difference between those exponents does not exceeds mantissa size) can be represented exactly. 0.1 for example cannot and only rounding to lesser precision allows you to see it when you output it to the screen. 10^-9 cannot be represented exactly too.
For completeness:

1
2
double cc=1/1e-9;
// cc	999999999.99999988	 


1
2
double cc=1/1e-10;
// cc	10000000000.000000	 


This pretty much shows the consequences from inability to represent 1e-9 as a binary floating point number as MiiNiPaa wrote.

I used the large (integer) number in my implementation now along with the appropriate mathematical operations for rounding to the billionth digit after the dot as supposed.

Thank you again for your comments
Topic archived. No new replies allowed.