First of all, keep in mind nans don't have signs. The meaning of the minus in the output is implementation-defined. The output really should be [-]nan(string), but nobody seems to like supporting that part of C I/O.
GCC evaluates the expression
INFINITY - INFINITY
as the NaN
0xff c0 00 00
, and the expression
inf - inf
where inf is a float containing +infinity, as the NaN
0x7f c0 00 00
.
The first of these NaNs has the first bit set, so the output displays it as "-nan" Why are they different? I have no idea. The exact value of the NaN is implementation-defined in all situations. These two NaNs are both evaluated at compile time (in my test), so it's somewhere deep in gcc implementation.
Interestingly, that minus is actually due to a recent (2009-08-23 to be exact) change to the GNU C I/O. Earlier versions of GCC (as recent as 4.3.4, but not 4.5.3, printed "nan" for both:
http://ideone.com/lquru vs
http://ideone.com/AP4lu )
For variety, HP compiler generates a
0xff c0 00 00
for both (displayed as -nan), and clang++ generates a
0x7f c0 00 00
for both (displayed as nan)