Re: strange casting problem

From: Kevin B. McCarty <kmccarty_at_gmail.com>
Date: Fri, 25 Apr 2008 20:57:35 -0700


nguler_at_jlab.org wrote:
> Thanks for your input.
> I verified that the source of this problem is as described by Kevin.
> And interestingly, the problem goes away if I change:
>
> double aa = W/W_width ;
> to:
> float aa = W/W_width ;
>
> So, this looks like some kind of imperfection of "double" type.

Not exactly, no. What happens in that case is that the result "W/W_width" is still treated (by rules of C/C++ grammar) as a double, but then you cast it to a float. This is just as if you had written

  double bb = W/W_width;
  float aa = float(bb);

Since floats have less precision than doubles, the nearest possible float to the value of that double (which is not in general an integer) is in fact the integer you were expecting. I don't know whether that will always hold true, though.

Basically, by using the intermediate float variable, you're just rounding, but in an unpredictable way. I'd highly recommend rounding explicitly as with "return int(floor(aa + 0.5));" instead.

best regards,

-- 
Kevin B. McCarty <kmccarty_at_gmail.com>
WWW: http://www.starplot.org/
WWW: http://people.debian.org/~kmccarty/
GPG: public key ID 4F83C751
Received on Sat Apr 26 2008 - 05:57:51 CEST

This archive was generated by hypermail 2.2.0 : Sat Apr 26 2008 - 11:50:02 CEST