|
|
Posted by Ian B on 12/05/05 17:10
Gordon Burditt wrote:
> >Variables are held to s certain degree of precision. 200.15 cannot be
> >held accurately as a normal binary number, for example, so errors can
> >occur.
> >
> >Use bcsub(200.15,200,n) where n is the number of decimal places you
> >want.
>
> Any attempt to round to "a number of decimal places" will virtually
> guarantee roundoff error. There are very few non-integer decimal
> numbers which can be represented exactly in binary floating
> point. All of them end in 5 (plus trailing zeroes).
>
> Gordon L. Burditt
bcsub doesn't round
Navigation:
[Reply to this message]
|