Precision and CD's

Gaston Gonnet gonnet at
Wed Jul 21 09:42:47 CEST 1999

I am afraid to say that I haven't seen so much misinformation
about floating point as I have seen in this discussion.

Lets state a couple of facts (indisputable to all the people
who have taken at least Numerical Analsys 101).

(1) All floats, bigfloats or IEEE floats, etc. represent exact

(2) All (arithmetic) operations are done exactly.

(3) The result is represented as a floating point number,
choosing the fp number (in certain representation) which is
closest to the exact result.

It is ONLY in step number 3 that the so called "precision"
will come into play; not in 1, not in 2.

Notes:  in (3), technically speaking, it is not always the
closest, typical hw will allow for results within some
epsilon from 1/2 a ulp.

	in (2), some people may say this is a red herring,
but I would like to insist that this is the CORRECT MODEL,
regardless on how the operations are actually done.

	in (2) it is extremely desirable to extend this to
all functions, not just arithmetic (e.g. exp, ln, sin, ...)
Most modern hw and sw comply with this.

	The above applies for ANY fp system, and any
representation, including mixed ones.

As you can see from the above, PRECISION IS A CONCEPT WHICH

And please, please, please do not tell me that 1, 1.0, 10000e-05
and 1.0000<one million 0's>000 are different....

Humbly yours, Gaston.

More information about the Om mailing list