Precision and CD's

[Richard J. Fateman]

I wonder if it is just a disagreement over the wording, but
in fact I view "precision" as an attribute of a representation.
The operations are performed with (or without) ERROR.  An
exact operation has no error.  An operation that has a small
relative error (say 1/10^8) might be said to have an error
of one unit in the last place (ULP) if carried to 8 decimal
digits of precision.

The OM CD for bigfloats could be though of as describing
values (any old way)  and
representation.

Thus  v=125x10^-3  is a value.  no error, no precision specified.
       v stored as a binary float exactly requires only one bit
in its fraction, and thus it can be exactly represented to
precision 1 bit.


RJF
-
om at openmath.org  -  general discussion on OpenMath
Post public announcements to om-announce at openmath.org
Automatic list maintenance software at majordomo at openmath.org
Mail om-owner at openmath.org for assistance with any problems