Precision and CD's

Richard J. Fateman fateman at
Wed Jul 21 16:58:52 CEST 1999

I wonder if it is just a disagreement over the wording, but
in fact I view "precision" as an attribute of a representation.
The operations are performed with (or without) ERROR.  An
exact operation has no error.  An operation that has a small
relative error (say 1/10^8) might be said to have an error
of one unit in the last place (ULP) if carried to 8 decimal
digits of precision.

The OM CD for bigfloats could be though of as describing
values (any old way)  and

Thus  v=125x10^-3  is a value.  no error, no precision specified.
       v stored as a binary float exactly requires only one bit
in its fraction, and thus it can be exactly represented to
precision 1 bit.

om at  -  general discussion on OpenMath
Post public announcements to om-announce at
Automatic list maintenance software at majordomo at
Mail om-owner at for assistance with any problems

More information about the Om mailing list