Precision and CD's
Richard J. Fateman
fateman at cs.berkeley.edu
Wed Jul 21 16:58:52 CEST 1999
I wonder if it is just a disagreement over the wording, but
in fact I view "precision" as an attribute of a representation.
The operations are performed with (or without) ERROR. An
exact operation has no error. An operation that has a small
relative error (say 1/10^8) might be said to have an error
of one unit in the last place (ULP) if carried to 8 decimal
digits of precision.
The OM CD for bigfloats could be though of as describing
values (any old way) and
representation.
Thus v=125x10^-3 is a value. no error, no precision specified.
v stored as a binary float exactly requires only one bit
in its fraction, and thus it can be exactly represented to
precision 1 bit.
RJF
-
om at openmath.org - general discussion on OpenMath
Post public announcements to om-announce at openmath.org
Automatic list maintenance software at majordomo at openmath.org
Mail om-owner at openmath.org for assistance with any problems
More information about the Om
mailing list