# Precision and CD's

Andreas Strotmann strotman at nu.cs.fsu.edu
Tue Jul 20 16:14:34 CEST 1999

```
On Mon, 19 Jul 1999, Richard J. Fateman wrote:

> I think that the rational numbers (ratios of arbitrary precision
> integers) solve the problem that you seem to be addressing
> (how to represent 1/7, etc.)

Sure... except they're at this point not allowed as OMFs.

> Basically I think of the floats as a selection of rational
> representatives from the REAL numbers, so a completion in the
> rationals is not interesting.
>
> Other considerations:
>
> The major advantage of floats/ bigfloats  is that the cost of
> operations is mostly a function of the desired precision, and
> not (usually) the value. The usual
> operations include sqrt, exp, log, sin, cos  etc. as well as +-*/.
>
If that's what you consider the essence of bigfloats, then the
representation should include a specification for the precision -- but
several people on the list (including you) have argued vehemently against
this: "bigfloats are exact rationals".

I'm sorry, but I don't think you can have it both ways.  Either you're
serious about the exact rationals interpretation of bigfloats, in which
case you'd have to do the simple extension to include all rationals; Or
you're serious about them having a precision, in which case you'd have to
specify that piece of information (and its semantics).

Leaving out *both* pieces of information does not make sense
mathematically, IMHO.

> Closing the floats exactly under +  is already a bad idea because
> 1+1*10^1000  requires 1000 decimal digits in the fraction.

closed under + for the simple reason that it is perfectly valid to have
1000 decimal digits in the fraction.  ((This is a closure "in principle"
-- that is within the *representation* --, though not always "in practice"
-- that is within an *application*.))

In practice, of course, applications would take the usual best shot at
interpreting a bigfloat, and would have their own rules for producing
them, with some internal definition of "precision".  But there's a
difference between sloppy use and sloppy specification -- in the former
case, an application decides to drop information, while in the latter, the
application is forced to do that even if it knows better.

> Closing the floats under division (except division by zero) just to
> make them a field has some esthetic mathematical appeal, but in my
> opinion it is not worth the complexity and cost of handling repeated
> decimals or unknown expansion size.

If a system knows how to really handle these, the added complexity is
negligible.  Most systems won't, but these systems will have to handle
arbitrary bigfloats in essentially the same way always, and the algorithm
is exceedingly simple:  truncate the series of digits in the mantissa to
the minimum that is required to determine all the bits in my internal
representation -- and treat that truncated sequence of digits just the way
any "normal" float would be treated.  Handling repeated decimals to fill
such a read buffer is only marginally more complicated than handling
non-repeated decimals (and well within the skill range of a CS 101
course).

-- Andreas

```