Precision and CD's

[Bruce R Miller]

J H Davenport wrote:
> 
> >>> Since it gets tiring to write (and read) numbers
> >>> like 1.000000000000000000000 and 0.00000000000000000000
> >>> you might prefer to say
> >>> 1.0e0 precision 35.
> 
> Aha, so precision is purely for the benefit of humans (and possibly
> some transmission economy, but if that interested us, we wouldn't be
> using XML!).

No, precision is for the conversion to internal form -- although,
you're right, we could also adopt a (somewhat risky) rule that
exactly the printed digits are precise, and drop any explicit precision.

[I should probably stay out of this part of the debate but...]

Any given float (or bigfloat) (of a particular representation on a
particular machine...) IS a rational number.
But the Set of floats (of a particular rep...) is NOT the set of 
Rational Numbers.  And they're used differently; as Fateman said
"the precision is bounded"
: you want some number `sufficiently close'.

While it may make sense when moving data into/outof the same 
representation to say you want exactly the same float (with 
exactly the same bits), I'm not so sure that makes sense 
when moving data between systems of unknown internal properties.
(I'm not absolutely sure it doesn't make sense, though)

So, in converting a float/bigfloat into an internal representation,
the precision simply specifies `how close' the internal float 
ought to be to the transmitted description of the float, be it a string
or bigfloat description.  (nothing to do with error bars!)

--
bruce.miller at nist.gov
http://math.nist.gov/~BMiller/