Precision and CD's

Andreas Strotmann strotman at nu.cs.fsu.edu
Wed Jul 21 01:11:12 CEST 1999


> 
> No, precision is for the conversion to internal form -- although,
> you're right, we could also adopt a (somewhat risky) rule that
> exactly the printed digits are precise, and drop any explicit precision.

Hmm, didn't someone mention that you sometimes need to send more decimal
digits than you'd expect in order to get the exactly right binary
precision?  In other words, the actual decimal precision of the number
would be less than the number of digits that you print in an important
class of cases, if I understood this correctly.

 -- Andreas




More information about the Om mailing list