[Proj] Numerical Precision in Proj, WKT, and everywhere

[Gerald I. Evenden]

On Thursday 01 April 2010 6:32:15 pm Craig Bruce wrote:
> "Gerald I. Evenden" <geraldi.evenden at gmail.com> wrote:
> > Prey tell, why is someone using %,16g to print out a numbet that is
> > rarely, if ever, specified to more 4 to 6 digits? %g would be completely
> > adequate and avoid the absurd gyration suggested in the original email.
>
> Sometimes those 4 to 6 digits aren't round in decimal, so %g truncates the
> value quite a lot. 

Au contraire. When you have a number with few significant figures that have 
trailing zeroes such as "0.999600....000" any format that has sufficient 
width to contain the significant figures will preserve the significance. 
There is no rounding nor trucation problem.

The only hassle occurs when the the fractions are irrational such as pi, then 
there is a problem. But in these situations (and probably others) the 
continual bouncing back and forth between decimal and binary should be 
avoided. I strongly suspect that there is no good reason for it anyway.

As and aside, all the binary to decimal conversion procedures that I have run 
into always round. The only hassles I have seen is when the precision is 
close to the limits of the floating point binary mantissa.

-- 
The whole religious complexion of the modern world is due
to the absence from Jerusalem of a lunatic asylum.
-- Havelock Ellis (1859-1939) British psychologist