[Proj] Numerical Precision in Proj, WKT, and everywhere
Gerald I. Evenden
geraldi.evenden at gmail.com
Thu Apr 1 18:38:28 EST 2010
On Thursday 01 April 2010 6:32:15 pm Craig Bruce wrote:
> "Gerald I. Evenden" <geraldi.evenden at gmail.com> wrote:
> > Prey tell, why is someone using %,16g to print out a numbet that is
> > rarely, if ever, specified to more 4 to 6 digits? %g would be completely
> > adequate and avoid the absurd gyration suggested in the original email.
> Sometimes those 4 to 6 digits aren't round in decimal, so %g truncates the
> value quite a lot.
Au contraire. When you have a number with few significant figures that have
trailing zeroes such as "0.999600....000" any format that has sufficient
width to contain the significant figures will preserve the significance.
There is no rounding nor trucation problem.
The only hassle occurs when the the fractions are irrational such as pi, then
there is a problem. But in these situations (and probably others) the
continual bouncing back and forth between decimal and binary should be
avoided. I strongly suspect that there is no good reason for it anyway.
As and aside, all the binary to decimal conversion procedures that I have run
into always round. The only hassles I have seen is when the precision is
close to the limits of the floating point binary mantissa.
The whole religious complexion of the modern world is due
to the absence from Jerusalem of a lunatic asylum.
-- Havelock Ellis (1859-1939) British psychologist
More information about the Proj