[Proj] Numerical Precision in Proj, WKT, and everywhere

Gerald I. Evenden geraldi.evenden at gmail.com
Fri Apr 2 09:41:17 EST 2010

On Thursday 01 April 2010 8:42:25 pm Craig Bruce wrote:
> "Gerald I. Evenden" <geraldi.evenden at gmail.com> wrote:
> > Au contraire. When you have a number with few significant figures that
> > have trailing zeroes such as "0.999600....000" any format that has
> > sufficient width to contain the significant figures will preserve the
> > significance. There is no rounding nor trucation problem.
> Obviously, but I'm not talking about specific values that happen to
> be round.

But this was the example you used in your first example.

> > The only hassle occurs when the the fractions are irrational such as pi,
> > then there is a problem.
> There's also a problem with rational fractions, like 1/60, as I showed
> in the WMS example.  You can easily write x = 1/60.0 in a program get a

I'm discussing decimal fractions--- xx.02935729350

> fully precise value for whatever representation you are using, but you
> can't put this expression on the proj command line (AFAIK).

No need to.

> > But in these situations (and probably others) the continual bouncing
> > back and forth between decimal and binary should be avoided. I strongly
> > suspect that there is no good reason for it anyway.
> I agree that it's best to avoid unnecessary conversions, but there are
> many interfaces that require text, such as the proj command-line or OGC
> web interfaces.  The same issue occurs with the database/application
> interface with BCD.  We are forced to obey the interfaces of the various
> systems we wish to chain together.

BCD? Binary Coded Decimal? Just that I have not heard the term in maybe 25 
years. Seriously. I am not being sarcastic.

In most cases there is an acceptible precision for representing a number. For 
example pi may only be needed as 3.1415927. It is assumed that zeroes are 
appended indefinitely. As long the the intermediate formating retains a 7 
digit significance, the number will be preserved regardless of binary/decimal 

As I recall, the only place you demonstrated precision problems was an example 
that went to full mantissa precision where I admit there may be problems. 
*But* I know of no place in cartography where such precision is required.

Lastly, if one is involved with "round tripping" geographic/Cartesian 
coordinates there will be a problem but this problem more likely to be a 
hassle with the inverse projection nor exactly producing original value in 
addition. Formatting may also create a problem and this is why all my program 
proj versions had a binary pipe so that a round trip could be made without 
going through the formatter. But this round tripping should be avoided at all 

The whole religious complexion of the modern world is due
to the absence from Jerusalem of a lunatic asylum.
-- Havelock Ellis (1859-1939) British psychologist

More information about the Proj mailing list