[OSRS-PROJ] Significant digits in parameters

Gerald Evenden gerald.evenden at verizon.net
Sun Jul 28 09:52:48 EDT 2002

Craig Bruce wrote:

> Gerald Evenden <gerald.evenden at verizon.net> wrote:
> > Using a larger number of digits in projection parameters will, for a
> > variety of reasons, not increase the accuracy of the results For example,
> > the major axis is only spec'd to a 7 digit integer, thus the accuracy
> > of the x-y values are also only to the nearest meter regardless of the
> > significant digits of the other parameters.
> I realize that what you are saying is geomatically correct, but talking
> about software issues.  It's advantageous if different software or even the
> same software produces results that are either the same or very close to
> being the same for the same input.

I presume you mean that prog A produces the same results as prog B
to 15+ digits.  I am sorry but I couldn't disagree with you more for
several reasons:
1) it would make an observer of the tabulation of results assume that
these values were accurate to the printed significance.
2)different programs that are perfectly usable in the case of say 1mm
precision will create chaos when compared with another program accurate to
1nm precision.  Which program is correct?  Which program do I throw

Overprinting the precision is just a bad practice.

> For this reason, the best plan is to
> represent all parameters and calculations using full 'double' precision,
> even if it contains many more digits than are geomatically significant.
> If the rest of the system computes values using full double precision,
> why would you want to unnecessarily remove consistency with other systems
> especially when the solution is as simple as changing the "%.9f" to a
> "%.16g" when auto-generating the tables.

I am not sure what is going on in the following paragraph but I must
say that the discussed method has a serious defect if it is dependent
upon knowing the precision of the input but has no way of having
this condition as controlling input and merely depends upon the
precision of double precision.

> There are many places where round-off error control is applied and
> full precision allows programs to produce the same results (e.g., in
> error-controlling the translation of projection coordinates to integer
> pixel coordinates) and to make optimizations when computed coordinates
> align properly.  A lot of software doing error control may not even know
> that it is dealing with ground coordinates, so it may assume that there
> are 12-15 significant digits (or "consistent digits" in our case).

PROJ.4 Discussion List
See http://www.remotesensing.org/proj for subscription, unsubscription
and other information.

More information about the Proj mailing list