[OSRS-PROJ] Significant digits in parameters

Craig Bruce csbruce at cubewerx.com
Sat Jul 27 16:53:56 EDT 2002

Paul Ramsey <pramsey at refractions.net> pointed out that the EPSG, et al.
coordinate-system definitions used in PROJ.4 look like the following:

+proj=lcc +lat_1=47.500000000 +lat_2=48.733333333 +lat_0=47.000000000
+lon_0=-120.833333333 +x_0=609601.219 +y_0=0.000 +ellps=clrk66
+datum=NAD27 +to_meter=+0.3048006096

A significant issue here is that all of the numbers appear to be given
with a fixed format of only nine decimal places (C "%.9f").  I strongly
recommend that a floating format of 16 significant digits be used instead
(C "%.16g") because most computations are performed using 'double'
values and sixteen significant digits is the (close-decimal-equivalent)
accuracy of this type.  If computations are carried out with only nine
or so significant digits, then the calculations will be unnecessarily
imprecise, and this can lead to problems with code that deals with
floating-point rounding control.

For example, we may want to check if a computed longitude is within
thirteen significant digits of being equal to another or to some to
round figure, and the needless imprecision will confound this useful
functionality.  (Most of the PROJ.4 computations seem to be accurate to
about 13 digits otherwise.)

The ESRI example:


has the right idea.

Dr. Craig S. Bruce        | Tel: 819-771-8303 x205 |             CubeWerx Inc.
Senior Software Developer |   Home: 613-565-1344   |  Gatineau, Québec, Canada
csbruce at cubewerx.com      | http://www.csbruce.com |  http://www.cubewerx.com/
    "Everything should be built from the top down, except the first time."
PROJ.4 Discussion List
See http://www.remotesensing.org/proj for subscription, unsubscription
and other information.

More information about the Proj mailing list