[Proj] Lat-Long to Lambert Conformal Conic(2SP) transformation inaccuracy issue

[Gerald I. Evenden]

On Sunday 18 February 2007 5:31 am, Tonu Lukk wrote:
> Thanks for the reply. First a correction - in my original posting the  
> backslashes were removed from the command line. They were actually present
>   when I ran the tests. My mistake.
> It does seem to be a windows' shell problem but I can't really figure out  
> what exactly is causing it.
>
> Two tests.
> 1.
> proj +proj=lcc +lat_1=58N +lat_2=59.333333N +lat_0=57.517553N +lon_0=24E  
> +x_0=500000 +y_0=6375000 +ellps=WGS84
>
> Input 24E 57d31'3.19415"N gives 500000.00       6375000.10
> Input 24E 57.517553N      gives     500000.00       6375000.00
> This hints that the problem lies in the command line parameters - as far  
> as I understand the difference in conversion is most likely caused by  
> rounding accuracy of an input value.
>
> 2.
> proj +proj=lcc +lat_1=58N +lat_2=59d20\'N +lat_0=57d31\'03.19415\"N  
> +lon_0=24E +x_0=500000 +y_0=6375000 +ellps=WGS84
>
> Input 24E 57d31'3.19415"N gives 500000.00       6375098.83
> Input 24E 57.517553N      gives 500000.00       6375098.72
> Obviously the problem is in command line parameters. Further testing  
> showed that it (in my case) is caused by the +lat_0=57d31\'03.19415\"N  
> parameter. Also tried with \" removed from the end, but with the same  
> results. Somehow it is interpreted wrong.
> Any ideas why?

In this case, the precision in the conversion from DMS to fractional degree 
causes the slight error.  From gp

? (31+3.19415/60)/60
%1 = 0.51755393055555555555555555555555555555

Thus a test script illustrates the correction:

#
echo "original script"
lproj +proj=lcc +lat_1=58N +lat_2=59d20\'N +lat_0=57d31\'03.19415\"N 
+lon_0=24E +x_0=500000 +y_0=6375000 +ellps=WGS84 <<EOF
24E 57d31'3.19415"N gives 500000.00       6375098.83
24E 57.517553N      gives 500000.00       6375098.72
EOF
echo "expanded decimal degree fraction (from gp)"
lproj +proj=lcc +lat_1=58N +lat_2=59d20\'N +lat_0=57d31\'03.19415\"N 
+lon_0=24E +x_0=500000 +y_0=6375000 +ellps=WGS84 <<EOF
24E 57d31'3.19415"N gives 500000.00       6375098.83
24E 57.51755393055556N      gives 500000.00       6375098.72 
EOF
       --------------------------------- change (bit of an overkill now  ;-)  
which generates:

original script
500000.00       6375000.00 gives 500000.00       6375098.83
500000.00       6374999.90      gives 500000.00       6375098.72
expanded decimal degree fraction (from gp)
500000.00       6375000.00 gives 500000.00       6375098.83
500000.00       6375000.00      gives 500000.00       6375098.72

Now both examples yield the same answer.  Your original latitude had a 6 digit 
fraction where the DMS was closer to an 8 digit fraction.

The devil is in the details.  :-)
-- 
The whole religious complexion of the modern world is due
to the absence from Jerusalem of a lunatic asylum.
-- Havelock Ellis (1859-1939)  British psychologist