[FWTools] (no subject)

Frank Warmerdam warmerdam at pobox.com
Mon Oct 12 23:00:27 EST 2009


Ernst Meier wrote:
> Hello,
> 
> I have created an SHP-File (polygon) with ArcGIS 9.2. I let ogrinfo show the coordinates of the vertexes and got decimals like .123000005 instead of
> .123 or .129999999999 instead of .130 
> 
> Any Ideas? What causes this change? Maybe a bug in ArcGIS or FWTools? 

Ernst,

Shapefile coordinates are stored in double precision IEEE floating point
and in that representation there are numbers that cannot be exactly
represented - perhaps including values like .123.  In these cases the
internal representation is very close and whether you see 0.123 or
0.12999999999999 will depend on the precision displayed.  So I believe
ogrinfo is just reporting so much precision that you see these small
inaccuracies.

There isn't really much to do about this.

Best regards,
-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent



More information about the FWTools mailing list