[ka-Map-users] KaMap performance tuning

[percy]

I recently was able to get an extremely complex geologic data shapefile set to render much faster (after doing all of the other optimizations mentioned by everyone else) by tuning the filter expressions.

I changed all class expressions from the following statement type
 CLASS
  NAME 'Dominant:Clastic sediment;'
  EXPRESSION ('[LithComp]' eq 'Dominant:Clastic sediment;')
  STYLE
   COLOR 255 255 0
  END #STYLE
 END #CLASS

to

 CLASS
  NAME 'Dominant:Clastic sediment;'
  EXPRESSION 'Dominant:Clastic sediment;'
 
  STYLE
   COLOR 255 255 0

  END #STYLE
 END #CLASS

I was stunned at the performance increase this gave me, though it's obvious why it helps so much (in hindsight). The only reason I was using the former EXPRESSION statements was because I had used QGIS or AMEIN to auto-generate the initial mapfile...

Hope this helps,
Percy



Date: Tue, 22 May 2007 19:13:03 -0700
From: Paul Ramsey <pramsey at refractions.net>
Subject: Re: [ka-Map-users] KaMap performance tuning
To: Jeff Dege <jdege at korterra.com>
Cc: ka-map-users at lists.maptools.org
Message-ID: <6207F842-AAF6-44EE-8A58-B08C960D4812 at refractions.net>
Content-Type: text/plain; charset=ISO-8859-1; delsp=yes; format=flowed

10:1 speed advantage for PostGIS is nice and all, but seems unlikely,  
even to this PostGIS zealot...

Does your layer have a FILTER on it that uses the attributes? PostGIS  
does that great, shapefiles do it terribly.

I would expect on a very large spatial file (your 500MB example) the  
PostGIS r-tree will be balanced better than the quadtree .qix file,  
but that should provide incremental advantage, not massive  
advantage.  One possibility is that you're running out of depth in  
your quadtree, maybe tree bumping up the depth in your shptree call.

P


>   
>> We'd never had trouble with the performance of shapefiles, until we  
>> had to deal with one that contained the California streets.    
>> 500MB .shp file, 600MB .dbf file, and performance with it was dog  
>> slow.
>>
>> We pulled out interstates, highways, and main roads, into separate  
>> shapefiles, so that we didn't have to touch the streets data until  
>> we were zoomed in tight, and performance was still dog slow when we  
>> did.
>>
>> We split the streets shapefile into hundreds of separate tiles,  
>> shptreed each, built a tileindex on the bunch, then shptreed the  
>> tileindex, and performance was a good bit better, but still not  
>> acceptable.
>>
>> The people I've been talking to have told me that when rendering  
>> everything, shapefiles are faster, but when rending small subsets  
>> of the data, PostGIS can be considerably faster.  Particularly if  
>> you use it's simplify() function to tune the resolution of the data  
>> returned to the scale of the map.  I've only begun experimenting  
>> with this, but nothing in my experience so far contradicts this.
>>
>> Shp2img, rendering a map with an extent of .04 decimal degrees,  
>> took 17.198 seconds to render the streets layer using (tiled)  
>> shapefiles, it took 0.219 seconds to render the streets layer using  
>> PostGIS.
>>
>>     
-- 
David Percy
Geospatial Data Manager
Geology Department
Portland State University
http://gisgeek.pdx.edu
503-725-3373