[ka-Map-users] KaMap performance tuning

[Steve Lime]

Did you index the shapefiles?

>>> On 5/22/2007 at 4:16 PM, in message
<34E056CD4D475841B8C9ABCE956F92F63709B2 at mrburns.KorOffice.local>, "Jeff
Dege"
<jdege at korterra.com> wrote:
> We'd never had trouble with the performance of shapefiles, until we
had to 
> deal with one that contained the California streets.   500MB .shp
file, 600MB 
> .dbf file, and performance with it was dog slow.
> 
> We pulled out interstates, highways, and main roads, into separate 
> shapefiles, so that we didn't have to touch the streets data until we
were 
> zoomed in tight, and performance was still dog slow when we did.
> 
> We split the streets shapefile into hundreds of separate tiles,
shptreed 
> each, built a tileindex on the bunch, then shptreed the tileindex,
and 
> performance was a good bit better, but still not acceptable.
> 
> The people I've been talking to have told me that when rendering
everything, 
> shapefiles are faster, but when rending small subsets of the data,
PostGIS 
> can be considerably faster.  Particularly if you use it's simplify()
function 
> to tune the resolution of the data returned to the scale of the map. 
I've 
> only begun experimenting with this, but nothing in my experience so
far 
> contradicts this.
> 
> Shp2img, rendering a map with an extent of .04 decimal degrees, took
17.198 
> seconds to render the streets layer using (tiled) shapefiles, it took
0.219 
> seconds to render the streets layer using PostGIS.
>  
> 
>> -----Original Message-----
>> From: Melanie Ritzau [mailto:MelanieRitz78 at web.de] 
>> Sent: Tuesday, May 22, 2007 4:01 PM
>> To: Jeff Dege
>> Subject: Re: [ka-Map-users] KaMap performance tuning
>> 
>> Hi Jeff,
>> 
>> I am quite surprised with the results of your performance 
>> test. In my experience (and others) shapefiles are a bit 
>> (5-10%) faster than POSTGIS.
>> I wonder if you optimized your shapefiles ?
>> 
>> Was your test with one big Shapefile (what size ?) or did you 
>> tile your shapefile ? 
>>
(http://mapserver.gis.umn.edu/docs/reference/utilityreference/tile4ms)
>> Are your shapefiles indexed with shptree ? 
>>
(http://mapserver.gis.umn.edu/docs/reference/utilityreference/shptree)
>> 
>> To your problem: If you have large data-sets you need to 
>> increase time and memory limits in PHP. You can add e.g. in tile
PHP:
>> 
>> ini_set("memory_limit","600M");
>> set_time_limit(1200);
>> 
>> This helped me with a similar problem.
>> 
>> 
>> Thanks,
>> Mel
>> 
>> 
>> > -----Ursprüngliche Nachricht-----
>> > Von: "Jeff Dege" <jdege at korterra.com>
>> > Gesendet: 21.05.07 21:42:23
>> > An: <ka-map-users at lists.maptools.org>
>> > Betreff: [ka-Map-users] KaMap performance tuning
>> 
>> 
>> > 
>> > We've been using KaMap with Mapserver and OpenLayers, with 
>> some success.
>> > We've recently begin looking at using PostGIS instead of
shapefiles,
>> > hoping to getter performance when generating maps that 
>> haven't yet been
>> > tiled.
>> > 
>> > So we loaded our shapefiles into PostGIS tables, and 
>> created a new map
>> > file that was a copy of the first, except that it got its 
>> map data from
>> > the PostGIS tables.
>> > 
>> > Our first comparison was using shp2img, which showed a 
>> dramatic decrease
>> > in the time needed to generate maps - with our data, 
>> PostGIS was eight
>> > to ten times faster.
>> > 
>> > So we cleared out our tile cache, pointed KaMap at the 
>> PostGIS map file,
>> > and saw very little difference.
>> > 
>> > So we created a batch file that would download a thousand
different
>> > tiles, using wget to make direct calls against tile.php, 
>> and timed how
>> > long it took to run, with the shapefile and the PostGIS map 
>> files.  The
>> > PostGIS test was only five percent that the shapefile test.
>> > 
>> > So we created yet another batch file that used wget to make 
>> direct calls
>> > against mapserver.exe, running in CGI mode.  When 
>> generating maps from
>> > PostGIS data, mapserver was eight times faster than against 
>> shapefiles.
>> > 
>> > So now the question:  If the image creation is so much faster,
when
>> > accessed through shp2img, and when accessed through 
>> mapserver.exe, why
>> > is that speed improvement not visible when loading tiles 
>> through kamap's
>> > tile.php?
>> > 
>> > It's clear to me that there must be some bottleneck, reducing
>> > performance.  My guess would be in Apache or PHP configuration.
>> > 
>> > Does anyone know how I can measure what time is being spent 
>> doing what,
>> > in this area?  What configuration options I should look at, 
>> to try to
>> > improve performance?
>> > 
>> > Thanks.
>> > 
>> > 
>> > _______________________________________________
>> > ka-Map-users mailing list
>> > ka-Map-users at lists.maptools.org 
>> > http://lists.maptools.org/mailman/listinfo/ka-map-users 
>> > 
>> 
>> 
>> _______________________________________________________________
>> SMS schreiben mit WEB.DE FreeMail - einfach, schnell und
>> kostenguenstig. Jetzt gleich testen! http://f.web.de/?mc=021192 
>> 
>> 
> 
> _______________________________________________
> ka-Map-users mailing list
> ka-Map-users at lists.maptools.org 
> http://lists.maptools.org/mailman/listinfo/ka-map-users