[FWTools] Performance

[Frank Warmerdam]

On 10/27/05, Evert Groenewald <evertg at telkomsa.net> wrote:
>  I'm using FWTools1.0.0a2 on a computer with the following specs.
>  Motherboard: MSI K8N NEO4/SLI
>  CPU:   AMD Athlon 64-X2 4400+ SOCKET 939 Dual Core 64-Bit With HyperT
>  Memory: 2x1GB Kingston DDR400
>  Graphics card: NVIDIA Geforce 6600 Turbo Edition 256 MB Memory
>  HHD: Western Digital 120GB SATA
>  OS: Linux SuSe 10.0  86_64 with a 4GB swapfile
>
>  The data I wish to view is in various file formats but the two primary
> files is in the .ecw format. The other files are mostly vector layers
> althought some of them contain a lot of data, my problem lies with the two
> .ecw files. The first is 142MB and the second 850MB in size. Opening the
> 142MB file noticably slows down the computer even when invoking the "-h"
> command during the start-up to enable hardware acceleration, but things are
> still workable. When I open the second file, the 850MB one, things virtually
> comes to a standstill. The two maps represented by the two .ecw raster
> layers overlap and ideally I would like to overlay them and then ad the
> vector layers as needed. At present this is impossible as everything grinds
> to a halt at the mere hint of opening the two files together.
>  My question is thus: Is there any way to improve performance by tweaking
> software or enabling settings I'm unaware of at present?
>  I did the install as suggested, eg:  ./install.sh   .

Evert,

This is an unfortunate side effect of several implementation
decisions I have made.

 1) The ECW driver does not represent itself as having
     overviews.  This means that applications like OpenEV
     with special logic to operate against overviews will not do
     so.

 2) OpenEV uses one screen texture for each 256x256 chunk
     of the file at full resolution.  For large images this means that
     that there are alot of textures often loaded at very low resolution.
     For some formats doing lots of little calls is no problem but for
     ECW this results in huge numbers of 256x256 setwindows with
     very small output buffers (ie. 4x4 or 8x8 in overview mode).  This
     is a worst case for a format like ECW.

If I correct (1) by having virtual overview layers then we can just
turn on forced block oriented access with the GDAL_FORCE_CACHING
ON config variable.  This will mean that (2) won't matter much
since the overviews will be read in big chunks (1024x128 I think)
and one read from the ECW can be used to populate many textures.

So, for now it is essentially impossible to get decent performance
from ECW files in OpenEV in overview mode.

Note that this issue is due to bad approaches on my part, and are
not a reflection on the ECW format or SDK.  For instance MapServer
get much better performance in overview mode from ECW because
it makes a single big request to GDAL and GDAL turns that into
one set window request to the ECW SDK.

Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent