[FWTools] Trying to translate gtif ASTER image to jpg but v. slow

[Frank Warmerdam]

Simon J. Hook wrote:
> Hi Frank,
> 
> I have put the file on my ftp site. It requires a username and password 
> to retrieve so I have not sent this message to the FWTools list (I hope 
> that is okay). I tried running it with the new FWTools (1.2) on a v. 
> fast machine (dual 3ghz, 4gb ram). Our code which generates the geotif 
> from the ASTER image took about 15 secs. The gdal code which generated 
> the jpg from the geotif took about 14.5 mins. Interestingly before I ran 
> it on my laptop (much slower machine), before so having more cpu does 
> not seem to help.
> 
> Here is the command I ran
> 
> gdal_translate -of JPEG -co WORLDFILE=ON ml_VNIR_geo.tif ml.jpg

Simon,

A gdalinfo on the file reports:

Band 1 Block=5796x4888 Type=Byte, ColorInterp=Gray
Band 2 Block=5796x4888 Type=Byte, ColorInterp=Undefined
Band 3 Block=5796x4888 Type=Byte, ColorInterp=Undefined

The key item is the block size.  The files are treated as a single big
strip.

So what happens is that the entire TIFF file needs to be re-read to
satisfy each scanline of jpeg output since any single tile is larger
than the default cache size.  A worst possible case of cache thrashing.

But if I set the cache max converting the file only takes 2.8 seconds of
wall clock time on my machine:

warmerda at amd64[127]% time gdal_translate --config GDAL_CACHEMAX 200 -of JPEG 
ml_VNIR_geo.tif out.jpg
GDAL: GDALOpen(ml_VNIR_geo.tif) succeeds as GTiff.
Input file size is 5796, 4888
0...10...20...30...40...50...60...70...80...90...100 - done.
GDAL: GDALOpen(out.jpg) succeeds as JPEG.
GDAL: GDALClose(out.jpg)
GDAL: GDALClose(ml_VNIR_geo.tif)
GDAL: GDALDeregister_GTiff() called.
2.452u 0.316s 0:02.82 97.8%     0+0k 0+0io 0pf+0w

You mentioned earlier that you tried:

 >
 > gdal_translate --config GDAL_CACHEMAX 512
 >
 > and then ran the gdal_translate but it is still v. sloow.

Did you actually include the --config request in the run of gdal_translate
with the data?  The --config switch only sets the configuration option for
the period of the program running - it doesn't set something persistent.

Best regards,
-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | President OSGeo, http://osgeo.org