[FWTools] Compressing geoid model?

[Frank Warmerdam]

On 11-06-07 09:36 AM, Mikael Rittri wrote:
> However, I have tried these commands:
>
>     gdal_translate -ot UInt16 -co "COMPRESS=NONE" egm2008-5.pgm egm2008-5-uint16-nocompression.tif
>
>     gdal_translate -ot UInt16 -co "COMPRESS=LZW" egm2008-5.pgm egm2008-5-uint16-lzw.tif
>
>     gdal_translate -ot UInt16 -co "COMPRESS=PACKBITS" egm2008-5.pgm egm2008-5-uint16-packbits.tif
...
> So, PACKBITS makes the result 0.7 percent larger than NONE,
> which is perhaps not a surprise, since run-length encoding
> should not be suitable for this kind of data.
> However, LZW makes the result 29.6 percent larger than NONE,
> which surprised me.  I would have thought the the geoid data
> should be fairly easy to compress (although I know almost
> nothing about how LZW works).
>
> Am I missing something obvious? Is there some other
> (GDAL-readable) compressed format that would be better
> for this kind of data?

Mikael,

I believe the problem is that each block written includes an LZW dictionary
table which can be a fair amount of overhead.  In the above case the image
is written out with one scanline per block.  Also, by default the literal
bytes are written.  Looking at the file it seems there are no obvious
simple patterns of values or large constant areas, so the LZW compressor
does poorly.  Using a horizontal differencing predictor (basically
compressing the changes from pixel to pixel) helped a great deal.

I found I got significantly better results (around 10MB) with the command:

gdal_translate egm96-5.pgm -co COMPRESS=LZW out.tif \
    -co PREDICTOR=2 -co TILED=YES

Best regards,
-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent