[ka-Map-dev] modification to tile cache structure

[Daniel Morissette]

I'm +1 with the change.

With respect to the update_cache_dirs.sh script, if you are extra 
paranoid like I am then you can run it in "simulation mode" if you 
comment out the "mkdir" and "mv" lines inside the update_dir() function:

##    [ -d $DIRNAME/$TOP ] || mkdir -p $DIRNAME/$TOP

     echo "mv $DIRNAME/$BASENAME $DIRNAME/$TOP/$LEFT"
##    mv $DIRNAME/$BASENAME $DIRNAME/$TOP/$LEFT


Then the script will just echo the commands that it would run but won't 
execute them.

Daniel


Paul Spencer wrote:
> This comes with absolutely no warranty at all ... if it erases 
> everything on your root drive, and emails itself to everyone you know 
> ... you've been warned ;)
> 
> Seriously though, use with caution ... we wrote this up to fix up some 
> of our caches and it may not be exactly right for a stock ka-Map.
> 
> You would run it using:
> 
> ./update_cache_dirs.sh /tmp/kacache
> 
> Also note that I have not yet updated tile.php in cvs, but the change is 
> trivial if you want to try it:
> 
> 135c135
> < $szMetaTileId = "t".$metaTop."l".$metaLeft;
> ---
>  > $szMetaTileId = "t".$metaTop."/l".$metaLeft;
> 
> 
> Cheers
> 
> Paul
> 
> #! /bin/sh
> # save as update_cache_dirs.sh
> 
> ## This function echos its argument and exits
> fatal_error ()
> {
>     echo >&2
>     [ -n "$1" ] && echo "! $1" >&2
>     echo >&2
>     exit 1
> }
> 
> ## Perform update operation on a directory
> update_dir()
> {
>     DIRNAME=`dirname $1`
>     BASENAME=`basename $1`
> 
>     TOP=`echo "$BASENAME" | cut -dl -f1`
>     LEFT=l`echo "$BASENAME" | cut -dl -f2`
> 
>     #echo "dir=$DIRNAME   metatile=$BASENAME   TOP=$TOP  LEFT=$LEFT"
> 
>     [ -d $DIRNAME/$TOP ] || mkdir -p $DIRNAME/$TOP
> 
>     echo "mv $DIRNAME/$BASENAME $DIRNAME/$TOP/$LEFT"
>     mv $DIRNAME/$BASENAME $DIRNAME/$TOP/$LEFT
> }
> 
> CACHEDIR=$1
> 
> [ -z "$CACHEDIR" ] && fatal_error "cachedir arg missing. Usage: $0 
> <cachedir>"
> 
> ##
> ## If called with 1st arg = processdir then call update_dir() with arg2
> ##
> if [ $CACHEDIR = processdir ] && [ -n $2 ]; then
>     update_dir $2
>     exit 0
> fi
> 
> 
> ##
> ## Otherwise default behavior is to look for all metatile dirs and 
> process them
> ##
> find $CACHEDIR -name "t*l*" -mindepth 5 -maxdepth 5 -type d -exec $0 
> processdir {} \;
> 
> 
> 
> On 23-Oct-06, at 10:19 PM, Morgan Pyne wrote:
> 
>> Hi Paul (and others!)
>>
>> I would definitely welcome the change, as we have already reached the 
>> limits of an ext3 filesystem (~32768 subdirs) on a number of 
>> occasions. While we could solve this by moving to some other 
>> filesystem, I think that adding the extra depth is a good thing anyway 
>> -  ext3 is a common default filesystem for many Linux distros.
>>
>> I was wondering however if it might be possible to migrate a cache to 
>> the new structure rather than completely invalidate it?  I'm sure I'm 
>> not the only one who would prefer to be able to keep my cache if 
>> possible - precaching again would take considerable time. If the 
>> change only involves adding a slash between the top and left values in 
>> the path then I'm guessing the cache migration could be a one-liner 
>> for a good Perl monger.
>>
>> Another thing to consider might be to make this new cache structure 
>> optional, and default to the old one.
>>
>> Thanks (for _all_ the good work on ka-map!),
>> Morgan
>>
>>
>> On 24/10/2006, at 1:33 PM, Paul Spencer wrote:
>>
>>> Devs (and others)
>>>
>>> Before we release 1.0, I'd like to change the tile caching structure 
>>> to include an extra level of directories by inserting a / between the 
>>> top and left values in the meta-tile directory.
>>>
>>> The change is trivial and safe, but it invalidates an existing cache 
>>> (well, makes it useless)
>>>
>>> What do you think of this change?
>>>
>>> Cheers
>>>
>>> Paul
>>> +-----------------------------------------------------------------+
>>> |Paul Spencer                          pspencer at dmsolutions.ca    |
>>> +-----------------------------------------------------------------+
>>> |Chief Technology Officer                                         |
>>> |DM Solutions Group Inc                http://www.dmsolutions.ca/ |
>>> +-----------------------------------------------------------------+
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> ka-Map-dev mailing list
>>> ka-Map-dev at lists.maptools.org
>>> http://lists.maptools.org/mailman/listinfo/ka-map-dev
>>
>> _______________________________________________
>> ka-Map-dev mailing list
>> ka-Map-dev at lists.maptools.org
>> http://lists.maptools.org/mailman/listinfo/ka-map-dev
> 
> +-----------------------------------------------------------------+
> |Paul Spencer                          pspencer at dmsolutions.ca    |
> +-----------------------------------------------------------------+
> |Chief Technology Officer                                         |
> |DM Solutions Group Inc                http://www.dmsolutions.ca/ |
> +-----------------------------------------------------------------+
> 
> 
> 
> 
> _______________________________________________
> ka-Map-dev mailing list
> ka-Map-dev at lists.maptools.org
> http://lists.maptools.org/mailman/listinfo/ka-map-dev


-- 
Daniel Morissette
http://www.mapgears.com/