[ka-Map-users] Best Linux file system type for ka-map cache?

[Stephen Woodbridge]

Attila Csipa wrote:
> On Tuesday 23 October 2007 16:44:38 Stephen Woodbridge wrote:
>> Our research has shown Reiserfs to be the fastest. All Linux FS suffer
>> from the block allocation issues. That is the fact the files are
>> allocated based based on a 4k block, but most water tiles are only 130
>> bytes but use up 4k of disk. We have found some SAN devices that do
>> their own internal space allocation that does not suffer from this
>> effect, but I believe that is specific to the internal SAN implementations.
> 
> I don't know if it has been thought of before, but there could be a special 
> small-tile cache for this purpose that would benefit all platforms. You could 
> cache for example repeating tiles (all black or all blue most often). The 
> easy way would be having softlinks to these (or some multiplatform 
> equivalent), or better yet having a master cache file that contains the list 
> of these tiles (so you do not have to actually have the files in cache dirs 
> in the first place). Identity of small repeat tiles could be checked via some 
> hash function (MD5, CRC) which is not all that much time (filesystem load is 
> far more dangerous). The advantage is that even though reiser is fastER for 
> massive small file filesystems, the fastest would be not to have these files 
> separated/repeated at all.

One of the easiest and fastest ways to do this is to just delete the 
files if you have a fully populated tile cache, and then create a 404 
handler that returns a standard blue/black/whatever tile if the 
requested tile is not found. This is easy to do with apache2.

-Steve