[ka-Map-users] Best Linux file system type for ka-map cache?

[Attila Csipa]

On Tuesday 23 October 2007 16:44:38 Stephen Woodbridge wrote:
> Our research has shown Reiserfs to be the fastest. All Linux FS suffer
> from the block allocation issues. That is the fact the files are
> allocated based based on a 4k block, but most water tiles are only 130
> bytes but use up 4k of disk. We have found some SAN devices that do
> their own internal space allocation that does not suffer from this
> effect, but I believe that is specific to the internal SAN implementations.

I don't know if it has been thought of before, but there could be a special 
small-tile cache for this purpose that would benefit all platforms. You could 
cache for example repeating tiles (all black or all blue most often). The 
easy way would be having softlinks to these (or some multiplatform 
equivalent), or better yet having a master cache file that contains the list 
of these tiles (so you do not have to actually have the files in cache dirs 
in the first place). Identity of small repeat tiles could be checked via some 
hash function (MD5, CRC) which is not all that much time (filesystem load is 
far more dangerous). The advantage is that even though reiser is fastER for 
massive small file filesystems, the fastest would be not to have these files 
separated/repeated at all.