Small files

MooseFS splits file data into chunks of at most 64 MiB. A Chunkserver writes a chunk in blocks of 64 KiB. Every chunk file also has 8 KiB of metadata of its own. This metadata is a 4 KiB header followed by a 4 KiB table of block checksums. The header holds the signature, the chunk ID, and the chunk version.

A one-byte file therefore occupies one 64 KiB block plus those 8 KiB, which is 72 KiB in all. The file occupies that much once per copy. With three copies, the same one-byte file accounts for 216 KiB. Under Erasure Coding, each part is a chunk file of its own with its own 8 KiB of metadata. The parts of a small chunk therefore consist mostly of headers rather than of data.

This is the size that the cluster accounts for. mfsdirinfo reports that size. The Master Server counts the same size toward quotas. The underlying file system on the Chunkserver usually uses less space, because a chunk file is written sparsely. Long runs of zeros in a chunk are left as holes.

HDD_SPARSIFY_ON_WRITE is enabled by default. With this option enabled, a Chunkserver drops the zeros at the start and at the end of a block. This applies to every block that the Chunkserver writes at or past the current end of the chunk file. The Chunkserver stores only the range between those zeros. The Chunkserver rounds that range outwards to a multiple of 512 B. A block that is zero from end to end is not written at all: it stays a hole. Reading such a block still returns zeros. Zeros that are between the first and the last non-zero byte of a block are stored like any other data. A write below the current end of the chunk file is stored as sent, zeros included, even in a block that was never written before. The chunk file therefore keeps its full length but occupies less of the disk than that length suggests. du on a Chunkserver reports a smaller number than ls by exactly that much.

Chunks written in the older on-disk format have 5 KiB of metadata instead of 8 KiB. This metadata is a 1 KiB header and the same 4 KiB checksum table. Both formats are read. Only the newer format is written. An old chunk is converted when internal rebalancing rewrites it.