Chunkserver memory

A Chunkserver holds one entry in RAM for every chunk that it stores. An Erasure Coding part counts as a chunk of its own. The entry records the amount of data in a chunk only as a number of blocks. The entry therefore has the same size whatever the chunk contains. On a 64-bit build, the entry is 144 bytes. The allocator that assigns the entry and the per-disk index that points to the entry each add a few bytes on top of that.

Chunkserver memory therefore grows with the number of chunks, not with the number of terabytes. A disk filled with full 64 MiB chunks costs a fraction of the memory that the same disk costs with chunks of a few kilobytes each. Many small files are expensive for this reason too (see Small files).

The Chunkserver also has a fixed cost. The Chunkserver allocates the whole hash table that indexes the chunks when it starts. The table has 16 777 216 entries, which is 128 MiB on a 64-bit build. An empty Chunkserver therefore uses the same memory for that table as a full Chunkserver.

A chunk that is in use costs more than its entry. The Chunkserver loads the chunk's 4 KiB table of block checksums into RAM. The Chunkserver frees that table 100 seconds after the last operation on the chunk. For 10 seconds, the Chunkserver also keeps the last 64 KiB block that it accessed. This part of the memory grows with the number of chunks in use, not with the number of chunks stored.

The Chunkserver has no limit on the number of chunks that it accepts. Requirements for Chunkservers lists the figure to use when you size the memory.