Skip to main content

Start from the workload

The hardware that a MooseFS cluster needs depends on what the applications do to the cluster, not on the amount of data that they hold. Compare two clusters that store the same terabytes. The first cluster keeps a few large files that are read end to end. The second cluster keeps millions of small files that are created and deleted all day. The two clusters need different machines. Describe the workload in the terms that decide the design. Then price the machines.

The object count sizes the Master Server​

Count the files, directories, and links that the workload will hold. Also count the chunks that those objects add up to. The memory of one machine has to cover that count. The count changes on its own, independently of the bytes that those objects hold. What metadata holds describes which records multiply with what. Master Server RAM from the object count calculates a figure from a count.

Typical file size decides what storage costs​

Ask what a typical file in the workload looks like. Each chunk costs the cluster a fixed minimum, once per copy, whatever the size of the file in that chunk. For a workload of small files, the Chunkservers that it needs therefore have little to do with the bytes that the workload stores. Small files covers the figure for that minimum. Limits and scale compares that figure with other costs that depend on the number of chunks rather than on the volume of data.

Metadata operations and streaming land on different machines​

Compare how many operations per second the applications issue with how many bytes per second the applications move. One Master Server answers every metadata operation. That Master Server serves all of its clients from a single loop. One machine therefore handles the operation rate. The rate does not divide over the cores of that machine. Streaming is instead spread over the Chunkservers and their disks. Limits and scale describes both. What the FUSE layer costs covers what one operation costs on the client before that operation leaves the machine.

Mount count decides what one host can move​

Ask whether the applications reach the file system through one mount or through several. A single mount has a ceiling of its own. The ceiling is set on the client machine rather than by the cluster (the per-mount throughput ceiling). To go past that ceiling, split the work across mounts (multiple mounts and parallel access).

The redundancy choice sets the cost per terabyte​

Decide how much redundancy the data needs, and whether the data is written once or rewritten afterwards. These choices set how much raw space the stored data occupies (raw space and user space). They also set what the cluster has to move when a disk is lost. Choosing between copies and Erasure Coding compares the two formats. Erasure Coding: part layout describes what the cluster does with parts once a Storage Class requires them.

A multi-site cluster is sized per location​

Ask whether the cluster is in one place. A cluster that is spread across sites is divided into locations PRO (PRO Edition only). The cluster maintains redundancy separately in each location, so you count once per location instead of once for the cluster.