Master Servers
The machine that runs the Master Server holds the metadata of the whole instance, so buy that machine for stability before anything else. Choose a server with a redundant power supply, disks in RAID 1, RAID 5, or RAID 10, and ECC memory. The operating system must be POSIX-compliant. A 64-bit system is strongly recommended.
A high clock speed matters more than a high core count
The Master Server answers all of its clients from a single loop in one thread. The operation rate therefore depends on one core and does not divide over the remaining cores. Pick a modern CPU with a high clock speed rather than a CPU with many cores. Clock speed matters most in deployments that manage a large number of small files. The periodic save of the metadata image runs in a separate process and does not compete with that loop. For the highest single-thread performance, disable hyper-threading and configure the machine for maximum performance. Turn off power-saving features. To compare processors, use the single-thread rating at cpubenchmark.net.
Plan 300–350 MiB of RAM per million objects
The Master Server keeps the whole metadata set in RAM. This figure is therefore measured against the count of objects that the instance will hold (files, directories, symbolic links, sockets, pipes). For planning, budget 300–350 MiB per million objects. An instance that is expected to hold a hundred million objects needs about 29–34 GiB for metadata alone. The operating system and every other process on the machine come on top of that. The figure comes from running instances and includes the cost of their chunks. An instance of large files holds more chunks per object and costs more.
Size the machine against that figure. The figure includes a margin and does not measure the size of the records. Master Server RAM from the object count calculates the same requirement record by record. That page arrives at a lower figure for a given count, because the page counts metadata records and nothing else. Records that the applications add, extended attributes above all, are not counted in either figure and raise the requirement further. The periodic save of the metadata image needs headroom while the save runs. That headroom adds to the requirement too. No arithmetic turns that headroom into a number.
Software corrects no memory error, so use ECC
The metadata structure is held in memory in its entirety and is not re-read from disk in normal operation. The cluster answers with whatever memory holds. The same content goes into the next saved copy of the metadata. The checksum stored alongside that copy is computed from the same bytes, so the checksum matches whatever memory held at the time. A single flipped bit is enough to change the Storage Class of a file, the permissions of the file, or the name under which the file appears in its directory. In normal operation, nothing reads a correct value back that would contradict the change.
Run the Master Server on ECC memory. Nothing in the software can catch a memory error on this machine: the saved copy certifies whatever memory held, so the correction has to happen in the hardware layer. ECC memory is that layer. It protects the metadata that every file in the cluster depends on.
The disk holds metadata images and changelogs
Two kinds of file in the data directory of the Master Server (usually /var/lib/mfs) set how much space the directory needs. These are the periodic images of the metadata and the changelogs recorded between the images. Files in a data directory lists everything else in that directory. None of these other files is large. The number of objects and chunks sets the size of an image, and extended attributes add to it. The number of operations performed on them sets how fast the changelogs grow.
Estimate the minimum space with:
SPACE = IMAGE * (BACK_META_KEEP_PREVIOUS + 2) + CHANGELOG * (BACK_LOGS + 1) [GiB]
where IMAGE is the size of one metadata image in GiB and CHANGELOG is the size of one changelog file in GiB. Neither value is a setting. An image takes about 350 bytes per object on an instance that uses few extended attributes, and close to twice that on an instance that uses them heavily. One changelog file is one hour of operations: about 1 MiB under a light operation load, and up to about 160 MiB under a heavy one. mfsmaster.cfg has no option for the size of that file. BACK_META_KEEP_PREVIOUS and BACK_LOGS are settings, both in mfsmaster.cfg. They set the number of previous metadata images kept (default 1) and the number of changelog files kept (default 50). An instance of 100 million objects with few extended attributes and a light operation load comes to 33 * (1 + 2) + 0.001 * (50 + 1) = 99 GiB. The same object count with heavy use of extended attributes and a heavy operation load comes to 56 * (1 + 2) + 0.16 * (50 + 1) = 176 GiB.
BACK_META_KEEP_PREVIOUS is set per module. Each module reads the value from its own configuration file. The defaults differ. The value above is the Master Server's. A Metalogger keeps the same kind of files under its own setting in mfsmetalogger.cfg. The default of that setting is higher, so run the estimate again for the Metalogger machine with the value that the Metalogger uses. IMAGE stays the value for the Master Server, because the images on that disk are the images of the Master Server.
Treat the result as an estimate. Two instances with similar object counts can differ by an order of magnitude in image size and by several orders in changelog size, depending on extended attributes and on the operation load. Allocate more space than the calculated minimum. Changelog files grow past their typical size in periods of increased activity. The extra space covers that growth.
In PRO, every Master Server is the same class
In a PRO instance, several machines have the Master Server role. Every one of them holds the whole metadata set and needs the same memory PRO (PRO Edition only). The same rule applies to the rest of the machine. Buy the same class of machine for each Master Server, because the Leader election decides which machine answers clients at a given moment. A Metalogger copies the metadata to a machine of its own and is sized separately.