Limits
The values that MooseFS refuses to go past, and what decides the ceiling instead where MooseFS sets no maximum of its own.
A limit on this page is either a number fixed in the product or the width of the field that the product stores the value in. Configuration does not change either of them. Values that you set yourself, and the range that each of them accepts, are listed on the page of the configuration file that contains them.
Units
Sizes are counted in binary units throughout this documentation, the way MooseFS counts them. A kibibyte (KiB) is 1024 bytes, a mebibyte (MiB) is 1024 KiB, and the same factor continues through GiB, TiB, PiB, and EiB. The units printed on a disk vendor's box are decimal and count in powers of ten. A terabyte (TB) is therefore 1 000 000 000 000 bytes. A tebibyte (TiB) is 1 099 511 627 776 bytes, about 10 percent more. The difference grows with each step: about 13 percent at the pebibyte, about 15 percent at the exbibyte. Solutions and the FAQ count their round example figures in decimal units instead, the way hardware is sold.
Where a figure is a ceiling, the binary unit leaves you more room. A license written for 200 TiB of used raw space covers about 220 TB as a disk vendor counts it. The 16 EiB that a cluster can represent is about 18.4 EB.
Names and objects
| Limit | Value | What it covers |
|---|---|---|
| Length of a name | 255 bytes | One component of a path: a file, a directory, a link. Bytes, not characters, so a name in a multi-byte encoding reaches the limit on fewer letters. An empty name is refused as well. |
| Length of a symbolic link target | 4096 bytes | The text a symbolic link holds. The Master Server stores it and never resolves it. A mount counts the terminating byte into the limit, so a target created through a mount reaches 4095 bytes. |
| Length of a path sent in one request | 1024 bytes | Only the calls that carry a whole path: the subfolder a mount registers as its root, and the path the Master Server remembers for a file in the trash. The mfsio library counts the terminating byte in and takes 1023. A mount resolves a path one name at a time, so the name limit bounds the path instead. |
| Objects in one instance | 2 147 483 646 | Files, directories, links, and every other object draw on one identifier space. The cluster reports this figure as its total. The limit you reach first is usually the memory the Master Server needs to hold them. |
| Hard links to one object | 8 to 65 000 | The range MAX_ALLOWED_HARD_LINKS accepts. A directory cannot be hard-linked at all, whatever the setting. |
| Extended attributes on one object | Name 255 bytes, value 65 536 bytes, all names together 65 536 bytes | The last figure counts a terminating byte for every name. Once the names of an object fill that figure, a further attribute is refused. |
| Trash retention | 65 535 hours | About 2730 days. Retention is a whole number of hours. An export can narrow the range at both ends. |
Files and chunks
| Limit | Value | What it covers |
|---|---|---|
| Size of a chunk | 64 MiB | A file longer than that is split into several chunks. The last chunk of a file holds only as much as the file needs. |
| Size of a block | 64 KiB | The unit a chunk is divided into for checksumming, 1024 of them to a full chunk. |
| Chunks in one file | 2 147 483 648 | A write past the last of them is refused. |
| Size of one file | 128 PiB | 257 bytes: the chunk count of a file multiplied by the size of a chunk. A mount and the mfsio library refuse a write that would reach the figure, so the largest file they produce is one byte short of it. |
| Space one chunk occupies | At least 72 KiB, once per copy | A chunk carries 8 KiB of its own and stores whole blocks, so even a one-byte file is accounted at this figure on every Chunkserver holding it. The disk beneath usually charges less, because the chunk file is written sparsely. |
| Capacity of a cluster | 16 EiB | 264 bytes less one byte. The counters can represent this much, but the product does not enforce it as a ceiling. |
| Chunks in one instance | 72 057 594 037 927 935 | A chunk identifier is 56 bits wide. The rest of the field names the part when the chunk is stored in Erasure Coding. |
Redundancy and placement
| Limit | Value | What it covers |
|---|---|---|
| Copies in one storage state | 9 per location | How many copies one definition of a Storage Class may ask for. A class that defines several locations asks for up to that many in each of them separately. |
| Erasure Coding data parts | 4 or 8 | No other width is accepted. |
| Erasure Coding parity parts | 1 to 9 PRO (PRO Edition only), 1 in the Community Edition | A definition asking for more is rejected when the class is created or changed. |
| Storage Classes in one instance | 255 | Includes the classes a fresh instance is created with. |
| Length of a Storage Class name | 255 bytes | The name you give a class when you create it, and the name every tool takes to address it afterwards. An empty name is refused, and so is a name that starts or ends with a space or holds a control character. |
| Labels on one Chunkserver | 26 | One per letter of the alphabet, case insensitive. |
| Length of one label expression | 127 bytes | The expression as the tools encode it, not as you type it: one byte for a label, one for a negation, and one for a whole chain of & or |. A longer expression is refused when you define the class. |
| Label expressions in one definition | 9 for copies, 2 for Erasure Coding | A copy definition carries one expression per copy. An Erasure Coding definition carries one expression or two. One expression puts data and parity parts on the same servers. Two expressions put them on different servers. |
| Patterns in one instance | 1024 | A request to add another pattern is refused. |
| Length of a pattern | 255 bytes | The name to match, wildcards included. |
Cluster layout
| Limit | Value | What it covers |
|---|---|---|
| Locations in one cluster | 8 PRO (PRO Edition only) | Every cluster starts with one location, named default. A request to create a ninth is refused. The Community Edition has that one location and no way to add another. |
| Length of a location name | 32 bytes | A name of exactly 32 bytes is accepted. No terminating byte is stored with it. |
| Chunkservers connected at one time | 60 000 | The Master Server tracks connected Chunkservers in a table of that size. The servers the Master Server picks to hold a chunk come from that table. A Chunkserver takes a place in it for each address and port it registers under, so two Chunkservers on one machine take two places. A Chunkserver that finds the table full is refused at registration, and the Master Server logs can't accept chunkserver <address>:<port> (cstab full). |
| Chunkservers one cluster remembers | 60 000 | The Master Server keeps every registered Chunkserver on its list, whether or not it is connected. A Chunkserver that is not on the list yet is refused once the list holds 60 000 servers, and the Master Server logs csdb: server not found (<address>:<port>), no more available slots - rejecting. A server already on the list still reconnects. |
Where MooseFS sets no maximum
MooseFS has no fixed number for the items below, so the ceiling is set outside the product. That ceiling is set by the memory of the machine, by what the operating system gives the process, and by whatever else that machine is doing at the time. A cluster that handles one of these figures on one machine can fail on a different machine with the same figure.
| No fixed maximum on | What sets the ceiling instead |
|---|---|
| Entries in one directory | The memory the Master Server needs for them. Reading a large directory costs more round trips rather than failing. |
| Mounted clients and open sessions | Master Server memory and CPU, and the descriptors and sockets the operating system gives the process on both ends. |
| Files one session holds open | The descriptor limit of the machine the mount runs on. mfsmount raises its own on startup, and -o mfsrlimitnofile=N moves the figure it asks for. |
Entries in mfsexports.cfg | Master Server memory. Every entry is compared on a client registration, so a long file costs time there too. |
| Disks on one Chunkserver | The memory the Chunkserver needs for the chunks they hold, and the I/O the machine can carry. |
| Snapshots of one object | The object identifier space and the quotas in force, the same as for any other object. |
Throughput has no fixed maximum either. The FUSE layer on the client machine, not the cluster, limits how much one mount can transfer. Across the cluster, the network interfaces of the machines limit throughput (limits and scale).