Label expressions and matching
Label expressions record where a chunk's copies and parts may be stored. The Master Server decides every placement by matching those expressions against the Chunkservers. This page describes that decision: what an expression is to the Master Server, how the matching assigns servers, and what each labels mode does when too few servers match.
An expression is one storage state's placement rule
A label expression turns the labels that you attach to Chunkservers into a test that a server passes or fails. Each storage state of a Storage Class has its own expression. Which state is in force for a given chunk is decided before any matching starts.
For copies, the expression is a list of subexpressions: one per copy, and at most nine of them. A subexpression names a label, combines labels with the operators and, or, and not, or accepts any server as *. A leading number means that many identical copies. For Erasure Coding, the expression names the format instead, for example @8+2 for 8 data parts and 2 parity parts. An Erasure Coding expression has at most two subexpressions. With one subexpression, the expression places all the parts. With two subexpressions, it places data parts and parity parts separately. With none, it accepts any server. The mfsscadmin(1) manual page describes the full syntax. The size limits are listed with the other limits.
The Master Server stores each subexpression in compiled form, with one byte per label and per operator. A subexpression whose compiled form is longer than 127 bytes is refused when the class is defined. The limit applies to that encoding, not to the number of labels. A long chain of the same operator is compiled into a single byte. Testing a server against a compiled expression costs almost nothing, because a Chunkserver's labels arrive as a bit mask when the Chunkserver registers. The compiled expression is evaluated directly against that mask.
Matching is an assignment problem, run after filtering
Matching is the last step of a longer selection. The selection first decides which servers are candidates at all. The page on allocation and space selection covers that step. That step checks room for another chunk scaled by the writes that the server is carrying, overload, and internal rebalancing. The step also includes the arithmetic that returns busy servers to the candidates when too many were set aside. Matching receives the remaining list, which is already ordered.
Against that list, the expression is an assignment problem. Each subexpression needs a server of its own. No server is assigned to two subexpressions. The Master Server solves that problem exactly, as a maximum matching, rather than first come, first served, because a greedy pass can waste a server. Suppose the expression is A,AB and one server has both A and B. If that server is given to the first copy, the second copy cannot be placed. The exact assignment gives that server to the second copy and finds an A server for the first copy. A distinguish extension is /[IP], /[RACK], or a list of labels after /. When the expression has such an extension, the candidates that share an address, a rack, or the same combination of labels from the list are grouped. A candidate that has none of the listed labels forms a group of its own. Each group supplies at most one server, so the copies are separated along the boundary that you named.
The same matching runs again whenever the Master Server reviews an existing chunk. A copy stored on a server that the expression would not choose is therefore recognized as misplaced. Moving that copy is the lowest-priority repair. It waits behind every chunk that is missing a copy or a part. Replication priorities describes that order.
The labels mode decides what a shortage costs
When every subexpression finds its server, the labels modes make no difference. They answer one question: what to do when the assignment is incomplete because too few candidates match. There are three modes: LOOSE, STANDARD, and STRICT. The tools print STANDARD as STD.
For a new chunk, STANDARD gives every unmatched subexpression a non-matching server, so the chunk is written complete. The misplaced copies are corrected later. There is one exception. If the servers dropped from the candidates for being overloaded would have completed the assignment, the Master Server tells the client to retry later instead. The write waits until the load drops, instead of accepting a placement that a load spike forced. LOOSE uses the non-matching servers without that wait. STRICT never uses them. Under STRICT, a new chunk is created only when the matching servers can hold every copy that the class requires. STRICT waits too when the servers dropped for overload would have made up the difference. Under every mode, a location with fewer usable servers than copies gets no copy of the new chunk, and the write fails when no location can take all of its copies. Up to 5.1.0, the chunk was created on the servers that could be assigned, and a write failed only when not a single server could take the chunk.
A missing copy of an existing chunk is repaired under the same modes. When the only matching servers are busy, STANDARD and STRICT wait for them. LOOSE replicates to any server at once. When no matching server has room, STANDARD and LOOSE replicate to any server. STRICT does not replicate at all. The copy stays missing until a matching server has space again.
An empty selection gives one of three answers
When the servers for a new chunk cannot be chosen, the answer that the client gets depends on what the Master Server ruled out. If candidates dropped for load would have been enough, or a Chunkserver is still registering its chunks, the answer is to try again later. The write waits without failing. If a known Chunkserver is disconnected, the answer is that Chunkservers are missing. If every known server is connected and the assignment still falls short of the copies that the class requires (under STRICT, too few matching servers could take the chunk), the answer is that there is no space. The page on allocation and space selection covers how the mount retries on each answer, and when a write finally fails with ENOSPC.
For Erasure Coding the modes count servers
A chunk is not created in parts. The chunk is converted to Erasure Coding later. Erasure Coding layout describes that conversion. The labels mode affects three decisions around that conversion. Each decision depends on which servers are counted. For a format with N data and X parity parts:
| Decision | LOOSE | STANDARD | STRICT |
|---|---|---|---|
Convert copies to parts, taken when N+2X servers are counted | Every server open to replication | Matching servers only | Matching servers only |
Keep a chunk in parts, given up below N+X servers | Every available server | Every available server | Matching servers only |
| Place or restore one part | Any server when no matching server can be used | Matching servers while they suffice, any server once they don't | Matching servers only |
In practice, STANDARD and STRICT differ in the last two rows. The middle row shows the difference that matters. STRICT reverts a chunk to copies as soon as the matching servers alone can no longer hold its parts. STANDARD keeps the parts on whatever servers are available and corrects the placement later. An expression with two subexpressions is also counted against each subexpression. There must therefore be enough servers for the data parts and enough for the parity parts, not merely enough in total. To convert, the data subexpression must select N+X servers and the parity subexpression must select 2X. To keep a chunk in parts under STRICT, the counts are N and X. A server that matches both subexpressions counts for either side. An Erasure Coding definition that has no labels runs as LOOSE, whatever mode the class declares. The modes therefore cost nothing until the parts are actually constrained.
A definition can override the mode that the class sets
Each state definition has its own mode field with four values: the three modes and a fourth meaning "as the class". A definition starts with the fourth value, so the class-wide mode governs everywhere in the class. You choose that mode with mfsscadmin -m. When you do not choose it, the mode is STANDARD. Appending :l, :d, or :s (LOOSE, STANDARD, STRICT) to one definition's expression overrides the mode for that definition alone. A class can therefore place its KEEP copies strictly and still let its ARCHIVE parts fall back to other servers.
Labels also affect reads, but in a different way. The client, not the Master Server, sorts the servers that the client reads from. Preferred labels describes that mechanism.