Storage Class evaluation
A Storage Class has a separate placement definition for each storage state that a chunk can be in. This page describes how the Master Server evaluates the class for one chunk. The page covers which definition governs the chunk at a given moment, when the archive rule moves the chunk between states, and in what order the governing definition becomes a placement.
The chunk's flags pick the governing definition
Two flags on the chunk decide which definition applies: the archive flag and the trash flag. The page on storage states covers what sets and clears the flags, and what the four states mean. The Master Server resolves the flags in a fixed order. The trash flag selects the TRASH definition first. The archive flag then selects the ARCHIVE definition. A chunk whose flags select nothing is governed by KEEP. The order skips any state that the class does not define. An undefined state does not block the order. Therefore, when a chunk has the trash flag set and its class has no TRASH definition, ARCHIVE governs the chunk if its archive flag is also set. Otherwise KEEP governs it. CREATE never governs an existing chunk. The CREATE definition is read once, at the moment the chunk is created. When the class does not define it, the definition falls back to KEEP. Allocation and space selection covers what is done with that definition.
Nothing moves at the moment a flag changes or a class is redefined. A changed flag adds the chunk to the repair queues at once. Redefining a class changes no chunk. Instead, the chunk loop finds the consequences of the redefinition. Each pass compares what the Chunkservers hold for a chunk against the definition that governs the chunk now, and adds the differences to the same queues. The queued work is a replication, a conversion between formats, or a deletion of the surplus. Either way, that work runs in the order that replication priorities describes. A redefinition therefore takes effect when the loop reaches the chunk. A change of state takes effect as soon as the queues drain.
The file loop sets the archive flag
The archive rule of a class has three parts: a set of mode flags, a period counted in hours, and a minimum file length. The file loop applies the archive rule. For each file, the Master Server reduces the rule to one result. The Master Server then visits the file's chunks and sets the archive flag where the result says so. The loop only ever sets the flag. The paragraphs below list what clears it. Two checks run before any timestamp is read. A file shorter than the minimum length is never archived, whatever the rest of the rule says. A deleted file is never archived either, whether it is in the trash or is only held open by a client. The loop also flags a chunk only when the file being examined has the class that governs that chunk. One chunk can belong to files with different classes. The page on class promises and chunks covers which single class governs such a chunk.
Three of the mode flags name timestamps. These flags combine rather than exclude one another. C measures the file's ctime, M its mtime, and A its atime. When several are set, the latest of the selected timestamps counts, so the file is archived only after the period has passed since all of them. Unless the class says otherwise, the rule measures ctime with a period of 24 hours. A class that measures ctime and mtime with a period of 48 hours archives a file two days after the last change to its data or metadata. Reading the file does not reset that count. Writing to the file does. A period of zero does not mean at once. It means never, so such a rule sets no flags at all. The period is a minimum, not a schedule. A chunk is archived no earlier than the period says. The chunk can be archived later, because the flag is set when the file loop reaches the file.
R, the reversible flag, modifies the timestamp flags and requires at least one of them. With R, an operation that changes one of the measured timestamps clears the archive flag of every chunk of the file. The chunks return to KEEP. The check runs during the operation itself, not in the loop. An operation that changes only a timestamp that the rule does not measure clears nothing. Since 5.2.0, the minimum length is checked on its own. With R, an operation that shortens a file below the class's minimum length returns every chunk of the file to KEEP, whichever timestamps the rule measures. Without R, a set flag stays set, however the timestamps change afterwards.
F, fast mode, replaces the timestamp rule: every chunk of the file is flagged as soon as the file loop reaches the file. The period is ignored.
P, per-chunk mode, is the only rule that evaluates each chunk separately. Each chunk is compared against the time when that chunk was last written, not against the file's timestamps. A file that keeps growing therefore has archived chunks at its start. The end of the file stays in KEEP. The period still applies to each chunk. A period of zero still means never. F and P are each used alone: a class cannot combine either of them with any other flag. R without a timestamp flag is not accepted.
Archiving is not one-way. Writing to a chunk whose governing definition stores it in Erasure Coding clears both flags first. The chunk returns to KEEP and to copies. A chunk archived into copies keeps its archive flag when it is written to. The flag is removed only by R or by clearing it by hand. The rule got its current form during the MooseFS 4 line. In MooseFS 3, the period counted days and always ran from ctime. In MooseFS 3, the flag never cleared on its own.
| Mode flag | What it does | What it may combine with |
|---|---|---|
C | Measures the file's ctime. | M, A, R |
M | Measures the file's mtime. | C, A, R |
A | Measures the file's atime. | C, M, R |
R | Clears the flag again when a measured timestamp moves, at the operation rather than in the loop. | C, M, A. At least one of them is required. |
F | Flags every chunk as soon as the file loop reaches the file, and disregards the period. | Nothing. |
P | Judges every chunk by the time that chunk was last written, not by the file's timestamps. | Nothing. |
When a class sets none of this, the rule measures ctime with a period of 24 hours. A period of zero means never. A file shorter than the rule's minimum length is never archived at all.
The definition becomes a placement late
The governing definition names a redundancy format and a label expression. The format is copies or Erasure Coding parts. The label expression says which Chunkservers may hold the copies or parts. The Master Server turns that definition into servers in a fixed order. The class is read at the end of that order. The Master Server first filters the candidates by space, load, and the other checks described in allocation and space selection. This step does not read the class at all. Only the remaining candidates are matched against the label expression. Label expressions and matching covers how the expression is evaluated and what each labels mode does when too few servers match it.
When too few servers match, the labels mode decides. Since 5.2.0, STRICT refuses a new chunk when the matching servers cannot hold every copy that the class requires, and the client gets ENOSPC. Up to 5.1.0, STRICT refused only when no server matched at all. Under STANDARD and LOOSE the data is not refused: the chunk is written to, or kept on, servers that the expression would not have chosen. The mismatch remains. Repairing it is the lowest-priority work that the Master Server schedules. A chunk stored on the wrong servers waits behind every chunk that is missing a copy or a part. The page on replication priorities describes that order.
In an instance divided into locations PRO (PRO Edition only), every state definition exists once per location. The evaluation above then runs once per location, against that location's part of the definition.
Erasure Coding layout covers how Erasure Coding parts are laid out under an ARCHIVE or TRASH definition. The page on how policies are applied covers which class an object has in the first place, and how patterns and inheritance assign the class.