Skip to main content

Storage states: CREATE, KEEP, ARCHIVE, TRASH

A storage state is the phase of a chunk's life. The state selects which definition of the file's Storage Class applies to that chunk.

What it is​

Every chunk is in one of four states. The Storage Class of the chunk's file has a separate definition for each state.

CREATE is the state of a new chunk while it is written for the first time. The CREATE definition decides where the chunk is put at the moment when it is created. KEEP is the state of a normal, active chunk. Every other state falls back to the definition of this state in the end. ARCHIVE is the state that a chunk reaches once the period that its class sets has passed. The class also sets the moment from which that period is counted. By default, this is the moment when the file's data or metadata last changed. TRASH is the state of a chunk whose file has been deleted and is waiting in the trash.

The state belongs to the chunk, not to the file. A large file that is still being appended to can hold archived chunks and an active chunk at the same time.

The name ARCHIVE is a little misleading. It is a storage state, not a statement that the file has been archived. An archived file keeps its name, its place, and its permissions. Such a file is read and written like any other file. Nothing about it is slower, locked, or put away. The state exists so that a class can store colder data differently. ARCHIVE and TRASH are also the only states in which Erasure Coding is allowed.

Storage states of a chunk. The phase that a chunk reaches on its own picks the Storage Class definition.Storage states of a chunk. The phase that a chunk reaches on its own picks the Storage Class definition.
Notes:
  1. A background loop sets the archive flag. The command that sets the flag by hand is mfssetarchive.
  2. Only if the class clears the flag, or the chunk is written to under an Erasure Coding definition. The command that clears the flag by hand is mfsclrarchive.

Erasure Coding is permitted only in ARCHIVE and TRASH.

A chunk shared by several files never takes the trash flag.

The state belongs to the chunk, not to the file. One large file can hold active and archived chunks at the same time.

Fig.

Storage states of a chunk. The phase that a chunk reaches on its own picks the Storage Class definition. ARCHIVE is a storage state, not a statement that the file has been archived: such a file is read and written like any other.

Why it matters​

Moving data between states as it ages is called tiering in the literature. In MooseFS the storage states do it. One class covers the whole life of the data. The transitions happen on their own, so nothing has to be moved, renamed, or copied by hand. The user sees no change.

The states also keep Erasure Coding out of the write path. A chunk that is being created or actively used is stored in copies, because CREATE and KEEP accept only copies. Only data that has stopped changing moves to the cheaper format.

How MooseFS implements it​

The Master Server keeps two flags on each chunk, one for archive and one for trash. Those two flags say which definition governs the chunk. CREATE is not a flag but the definition that the Master Server uses at the moment when it creates the chunk. A chunk never returns to CREATE. Storage class evaluation covers the order in which the two flags are read, and what a class falls back to when it leaves a state undefined.

The archive flag is set by the background loop that goes through the file structure. The class decides what that loop measures: which of the file's timestamps the period is counted from, and whether a change to the file clears the flag again. The class also decides whether files below a size are archived at all. The flag can also be set and cleared by hand, with mfssetarchive and mfsclrarchive. mfschkarchive prints the current value of the flag.

The period that a class sets is a minimum, not a schedule. The loop moves through the file structure at its own pace and stops altogether while Chunkservers are reporting their chunks. A chunk therefore changes state no earlier than the period says, and can change much later. The delay depends on the size of the file system and on what else the Master Server has to do.

The trash flag is set for chunks of files that are in the trash. The flag is cleared when a file is taken back out of the trash. Deleting the file is not enough to set the flag. The flag is set only when the trash sweep reaches the file. At that point, the content of the file must have been unchanged for five minutes. Its retention must also be at least the minimum that its class sets for this flag. Trash retention covers all of these conditions. A file with a short retention can therefore be destroyed before its chunks ever get the flag. A chunk that belongs to more than one file does not take the flag at all.

A chunk stored in Erasure Coding cannot be written to. Before the Master Server lets a client modify such a chunk, the Master Server clears both flags. Clearing the flags puts the chunk back under the KEEP definition and returns the chunk to copies.

  • Storage Class: the policy that has a definition for each of these states.
  • Erasure Coding: the format that only ARCHIVE and TRASH may require.
  • Copy: the format that CREATE and KEEP use.
  • Chunk: the thing that is in a state.