Skip to main content

Storage Class

A Storage Class is the named policy that decides in what form your data is stored and which Chunkservers should hold it.

What it is​

A Storage Class is a named definition kept by the Master Server and assigned to a file or a directory. You assign it with the mfssclass tools, one object at a time or a whole subtree at once. A new object otherwise takes the class of the directory it is created in. Changing the class of a directory applies to the objects already inside it only when you ask for that. A new instance comes with classes already defined. An administrator can add more.

Patterns are the third way to assign a class. A pattern matches an object's name and assigns the class for you at the moment when the object is created or renamed. The class from the pattern replaces the class that the parent directory would have given the object. A class that you set by hand afterwards stays until that object is renamed again.

A class has four placement rules, one for each storage state that a chunk can be in. Each definition says two things. The first is the redundancy format, either copies or Erasure Coding. The second is which Chunkservers may hold the data, written as a label expression over labels. The Redundancy Level is how many failures a chunk survives under a given definition.

Only the definition for the KEEP state has to be present. A state left undefined falls back to another state, so a class can be as short as a single rule.

Why it matters​

Redundancy in MooseFS is a property of a file. It is not a cluster-wide setting. The Storage Class records that property, so one instance can hold data of different value under different policies at the same time.

Because a class defines each storage state separately, one policy covers the whole life of the data. Chunks stay in copies while the file is in use and move to Erasure Coding once it has been left alone long enough. The file's name, permissions, and place in the file system do not change along the way. Because placement is written with labels, the same class also decides which failure the data survives. Copies spread over two server rooms survive the loss of a room. Copies that all happen to be stored in one room do not.

A class sets a target. The placement of a chunk does not always match it. When too few Chunkservers match a definition, or the matching Chunkservers are full, the Master Server keeps the data safe first. The Master Server writes the chunk on other servers and brings the chunk back in line with the class afterwards. A new chunk still needs one usable Chunkserver for each copy that the class requires. With fewer, the write fails. When the matching servers are merely overloaded, the Master Server waits for the load to pass instead. An overload that lasts therefore delays the creation of new chunks. A class can also be set to skip that wait, or to use only the matching servers. In that mode, the write fails when the matching servers cannot take every copy. The class also sets how strictly the rule applies. Label expressions and matching covers what each mode does.

One chunk can belong to several files. Those files can have different classes. The Master Server then applies a single class to that chunk. The Master Server picks that class by the priority and the redundancy that the candidate classes require.

How MooseFS implements it​

Class definitions are part of the metadata: the Master Server keeps the definitions with everything else that it holds and reloads them at startup. Every file and directory has the identifier of its class. The mfsscadmin tools define and change classes. The mfssclass tools assign them to objects. Both reach the Master Server through a mount.

When a client asks for a new chunk, the Master Server reads the CREATE definition of the file's class and picks Chunkservers that match that definition. For a chunk that already exists, the Master Server reads the definition of the chunk's current state and re-checks the chunk in the background. A chunk short of copies or parts is replicated. A chunk that holds more than the definition requires has the surplus deleted.

A change to a class takes effect on the chunks that already exist, and no file is rewritten. Each chunk refers to its class by number, so every check reads the definition that is current at that moment. After a definition is changed, the background check finds each chunk that no longer matches it. The Master Server then replicates that chunk, deletes its surplus, or moves it to Chunkservers that match the labels. After a file is given another class, the Master Server queues the chunks of that file for the check at once.

  • Storage states: the four phases that a chunk passes through, each with its own definition in a class.
  • Label: the building block of the expression that says where chunks may go.
  • Copy and Erasure Coding: the two redundancy formats that a definition chooses between.
  • Redundancy Level: how many failures a chunk survives under a given definition.
  • Chunk: the unit that a class is applied to.