Skip to main content

What the class promises and what the chunks do

A Storage Class is a promise made about a file: this is how its data is to be kept. Everything that keeps the promise happens to chunks, one by one, each on its own schedule. This page separates the two: which facts belong to the file and which belong to each chunk. The page also describes what a change of class does at once and what that change leaves for later. The rest of this chapter describes a file on that basis.

The promise is on the file, the facts on each chunk​

About storage, a file's inode records the identifier of its Storage Class. That identifier is the whole promise. Every fact about how the data is actually held is recorded on each chunk separately:

  • the archive and trash flags that select the chunk's storage state.
  • the format the chunk is held in, copies or Erasure Coding (EC) parts.
  • the actual set of copies and parts on the Chunkservers, each with its own version.
  • the class that governs the chunk. This class is recorded on the chunk itself, because a chunk can belong to more than one file. The class of such a chunk cannot be read from a single inode.

A file is therefore never "in EC" and never archived as a whole. The file's chunks are, each according to its own flags. A directory has a class too, but only to pass the class on: the directory has no chunks. An object created in a directory gets that directory's class. How an object gets a class at all is covered in how policies are applied.

One file's chunks are in different states​

Because the flags are per chunk, a growing file is a mixture by design. The chunk being appended to was placed moments ago under the CREATE definition and is in KEEP. The chunks written long ago are in ARCHIVE, possibly as Erasure Coding parts. Each chunk is evaluated on its own flags against its own definition of the class. Storage Class evaluation covers which definition governs one chunk at one moment and what moves a chunk between states.

The promise is kept later, by the loops​

Nothing brings a chunk in line with its class at the moment the class starts to govern that chunk. The change files the chunk in the repair queues. Where nothing files the chunk, the chunk loop (the Master Server's periodic pass over every chunk) finds the difference and files the chunk too. The resulting work is done in the order that replication priorities defines. The distance between promise and fulfillment is a property of the mechanism, not a fault. That distance is the chunk's place in the queues. At worst, the distance is a loop pass.

A shared chunk has one governing class​

A snapshot makes one chunk belong to several files at once. Those files can have different classes. How a chunk comes to be shared is covered in snapshot copy-on-write. The chunk is stored once, so a single class has to govern it. The Master Server picks that class by priority from the classes of all the files that the chunk belongs to. The Master Server re-derives the choice when that set of files changes. The selection and its tie-breakers are covered in class priority and export groups.

A class change takes effect in metadata now, in data later​

Setting a file to another class takes effect in metadata immediately. The inode records the new identifier. Every chunk of the file re-derives which class governs it and is re-checked against the repair queues. The file's ctime changes. No data moves at that moment. The new class implies conversions: a different number of copies, a different format, different Chunkservers. These conversions are ordered as the repair queues drain, behind everything more urgent that the queues hold. A file switched to a class that archives into Erasure Coding is still held in copies for a different reason. The file's chunks are not archived yet. The file loop sets the archive flag. That intermediate picture is correct, not stale.

The gap between promise and state is counted​

At any moment some chunks hold less than their class requires. Some hold more. The cluster grades every chunk against its class as undergoal, overgoal, or one of the other grades. The cluster then reports all chunks as a matrix of the redundancy that each chunk should have against the redundancy that the chunk does have. The grades, the matrix, and where to read them are covered in chunk versions and states.