Skip to main content

Copy (replica)

A copy is a full duplicate of a chunk, stored on a different Chunkserver.

What it is​

Copies are one of the two redundancy formats of MooseFS. The other format is Erasure Coding. A copy holds the whole chunk, so one Chunkserver can serve it without any other Chunkserver taking part. Every copy of a chunk is stored on a different Chunkserver. No Chunkserver holds the same chunk twice.

The number of copies that a chunk has is a property of the Storage Class assigned to its file, not of the copy. The class defines a count and a placement rule for each storage state separately, one rule per copy. Redundancy Level covers how many failures a given number of copies survives. The count of copies that a class requires is known in the literature as the replication factor.

Why it matters​

Copies are the only format that a chunk can be created and kept in. A class may require Erasure Coding in the ARCHIVE and TRASH storage states only. Data is therefore always written as copies and stays in copies until the class moves the chunk to another state.

The two formats cost differently. Every copy takes the full size of the chunk, so copies are the more expensive format in space. In exchange, a read is answered by one of the Chunkservers that hold the chunk. A lost copy is restored by transferring an existing copy, with no data from other chunks and no computation involved.

How MooseFS implements it​

For the Master Server, a copy is a Chunkserver that reports a chunk ID together with the current version of that chunk. The Master Server keeps a list of these Chunkservers for each chunk and counts the entries. A Chunkserver that reports an older version does not count. Such a chunk is marked for repair or deletion. For this reason, an outdated copy is never accepted as a valid copy.

New copies appear in two situations. The first is the write itself. The Master Server chooses the Chunkservers for a new chunk and orders each of them to create the chunk. The Master Server then sends the client the list of these Chunkservers. The client sends the data to the first Chunkserver on the list only. That Chunkserver stores the data and at the same time forwards the data to the next Chunkserver on the list. The next Chunkserver does the same, and so on until the chain ends. Because of this chain-like write, the client sends the data once, no matter how many copies the chunk has.

The second is the chunk loop, the background pass in which the Master Server reviews the registered chunks. The Master Server compares the copies that a chunk has with what its Storage Class requires. Copies go missing when a Chunkserver disconnects or a disk is lost. Copies also go missing when the Storage Class requires more copies than before: its definition was changed, or the file was given another class. When copies are missing, the Master Server chooses a Chunkserver that does not hold the chunk. The Master Server orders that Chunkserver to fetch the chunk from a Chunkserver that still has a valid copy. The transfer runs between the two Chunkservers. Chunk data never passes through the Master Server.