Skip to main content

Staying available

What keeps a cluster serving while parts of it fail, and what the cluster needs to keep serving.

No single point of failure covers the claim. The page splits the claim in two because two different things can go wrong: you cannot reach the file system while a component is down, or you lose something that cannot be read back afterwards. A component failure can threaten one of these two things and not the other. That page says, component by component, which of the two a failure threatens.

Metadata answers the first question through leader election and quorum PRO (PRO Edition only). Several Master Servers run, but only one of them may manage the instance at a time. Leader election decides which Master Server is the manager. Data answers the same question on a schedule of its own. Self-healing is how the cluster detects that a chunk no longer holds what its Storage Class requires. The cluster then restores the redundancy on its own. Self-healing works through named states. The GUI and the CLI print the same names. Maintenance mode lets you stop self-healing on purpose. Maintenance mode tells the Master Server that the absence of one Chunkserver is expected, so the cluster waits instead of rebuilding.