Skip to main content

Placement and policy

Which Chunkservers a file's data may be stored on, and what decides that.

The policy is the Storage Class: a named rule that says in what form the data is kept and which servers may hold it. The class has a separate definition for each storage state, the phase of a chunk's life. A chunk being written for the first time, a chunk in ordinary use, and a chunk whose file has been deleted can therefore each be stored differently. Patterns apply a class to a file without anyone setting the class by hand. A pattern is a rule matched against the name at the moment when that name appears.

The other three terms describe what a class refers to. A label is a letter that you attach to a Chunkserver so that a rule can tell your servers apart at all. The letter means whatever you decide it means. Topology measures and does not decide: it maps how far apart two machines are. That measure lets the cluster prefer the nearer of two equally good copies. A location PRO (PRO Edition only) is a named group of the cluster's own modules, managed and filled as one. Labels and topology describe single machines and the distance between them. A location groups many machines at once, for example all the modules of one site. A Storage Class can define placement separately for each location.

  • Data placement tasks: labeling Chunkservers, defining a Storage Class, assigning it, and checking where the data went.
  • Location tasks: the work that a cluster spread over several locations needs on top of that.