MooseFS PRO only.

Location tasks

This page describes how to divide a running cluster into locations and how to run a cluster that is already divided. The tasks are converting a cluster that separates its sites by other means, adding a location, and moving modules between locations. The page also covers reading the cluster location by location, taking a location out of service on purpose, and retiring a location. Every task below changes the division with the mfslocadmin tools (mfslocadmin(1)) through a mounted file system. Every change applies as soon as the command returns. Divide a new cluster into locations describes where the division is stored and how the tools are run. That page also covers the first-time setup of a new cluster. The Location page explains what a location is and what its three states mean.

The examples use a two-site cluster: west on 192.0.2.0/24 and east on 198.51.100.0/24. The first section brings the cluster to that division. The sections after the first section extend the divided cluster with a disaster-recovery site dr-site on 203.0.113.0/24.

A label-separated cluster converts to locations

Before a cluster spread over sites is divided, it keeps the sites apart with labels. The Chunkservers of each site have a letter of their own, and one class requires copies on each letter. A conversion to locations changes the mechanism but not where the chunks are stored. The order below keeps the chunks where they are. The example converts a cluster whose Chunkservers on 192.0.2.0/24 are labeled W and whose Chunkservers on 198.51.100.0/24 are labeled E. A class in that cluster keeps two copies on each letter.

One fact decides how much replication the conversion costs. Every definition that the existing classes have belongs to the default location that the instance has always had. The sites that you create as locations are therefore the additional sites. The site that keeps the default location gets its name by renaming the default location. If you created a location for a site that is already in the default location, every class would stay defined for a location that the site no longer belongs to.

Pause chunk work for the length of the conversion. Between the mapping and the class rewrite, the old expressions require from each site copies that the other site now holds. The Master Server acts on that difference. The Master Server schedules no chunk jobs in a location while at least one Chunkserver there is disconnected and every disconnected Chunkserver there is in maintenance mode. Set maintenance mode by hand on one Chunkserver of each site and stop that Chunkserver. The condition is evaluated per location. A Chunkserver's maintenance state moves with its address into the location that the mapping assigns to that address. One server in maintenance mode per site, set before the mapping, therefore covers every location for the whole conversion. The servers in maintenance mode are unreachable while they are stopped, so do not do this while any data is stored in a single copy.
Create a location for every site except the site that keeps the default, and switch each new location on. The Chunkservers of each new location are already connected, so an OFF location would drop them at the mapping step:
```
mfslocationcreate east
mfslocationstate -s ON east
mfslocationrename default west
```

Map the addresses of each site, including every module:

mfslocationmap -i 192.0.2.0/24 west
mfslocationmap -i 198.51.100.0/24 east

Rewrite the classes. The label expression that represented a site becomes that location's own definition. The redundancy that each site held stays the same number. The class that kept 2W,2E across the cluster keeps two copies in each location instead:
```
mfsscadmin modify -l west -K 2 -l east -K 2 mydata
```
The page data placement tasks describes how to write the definitions, what a zero definition does, and when a location needs such a definition.
Put the default location where your plan places it, with mfslocationsetdefault. Quorum across sites explains which site that is and what the default location decides.
Switch maintenance mode off and start the Chunkservers that you stopped. The review then checks every chunk against the new definitions. These definitions now describe the same servers as the labels did, so the review finds the placement already correct and moves nothing.

Verify that the conversion moved nothing. mfscli -SLO shows west and east both in state ON, each with the full count of connected Chunkservers of its site. After the stopped Chunkservers have reconnected, mfscli -SIC -l west and mfscli -SIC -l east count no missing, endangered, or undergoal chunks in either location. Run mfsfileinfo on a file of the rewritten class before step 1 and again now. Both times, it prints the copies on the same addresses. The second output also shows the location of each copy.

After no class names the letters, the letters have no effect. You can remove them from the Chunkservers at any time afterwards (data placement tasks). The division adds new units rather than new placement. A whole site is switched to IO or OFF with one command instead of a maintenance flag per Chunkserver. Redundancy is evaluated per site. The leader-election majority is counted by site.

A new location joins without disturbing the cluster

Adding a site to a running cluster depends on one fact: a location is created in the OFF state. You can therefore map addresses and rewrite Storage Classes at your own pace. The cluster changes nothing until the final switch.

Create the location:
```
mfslocationcreate dr-site
```
The command refuses to create more locations than the maximum in Limits allows.
Map the addresses of the new site into the location. Map every module that the site will have: Chunkservers, a Master Server and Metalogger if the site gets them, and the machines that will mount from there:
```
mfslocationmap -i 203.0.113.0/24 dr-site
```
A range that another location already has is refused. With -f, the command moves the overlapping addresses instead of refusing. In a cluster that remaps server addresses, every address that a Chunkserver answers on has to be mapped to the same location. The warning in Divide a new cluster into locations covers that requirement.
Give your classes a definition for the new location, so that the classes define which data gets copies there and in what redundancy. You define placement per location with mfsscadmin and its -l switch, as described in data placement tasks. A class can create chunks only at the writing site and replicate to the new location afterwards. That class saves link traffic, but the data has no far copy for a period. The WAN cost of writes describes that choice.
Install and start the servers of the new site (install the server components). The Master Server refuses Chunkservers of an OFF location. The order of starting them and the next step therefore does not matter, because nothing joins before the switch.
Switch the location on:
```
mfslocationstate -s ON dr-site
```

The Chunkservers of the site connect. The Master Server starts replicating the chunks that the new definitions require. Verify with mfslocationlist, or with the per-location reading. The row of the new location shows state ON and the expected count of connected Chunkservers. The chunk matrix of the location fills as replication proceeds.

Moving a module is a mapping change

A module belongs to a location other than the default only when its IP address is mapped there. A module belongs to the default location when its address is mapped there or not mapped at all. Moving a Chunkserver or a client between locations therefore means changing the mapping of its address:

mfslocationmap -f -i 192.0.2.17 dr-site

-f is needed whenever the address is currently mapped to another location. To move a module to the default location, remove its mapping instead:

mfslocationunmap -f -i 192.0.2.17

Without -f, unmapping is refused while the range still covers a Chunkserver of a non-default location. A wide range in the command therefore cannot silently move servers into the default location.

The change takes effect at once. The module itself needs no restart and no setting. The Master Server disconnects a Chunkserver whose location changed, and the Chunkserver registers again in its new location. Clients, Master Servers, and Metaloggers are reassigned without losing their connection.

Verify with the per-location reading. The location column of the module's row shows dr-site. For a Chunkserver, this row is on the Chunkservers tab of the GUI or in mfscli -SCS. For a client, the row is in the Mounts tables. After an unmapping, the same column shows the name of the default location.

warning

A moved Chunkserver counts toward its new location immediately. The Master Server reviews every chunk once per location, against that location's own definition, so moving a Chunkserver that holds data starts replications and deletions until placement matches the definitions again. Check which classes the move touches before you remap.

The GUI and mfscli read location by location

The Locations table, on the Info tab of the GUI, lists every location with its ID, name, state, and assigned IP ranges. Under a Connected heading, the table marks which location holds the Leader Master Server. The table also counts the Follower Master Servers, Metaloggers, Chunkservers, and clients connected in each location. A flag icon marks the default location.

mfscli -SLO (mfscli) lists the same locations with other columns. It omits the IP ranges and names the default in a column instead of an icon. It also has a column that the GUI does not have: health. It does not show which location holds the Leader Master Server. mfscli -SIN prints the same table. mfslocationlist prints the states, the default, and the mappings.

The health column of mfscli -SLO shows whether the Chunkservers that count in the leader-election vote of the location are still connected. ALL CHUNKSERVERS means that all of them are connected. SOME CHUNKSERVERS WITH QUORUM means that some Chunkservers are disconnected but more than half are connected, so the location still votes. SOME CHUNKSERVERS WITHOUT QUORUM means that the location has lost its majority and its vote. The last value is NO CHUNKSERVERS. Quorum across sites explains what the vote decides.

Membership appears in the module listings. The Chunkservers tab, the Mounts tables, and the Metadata Servers and Metaloggers tables each have a location column. On the Chunkservers tab, a server of the default location has a flag icon. The tooltip of the icon shows whether the assignment is explicit or implicit. The implicit flag is a warning, because an unmapped Chunkserver also blocks moving the default location. In mfscli, -l followed by a location name narrows a listing to one location. The GUI's location filter narrows its views the same way.

Redundancy is evaluated location by location. The chunk matrix table, on the Info tab, shows the matrix over all locations. If you pick one location in the filter of the table, the table shows the matrix for that location alone. In mfscli, -SIC prints the matrix over all locations, and -SIC -l dr-site prints the matrix of one location. The Storage Classes table, on the Redundancy tab, lists each class once per location. One class therefore shows a separate definition and separate counters for every location that it names.

A planned outage is a state you set

A whole site can go offline on a schedule, for example for power work or for a network cut with a known end. In that case, set the state of its location. The cluster then stops expecting the site instead of treating the outage as a failure. IO (mfslocationstate -s IO dr-site) is for a site that is reachable but whose link is not reliable enough for bulk traffic. Clients keep reading and writing in the location. The Master Server stops replication, rebalancing, and chunk deletions there. OFF is for a site that is down:

mfslocationstate -s OFF dr-site

The Master Server then disconnects the Chunkservers of the location and refuses them when they try to connect. The Master Server runs no chunk work in the location and no longer counts the location in leader election.

mfslocationlist shows the state. The line of the location shows state: OFF, or state: I/O for the IO state. In the Locations table and in mfscli -SLO, an OFF location shows the same state with no connected Chunkservers, because the Master Server has disconnected them.

Two rules restrict the switch to OFF:

The default location cannot be switched OFF. Make another location the default first, with mfslocationsetdefault. That move is itself refused while any known Chunkserver is unmapped or while the new default location is OFF.
The location holding the Leader cannot be switched OFF. Move the Leader role to another site first, with mfssupervisor (mfssupervisor(8)).

Making another location the default does not lift the second rule at once. The Leader Master Server re-reads its own location only when it accepts the next Master Server or Metalogger connection. Until then, mfslocationstate -s OFF on the old default is refused with Can't deactivate a location with an active LEADER. That message names the second rule, not the first. Unmapped Master Servers belong to whichever location is the default, so they moved with the default. Repeat the command after half a minute. The Leader role does not have to move.

The same refusal has a second cause, and this cause is not tied to any location. A Master Server that has not yet determined its own location refuses OFF for every location, not only for the location that holds the Leader. This state lasts a few seconds after the Master Server starts and clears itself. In this case too, repeat the command.

When the site returns, set the state back:

mfslocationstate -s ON dr-site

The Chunkservers of the location reconnect. The Master Server resumes chunk work in the location, and the review replicates what the site missed while the site was offline. A site that disconnects without warning is a failure that you have to diagnose. Start the diagnosis in Troubleshooting. When you know that the outage will last, mfslocationstate -s OFF turns the failure into this planned state.

`mfsfileinfo` shows which location holds each copy

To verify how a file is spread across the sites, run mfsfileinfo (mfsfileinfo(1)). When locations are active, the tool prints every copy and every Erasure Coding part with the location that holds it and the state of that location:

mfsfileinfo /mnt/mfs/projects/results.dat

/mnt/mfs/projects/results.dat:
	chunk 0: 0000000000000052_00000001 / (id:82 ver:1) ; mtime:1787227200 (2026-08-20 14:00:00)
		copy 1: 192.0.2.21:9422 ; location:west:ON ; status:VALID
		copy 2: 203.0.113.31:9422 ; location:dr-site:ON ; status:VALID

Placement works as defined when the output has one copy line for each location that the class names, on addresses mapped to that location.

A location is deleted only when empty

Deleting a location is refused while anything still refers to it. The location must be in the OFF state. No address range may map to it, and no Storage Class may have a definition for it. The default location cannot be deleted. Retire a location in this order:

Remove the location from your class definitions (data placement tasks). The copies that the location held become surplus, and the review deletes surplus copies on its own.
Stop the Chunkservers of the site, then unmap its address ranges with mfslocationunmap. If you unmap running modules, they move into the default location. That is a regular move, not a removal.

Switch the location OFF and delete it:

mfslocationstate -s OFF dr-site
mfslocationdelete dr-site

To confirm, check that mfslocationlist no longer shows the location.

When you delete the locations down to one, the division also disappears from most of the cluster's output. The GUI, mfscli, and mfsfileinfo report locations only while more than one location is defined. mfslocationlist is the exception. It prints the remaining location, the state of that location, the default marker, and every remaining mapping, however few there are. Rename the location back to default and remove the remaining mappings anyway. The listing then looks like the listing of an instance that was never divided: one location named default, and no mappings. A configuration that still names sites that the cluster no longer has misleads the next reader.

A label-separated cluster converts to locations​

A new location joins without disturbing the cluster​

Moving a module is a mapping change​

The GUI and mfscli read location by location​

A planned outage is a state you set​

mfsfileinfo shows which location holds each copy​

A location is deleted only when empty​