Access control

This page covers the tasks that control who may reach a running cluster and with what rights. The tasks cover admitting machines to mounts, putting a password on an export, mapping the identities a mount acts with, and granting administrative privileges. They also cover shielding single files, capping a mount's operation rate, authenticating server modules, and restricting access to the GUI.

Export entries decide who may mount what

Every mount is admitted by an entry in mfsexports.cfg on the Master Server. The entry that admits a mount decides what the mount may do. Edit this file whenever a machine is to gain, lose, or change access. An entry names an address range, the directory that the entry exposes, and a list of options. The syntax and the full option list are in the reference and in mfsexports.cfg(5). Export rule evaluation covers how the Master Server picks the entry when several match. Session and export covers what the selected entry gives the session.

The file shipped with the Master Server admits every address to the whole tree. The access is read-write, with any subdirectory mountable, with administrative privileges, and with root unmapped. A second entry in the file admits every address to the meta file system. Replace the file before the cluster is reachable from a network you do not control.

The Exports tab of the GUI shows the contents of the same file, one row per entry. Each row shows the entry's address range, path, and restrictions.

1. Edit /etc/mfs/mfsexports.cfg so that only the machines that need access have an entry and no entry grants more than its machines need:

   # Administrator hosts: full rights, root unmapped
   192.0.2.0/24     /      rw,alldirs,maproot=0,admin
   # Application servers: their subtree only
   198.51.100.0/24  /apps  rw
   # Machines with no entry cannot mount at all

2. Reload the Master Server with mfsmaster:

   mfsmaster reload

If the checksum of the entries changed, the reload disconnects every mount registered under the previous contents. Adding a comment or reordering the lines does not change that checksum, so the reload disconnects nobody. Each mount reconnects on its own within seconds and registers again under the current entries, so the change needs no unmounting. A client whose entry was removed is refused at that point. This applies to every change on this page that is made in mfsexports.cfg. Since 5.2.0, a reload that finds the file missing, unreadable, or without a single valid entry stops the Master Server instead of keeping the previous entries (export rule evaluation). Check the file before you reload.

To verify, use mfscli to list the entries that the running Master Server uses:

mfscli -SEX

A password asks for more than an address

To make admission depend on a secret rather than only on the client's address, add a password to the entry. Use password=, or use md5pass= to keep the clear text out of the file:

192.0.2.0/24  /apps  rw,password=ExamplePass

For md5pass=, generate the MD5 digest of the password and put the digest in the entry:

printf '%s' 'ExamplePass' | md5sum

After the reload, a client presents the password with one of the mount options. -o mfspassword=PASS gives the password in the command. -o mfspassfile=FILE reads the password from the first line of a file. This keeps the password out of the command and out of /etc/fstab. -o mfsmd5pass=MD5 gives the digest directly. -p (or --password) prompts for the password interactively. mfspassfile cannot be combined with mfspassword or mfsmd5pass. In the 5.x line, -p takes precedence when given together with mfspassword, mfsmd5pass, or mfspassfile. The password itself is never sent over the network in either direction. The client proves the password with a random challenge, as described in authentication between modules.

To verify, mount from an admitted machine without the password and check that the mount is refused. Repeat with the password, and the mount succeeds.

maproot and mapall remap remote users

When you do not trust the users of a mounting machine, map them. maproot=USER[:GROUP] makes the Master Server treat uid 0 from that mount as the given user. mapall=USER[:GROUP] does the same for every uid, root included, unless the entry also has maproot, which then decides for uid 0. An entry that names neither option maps root to uid 999 and gid 999. To leave remote root unmapped, you need an explicit maproot=0. Give this grant deliberately. An entry for the meta file system (directory field .) is the exception: it leaves remote root unmapped. maproot and mapall given on such a line are ignored with a warning in the log. Both options replace the user ID and the primary group only. The supplementary groups that the client sends stay as they are, so permissions granted through them still apply.

203.0.113.0/24  /public  rw,mapall=1000:1000

User and group names in these options are resolved on the Master Server host, not on the client. When the machines do not share a user database, numeric IDs are therefore the safer form. A mount admitted with mapall keeps the kernel attribute cache off.

To verify, create a file as root on the client and read its owner with ls -l. The file belongs to the mapped user, not to root.

The admin option gates cluster-changing operations

The Master Server refuses a set of operations unless the session that performs them was admitted by an entry with the admin option. The set covers changing quota values and adding and deleting patterns. It also covers creating, modifying, and deleting Storage Classes, applying Storage Classes flagged as admin-only, and setting the protective flags described below. Since 5.2.0, the set also covers every change to locations PRO (PRO Edition only): creating, renaming, and deleting a location, switching its state, setting the default location, and mapping or unmapping address ranges. Listing the locations needs no admin. The MooseFS tools perform these operations through a mount, so an administrator needs a mount admitted with admin. On a session without that option, the operations are refused. Keep admin on the entries for administrator machines and off all other entries.

Protective flags pin a file against change

To protect a single object against deletion or change for every client, set one of the protective extra attribute flags. undeletable blocks unlinking, appendonly allows only appending, and immutable blocks every change. The Master Server enforces these flags, so no mount and no mount option bypasses them. mfsseteattr, documented in mfseattr(1), sets them. Run it from a mount admitted with admin:

mfsseteattr -f undeletable /mnt/mfs/apps/ledger.db

Repeat -f to set several flags at once. -r applies the flags through a directory tree. The three flags are not inherited: objects created later under a flagged directory start without them. Clear a flag with mfsdeleattr, for example mfsdeleattr -f undeletable PATH.

To verify, read the flags with mfsgeteattr and try to remove the file. The removal is refused on every mount, including the mount that set the flag.

To block deletion for every client of one export regardless of flags, disable the operations in the entry instead, for example rw,disable=unlink:rmdir. The operations that disable accepts are listed in the reference.

Operation limits cap what one mount asks

Two export options, added in 5.1.0, limit the operation rate of clients. opslimit=OPS caps all I/O operations of a mount, root included, at OPS per second. opsperuidlimit=OPS caps each uid on the mount separately, and uid 0 is exempt. When an entry defines both options, both apply. Every mount admitted under the entry gets its own limit. The values are not shared across mounts.

198.51.100.0/24  /apps  rw,opslimit=5000,opsperuidlimit=200

Under this entry, each uid other than root performs at most 200 operations per second. The mount as a whole, root included, performs at most 5000. The mount itself enforces the limits. Operations over the limit wait in the client and do not reach the Master Server, so a runaway client does not overload the Master Server. The count excludes lock operations, statfs, open, and the operations of the MooseFS Meta mount. A chunk write counts once, when the write starts. The exchange that closes the write on the Master Server is not counted again. Operations issued through the MooseFS tools all count against one shared virtual uid. This matters only for the per-uid limit. The permissions that the tools need are checked as usual.

A mount receives the limits when it registers, so reload the Master Server after you change them. The reload then disconnects and re-registers the mounts, as described for export changes. To verify on the client, read the mount's .params file as root. The opslimit_global and opslimit_per_uid lines show the values in force. A mount admitted without limits shows 0 in both.

A shared code decides which modules may register

Set the same authentication code on the Master Server and on every module that connects to the Master Server. A module that you did not configure then cannot register as part of the cluster. Metaloggers and Follower Master Servers accept the code at any time before the Master Server has it, so configure them first. Chunkservers have no such order. A Chunkserver stops serving the cluster as soon as its code differs from the Master Server's setting, whichever of the two you change first. Turning authentication on therefore needs planned downtime for the cluster. The Chunkservers do not stay up through the transition. Authentication between modules explains why the Chunkserver is that strict.

From 5.0.0 on, a Master Server without AUTH_CODE still sends each Chunkserver the packet that carries the secret for chunk I/O. The packet is then empty. The packet is sent at registration and then every 60 seconds. A Chunkserver that has the code expects 48 bytes in that packet and closes the connection when the packet is empty. The Chunkserver reconnects after MASTER_RECONNECTION_DELAY and closes the connection again on the next empty packet. In the opposite order, when the Master Server gets the code first, the Chunkserver refuses. A Chunkserver with no code of its own drops the connection as soon as the Master Server asks it for a code.

1. Set AUTH_CODE in mfschunkserver.cfg on every Chunkserver, in mfsmetalogger.cfg on every Metalogger, and in the mfsmaster.cfg of every Follower Master Server PRO (PRO Edition only). Then reload each of these modules. The Chunkservers leave the cluster at this point and reconnect only after the next step, so run both steps within the planned downtime.
2. Set the same AUTH_CODE in the mfsmaster.cfg of the Master Server and run mfsmaster reload. The Master Server closes the connections of these modules. Each module reconnects and proves the code with a challenge.

The exchange itself, pair by pair, and the related USE_AUTH_FOR_IO option that extends authentication to chunk I/O are described in authentication between modules. That page also covers what the code does not give you: no transport between modules is encrypted. The code is not sent over the network, but the data is. To verify, start one module with a wrong code. Check that the Master Server refuses the module and logs the denied attempt.

Changing the shared code disconnects every module

To replace AUTH_CODE on a running cluster, run the procedure above a second time. Some module is disconnected at every point of the procedure. The Master Server keeps one code, not a set of codes. When a reload finds a value different from that code, the Master Server closes every Chunkserver, Metalogger, Follower PRO (PRO Edition only) and supervisor connection at once. It does so whatever code each module has. Configure the modules first, as at the start. A module reloaded in step 1 is refused by a Master Server that still has the old code. The connection itself is made, but the Master Server rejects the registration, because the module uses the new code to answer a challenge built from the old code. The refused modules keep trying until step 2. Every attempt leaves a warning in the log of the Master Server.

1. Set the new AUTH_CODE in mfschunkserver.cfg, mfsmetalogger.cfg, and every Follower's mfsmaster.cfg. Then reload each module. A Chunkserver reloaded here disconnects itself and reconnects only after step 2. The Master Server records the break as a temporary maintenance mode rather than a failure, so no rebuild starts. The temporary mode has its own expiry time, which is long enough for both steps.
2. Set the same new code in the Master Server's mfsmaster.cfg and run mfsmaster reload. Every module of these kinds loses its connection and reconnects with the new code. Each module reconnects on the next tick of its own timer. The wait is therefore at most MASTER_RECONNECTION_DELAY, which is 5 seconds by default in each of the three files. A module can also reconnect at once.
3. Verify in the system log of the Master Server. Each Chunkserver reports chunkserver <address>:<port> register begin, with its used and total space later in the same line. Then it reports chunkserver <address>:<port> register end. Each Metalogger reports metalogger <address> registered (using advanced register protocol), or metalogger <address> registered (using simple register protocol) when it starts with no changelogs of its own. Each Follower reports master-follower <address> registered. A module that still has the old code is refused instead and keeps retrying. The Master Server writes that refusal at the end of a longer line: CSTOMA_REGISTER (BEGIN,<address>) - access denied - check password for a Chunkserver, ANTOMA_REGISTER (logger 1,<address>) - access denied - check password for a Metalogger. A refusal line names the address without the port, because the Master Server receives the port only when the registration succeeds. On the Chunkservers tab of the GUI, no server remains among the disconnected servers.

The code is checked at registration and never again. A module's connection survives a reload of the Master Server only when the reload leaves the Master Server's code unchanged. A connection that survives such a reload is therefore not proof that the module's code matches. Check the pair of registration lines after the reload instead.

danger

Do not switch authentication off on a Chunkserver by deleting AUTH_CODE or commenting it out and reloading. A Chunkserver reloaded that way ends. The process crashes with a segmentation fault in the middle of the reload and writes no word about it to its log. Since 5.2.0, the Master Server applies the same change on a reload instead of ending. Up to 5.1.0, that reload ended the Master Server too. To leave authentication behind, take the Chunkservers through a stop and a start with the option already absent from their configuration files, rather than through a reload.

Changing an export password disconnects every mount

To replace a password, edit the entry in mfsexports.cfg and reload the Master Server. The change affects more than that entry. The Master Server keeps one checksum for the whole file. The Master Server stores this checksum in every session when the session is created. After a reload, the Master Server compares that stored checksum with the current checksum. Then it closes the connection of every client whose value no longer matches. After any change to an entry, this applies to every client. Comments and the order of the lines are not part of the checksum.

The clients then behave in two different ways. A mount whose own entry did not change registers again on its own. The mount keeps its session and the session's open files, and reports registered to master using previous session. Nothing on that machine is affected. A mount admitted by the entry whose password changed answers the challenge with the digest that it was started with. The mount is refused with Incorrect password and retries every two seconds without ever succeeding. It does not prompt anyone. It does not read the password from disk again, so unmount it and mount it again with the new password (mount options pass it).

Plan the change as an interruption for the machines that use that entry. Give them the new password before the reload, not after it. To verify, read the Exports checksum column on the Metadata Servers (Masters) subtab of the Info tab in the GUI, or the exports_checksum field of mfscli -j. The value changes with every edited entry. Then confirm on each affected machine that the mount responds, because a refused mount reports nothing that the Master Server could count.

The GUI is guarded by where it listens

The GUI server has no built-in authentication and serves plain HTTP. Its pages include commands that act on the cluster. Disconnecting a client session is one of them. To use these commands, a user only needs to reach the GUI port, 9425 by default (see network ports). Access must therefore be restricted outside the GUI. Bind the GUI to a local address and put an authenticating reverse proxy, or at least a firewall, in front of it.

1. In mfsgui.cfg, bind the GUI to the loopback interface and reload mfsgui:

   GUISERV_LISTEN_HOST = 127.0.0.1

   mfsgui reload

2. On the same machine, configure a reverse proxy that adds authentication. Add TLS as well if the traffic leaves a trusted network. For nginx:

   server {
       listen 80;
       server_name mfsgui.example.com;
   
       location / {
           auth_basic           "MooseFS GUI";
           auth_basic_user_file /etc/nginx/.htpasswd;
           proxy_pass http://127.0.0.1:9425;
       }
   }

To verify, open port 9425 from another machine and check that the connection is refused. Then open the proxy address and authenticate. Protecting the GUI port does not protect the Master Server's client port. That port answers the same queries and commands without credentials. The conditions under which the client port answers without credentials are described in authentication between modules.