Authentication between modules

Every connection in a MooseFS instance has a side that accepts it and a side that presents itself. The presenting side must show something different for each pair of modules. This page covers each pair. Protocol and ports covers which port each connection reaches.

The rest of this page describes the arrangement of MooseFS 5. Version 5.0.0 reworked how modules authenticate to the Master Server, extended that authentication to the Metalogger, and added authentication of chunk I/O. Before that version, authentication could be required only at Chunkserver registration, optional since 3.0.65, and at a client mount.

Server modules prove a shared code

Chunkservers, Metaloggers, Follower Master Servers PRO (PRO Edition only), and the supervisor all register with the Master Server. By default the Master Server asks them for nothing: any module that reaches the port may register. Defining AUTH_CODE in mfsmaster.cfg makes the code a requirement for all of them at once. Chunkservers can be required to present a code since MooseFS 3.0.65, Metaloggers and the supervisor since 5.0.0. A Follower is required to present the code by the same mechanism as a Metalogger.

The code itself is never sent. A module that must present it registers twice. The Master Server answers the first attempt with a 32-byte challenge. The module then repeats the registration with a SHA-256 HMAC of that challenge, keyed with its own copy of the code. A Chunkserver older than 5.0 answers with an MD5 digest computed over the challenge and the code instead. The Master Server computes the same digest from its own copy. On a mismatch the Master Server closes the connection and logs the denied attempt. A module that has a code configured presents it only when the Master Server asks. The code can therefore be set on a Metalogger and on a Follower before the Master Server starts to require it. A Chunkserver cannot be prepared that way. A Chunkserver that has the code closes its connection to a Master Server that has none. A Chunkserver without the code refuses a Master Server that requires it. No order of the two changes keeps Chunkserver connections up. The Chunkserver closes the connection on purpose. Accepting a Master Server without a code would leave the authentication between Chunkservers and clients open to a man-in-the-middle attack. Each module takes the code from its own configuration file: a Chunkserver from mfschunkserver.cfg, a Metalogger from mfsmetalogger.cfg, a Follower from its own mfsmaster.cfg. Like any other configuration option, the code can also come from the command line (-o AUTH_CODE=<code>) or from the prefixed environment variable MFSCFG_AUTH_CODE. Both take precedence over the configuration file. The supervisor (mfssupervisor) has no configuration file. With -p the supervisor prompts for the code interactively. Without -p the supervisor takes the code from the MFSAUTHCODE environment variable.

The code is checked only at registration. A registered connection is never asked again. Adding a code, or changing a code that is already set, takes effect when the Master Server reloads its configuration. The Master Server then closes every connection from these modules at once. Each module reconnects on its own and answers a new challenge. A module answers with the code it holds in memory. Only a reload of that module replaces that code. A module that has not been reloaded answers with the old code and is refused. Since 5.2.0, removing the option takes effect on the Master Server like any other change. The Master Server closes the connections, and the modules register again without a challenge. Up to 5.1.0, a reload that found AUTH_CODE removed after it had been set ended the Master Server process. A Chunkserver process still ends after such a reload. The process crashes with a segmentation fault partway through the reload and logs no message that explains why. Only the service manager records the crash. By then the reload has applied the settings it handles before the Master Server connection, but not the connection settings. Remove the option from a running Chunkserver by stopping and starting the process, not by reloading it.

An export entry admits a client, password as digest

The entries in mfsexports.cfg on the Master Server decide which clients may mount. An entry admits a range of client IP addresses to a part of the namespace. It can also require a password, given either in plain text (password=) or as its MD5 digest (md5pass=). The Master Server keeps only the digest either way. An entry that names no password admits by address alone. A mounting client with a password first asks the Master Server for a 32-byte challenge. The client then registers with a SHA-256 HMAC of that challenge, keyed with the MD5 digest of the password. Older clients answer the same challenge with an MD5 digest instead. The password itself is never sent. A captured answer cannot be replayed, because a new challenge is drawn for every connection.

Export rule evaluation covers how the Master Server picks the entry when several match the same client. Session and export describes what the chosen entry grants to the resulting session, including the admin flag.

Chunkserver I/O can require a per-chunk code

A read or write request to a Chunkserver names a chunk and its version. Both come from the Master Server. By default the request carries no credential beyond that. Since MooseFS 5.0.0 the Master Server can require a credential. With USE_AUTH_FOR_IO set to 1 in mfsmaster.cfg, the Master Server generates a random secret. The Master Server sends the secret to the Chunkservers encrypted under a key derived from AUTH_CODE. The Master Server sends the secret to each mounted client encrypted under a key derived from the export password of that client. The client then stamps every read and write with a keyed hash of the chunk ID, the version, and the kind of operation. The Chunkserver recomputes the stamp and refuses the request when the stamps do not match. A Chunkserver stamps the replication traffic it pulls from another Chunkserver the same way.

The secret is short-lived. The Master Server draws a new secret every 60 seconds and distributes it again. A Chunkserver accepts a stamp made under any of the last three secrets it received, so a request that is in transit during the change still passes verification.

Both keys must exist for the stamps to work. USE_AUTH_FOR_IO set without AUTH_CODE disables only the stamps. The Master Server clears the flag in the module that serves Chunkservers and logs a warning that the option won't be used. The module that serves clients keeps its own copy of the flag and reads that copy whether or not AUTH_CODE is set. While the option is on, the Master Server refuses every mount that presents no password, even when no AUTH_CODE is set. Since 5.2.0, the refusal reaches the client as Password is needed. The Master Server also skips every entry of mfsexports.cfg that has no password, so a file without passwords leaves no entry at all (export rule evaluation covers what follows). A reload that switches the option on or off closes every client connection. While the option is on, the Master Server does not accept a returning client on its session ID alone, because a session ID can be stolen. Every reconnection repeats the full challenge and digest exchange. Only clients and Chunkservers from version 5.0 up can present the stamps. An older client is refused while the option is on. An older Chunkserver is refused at registration the same way.

The GUI and the CLI present no credential

The monitoring interfaces connect to the same Master Server port that clients mount through, but they never register a session. The Master Server answers their queries and their module commands without asking for any credential. Module commands include putting a Chunkserver into maintenance mode and removing a session. Reaching the port is the only requirement. In MooseFS PRO PRO (PRO Edition only) the license data shown by the GUI and mfslicinfo is served on this port on the same terms. Commands that change placement rules, Storage Class and pattern administration and, since 5.2.0, location changes PRO (PRO Edition only), do not take this path. They go through a mounted session and require its admin flag, which comes from the admin option of the export entry that admitted the mount.

No transport between modules is encrypted

There is no encrypted transport between any pair of MooseFS modules. The exchanges above never send the codes and passwords themselves over the network, but everything else, file data and metadata alike, is sent unencrypted.

Encryption is therefore the job of the layer below. MooseFS traffic can be carried over an encrypted link between the machines, such as a VPN or any comparable tunnel. Neither side of a MooseFS connection needs any setting for that link. An instance whose traffic leaves a trusted network is protected that way, not by an option inside MooseFS.

What each connection has to present

Connection	What it presents	What switches it on	Since
A Chunkserver, Metalogger, Follower, or supervisor registering	A keyed digest of the Master Server's challenge. The code itself never travels.	AUTH_CODE, set on the Master Server and on the module	Chunkservers 3.0.65. Metaloggers and the supervisor 5.0.0. Followers by the same mechanism as a Metalogger.
A client mounting	The export entry that admits its address, and the entry's password as a digest when the entry sets one.	An entry in mfsexports.cfg
A client reading or writing a chunk	A stamp over the chunk ID, the version and the kind of operation, keyed with a secret the Master Server redraws every 60 seconds.	USE_AUTH_FOR_IO, which needs AUTH_CODE and export passwords	5.0.0
The GUI, the CLI and the exporter	Nothing. Reaching the port is the whole requirement.

None of the connections in the table is encrypted in transit, whatever it carries.