Skip to main content

Serving MooseFS over FTP

MooseFS implements no FTP service. There is no MooseFS-aware FTP server to install. Any FTP server that keeps its data in a POSIX file system serves MooseFS instead. Everything that such a server does to a file is an ordinary file call: opening the file, reading it, writing it at an offset, and listing the directory that holds it. The client answers each of these calls. Mount MooseFS on the machine that runs the server, and point the server's directories inside the mount. The FTP clients then get cluster data.

This page covers what that machine must have switched on, and whose identity reaches the Master Server. The page also covers what uploads, deletions, and a second gateway change. Nothing MooseFS-aware runs on the FTP clients. On a machine that can run a MooseFS client of its own, mount the file system and skip the second protocol.

The gateway runs a mount and an FTP server​

Install the MooseFS client on the gateway and mount the file system. The steps are in installing MooseFS clients. Then give the FTP server a path inside the mount point. That path can be the directory that the server serves anonymous sessions from, or the home directory of each account. The path can also be any other directory that the server opens files under. The server needs nothing MooseFS-specific, and no build option of its own. If the server should reach only part of the namespace, mount a subtree instead of the root. This is described in mount options.

Start the FTP server after the mount is up. A mount point with nothing mounted on it is an ordinary local directory. Uploads that go there fill the gateway's own disk, but the sessions that write them report success.

The server's own state belongs on that local disk as well. The process ID file, the logs, and the run-time record that some servers keep of the sessions in progress describe one running server on one machine. A second gateway pointed at the same file would overwrite the record of the first gateway.

The mount is already open to those users​

mfsmount mounts with allow_other among its default options. Processes that belong to users other than the user that made the mount therefore reach the file system. An FTP server depends on that option. The server starts as one user and then works on the files under the identity of whoever logged in. The -n option omits the default options. A mount made that way admits only the user who made it. Every session that works under a different identity is refused inside that mount.

Permissions follow the identity the session runs under​

The Master Server decides every permission question from the identity in the request. That identity is the user ID, the group ID, and the supplementary groups that the client sends with the request. A server that logs users in as system accounts gives them exactly the access that those accounts have. POSIX compliance covers what that access includes and where MooseFS stops behaving like a local file system.

The client reads the supplementary groups of the calling process from the operating system. A server that sets up the group list of the session before it accesses a file therefore has that membership applied. The list is cached per calling process for mfsgroupscacheto seconds, 300 by default. A membership change therefore reaches a long-running process when the cache entry of that process expires. A process started after the change gets the new membership at once.

Virtual users are a different case. MooseFS cannot tell them apart. A server can keep an account database of its own and run every session under one system account. Such a server writes every file with one owner and one group, whoever uploaded the file. Permission bits inside the mount then draw a boundary around the server rather than between its users. The export entry that admits the gateway is the boundary that still applies. In that entry, mapall replaces the identities that the mount acts with. disable= refuses whole operations. For a download-only server, a read-only entry is all that is needed. Access control covers the task, and mfsexports.cfg the full option list.

A confined FTP daemon needs FUSE allowed​

Distributions that ship an SELinux policy confine the FTP daemon. The policy decides which kinds of file system that daemon may access. A MooseFS mount is a FUSE file system. The boolean that covers FUSE file systems, ftpd_use_fusefs, is off until you turn it on:

setsebool -P ftpd_use_fusefs 1

Without this boolean, the session logs in and changes directory. Every file call inside the mount then fails. A listing is returned empty rather than refused. A download ends in 550 Failed to open file., an upload in 553 Could not create file.. The same paths open at the gateway's shell. This leads you to check file permissions instead. This is the distribution's policy rather than a MooseFS setting. A daemon that runs unconfined has nothing to change.

An interrupted upload leaves a file, maybe a hole​

FTP resumes a transfer by restarting it at an offset. The server performs that restart as a write at that offset, the way any other program would. A chunk index that the transfer never reached holds nothing and costs nothing. A gap inside a chunk that was written costs the whole of that chunk, because the file's length is what the cluster charges for, as length, holes, and truncation works out. Quota covers the difference between what a file measures and what it occupies.

A transfer that stops halfway leaves the data that arrived under the name that the transfer was writing to. The cluster cannot detect that more data was coming. Servers that write to a temporary name and rename on completion keep that distinction visible in the file system itself. The temporary name also keeps a half-written file away from whatever reads the directory next.

Deleting through the share frees nothing at once​

A file deleted by an FTP client is deleted like any other MooseFS file. With a trash retention above zero, the file goes to the trash. The file keeps its space until that retention passes or somebody removes the file from the trash. The file also stops counting against the quota of the directory that it was deleted from. An FTP area limited by a quota can therefore report free space that the Chunkservers do not have yet.

On a drop box, files arrive and are deleted the same week. The retention set on that directory decides how much of the cluster the turnover occupies at any moment. Space management covers reading and changing that retention.

One gateway carries one mount's traffic​

All the data that the FTP sessions on a gateway transfer passes through the mount of that gateway. A mount has a throughput ceiling of its own, set by its FUSE connection rather than by the cluster. More sessions on one mount do not raise that ceiling.

Adding gateways raises that ceiling. Each gateway registers a session of its own and exchanges file data with the Chunkservers directly, so the data paths multiply. The same Master Server still answers all of these gateways. The details are in multiple mounts. The FTP side decides which gateway a user reaches, with a DNS record or with a load balancer in front of the gateways.

Neither protocol on this path encrypts anything​

FTP carries its credentials and its data in clear text. MooseFS traffic between the gateway and the cluster is not encrypted either. The two halves are protected separately. The TLS support in the FTP server itself, called FTPS, protects the FTP side. The layer beneath the MooseFS connection protects the MooseFS side. The details are in authentication between modules.