Skip to main content

SMB and CIFS

MooseFS does not implement SMB. Windows machines, macOS clients, and applications that expect a network share use this protocol. Those clients reach a MooseFS instance through a gateway. A gateway is a machine that mounts the file system with the MooseFS client and runs an SMB server, such as Samba. The shares of that server point into the mount. Nothing MooseFS-aware is installed on the SMB clients. They use an ordinary share.

This page covers what a gateway must keep switched on, and how identities and locks pass between the two protocols. The page also covers what changes when several gateways serve the same tree. A machine that can run a MooseFS client of its own needs no share at all. Clients describes the forms of the client.

The gateway runs a mount and an SMB server​

Install the MooseFS client on the gateway and mount the file system. The steps are in installing MooseFS clients. Then point each share at a directory inside the mount point. The SMB server needs nothing MooseFS-specific: it reads and writes the mount with ordinary file calls. The client does all the work below those calls.

Start the SMB server after the mount is up. A mount point with nothing mounted on it is an ordinary local directory. A share that points at that directory therefore stores data on the gateway's own disk, outside the cluster.

The mount is already open to those users​

mfsmount mounts with allow_other among its default options. Processes that belong to users other than the user who mounted the file system therefore reach the file system. An SMB server depends on that option. The server runs as one user and works on the share under the identity of whoever connected. The -n option omits the default options. A mount made that way admits only the user who made it.

MooseFS compares names byte for byte, so Report.doc and report.doc are two files in one directory. SMB clients do not tell such names apart. The SMB server, not MooseFS, decides which of the two files a client reaches.

Permissions follow the identity the server works under​

The Master Server decides every permission question from the identity in the request. That identity is the user ID, the group ID, and the supplementary groups that the client sends with the request. POSIX compliance describes what those decisions cover and where MooseFS stops behaving like a local file system.

The client reads the supplementary groups of the calling process from the operating system. The group membership that the gateway resolves is therefore the membership that MooseFS applies. This includes membership that a directory service supplies through the gateway's name service. The list is cached per calling process for mfsgroupscacheto seconds, 300 by default. A membership change therefore reaches a long-running process when the cache entry of that process expires. A process started after the change gets the new membership at once. Setting the option to 0 turns supplementary groups off. Requests are then evaluated against the primary group alone. The option is listed with the rest in mount options.

The export entry that admits the gateway sets the outer limit for all of this. Give the gateway an entry of its own. maproot and mapall replace the identities that a mount acts with. disable=setfacl and disable=setxattr refuse the calls that an SMB server makes to store permission records of its own. Access control covers the task. mfsexports.cfg lists all the options.

The Master Server holds ACLs and extended attributes. An SMB server that keeps its own permission records or alternate data streams there therefore finds them again from every gateway. A record kept under a name in the security. namespace is the exception: only root writes those names, and the gateway keeps root only when its export entry sets maproot=0. A mount started with mfsnoxattrs refuses the extended-attribute calls instead. The limits page states how large a name and a value may be.

Locks and their data reach every gateway​

The Master Server holds both families of file lock. A lock taken through one gateway is visible through every other gateway. SMB byte-range locks that the server translates into POSIX record locks therefore serialize work across the gateways, not only inside one gateway.

The data follows the lock. Before a mount asks the Master Server to release a POSIX record lock or a BSD lock, the mount flushes what it still holds for that file. Everything written under the lock is therefore in the cluster before the lock is released. Multiple mounts covers what other mounts then do with their cached data.

A mount can keep either family inside its own machine, with mfsnoposixlocks and mfsnobsdlocks. On a gateway those options remove the coordination exactly where the gateways need it. Two gateways can then both hold what each gateway treats as an exclusive lock on the same file.

The same property also covers the shared lock that a clustering layer such as CTDB keeps for itself. A file inside the mount is locked cluster-wide, so one place decides which node is authoritative. Every gateway asks the same Master Server about that lock. A gateway that fails holds the lock for as long as its session outlives the gateway. The next section describes the limit on that time.

Stopping without unmounting keeps the locks​

Locks belong to a session. A session outlives the connection that created it. A gateway that unmounts closes its session at once. Everything that session held is released. A gateway that crashes, or loses the network and does not reconnect, leaves its session behind for SESSION_SUSTAIN_TIME, one day by default. That session keeps its files open and its locks held for that time. A conflicting lock requested on another gateway is therefore refused or waits.

Set SESSION_SUSTAIN_TIME in mfsmaster.cfg to the time you are willing to wait for a failed gateway's locks. A value below one minute or above one week is raised or lowered to that limit.

One gateway carries one mount's traffic​

All the data that the shares on a gateway transfer passes through the mount of that gateway. A mount has a throughput ceiling of its own, set by its FUSE connection rather than by the cluster. More shares on one mount do not raise that ceiling.

Adding mounts raises that ceiling, on another gateway or on the same gateway. Each mount registers a session of its own and exchanges file data with the Chunkservers directly, so the data paths multiply. The same Master Server still answers all of these mounts. The details are in multiple mounts.