Skip to main content

What MooseFS is

MooseFS is a distributed file system. It joins the disks of many servers into one file system that client machines mount and use the way they use a local file system. The storage is spread over the machines. The software on top of the storage accesses a single directory tree. Since 2005 MooseFS has provided a state-of-the-art solution for demanding users: a storage platform designed for stability, high availability, data tiering, and load balancing.

One server manages, the other servers store​

The work is split in two. The Master Server holds the metadata: the directory tree, the attributes of every object in it, and the place of every piece of file data. The Chunkservers hold the data itself, cut into chunks and spread over their disks. A client asks the Master Server where a file is and then reads and writes the data with the Chunkservers directly. The data bytes therefore never pass through the Master Server. The system overview covers the parts of an instance and the traffic between them.

Applications see a POSIX file system​

Through a mount, MooseFS behaves like an ordinary Unix file system: a directory hierarchy, POSIX ownership, permissions and timestamps, symbolic and hard links, special files, and access control lists. MooseFS keeps locks that hold across every client that has the file system mounted. Software written for a local file system needs no changes to store its data in MooseFS. POSIX compliance covers how far MooseFS follows the standard and what MooseFS adds on top of it.

A mount is the usual way to reach an instance, but not the only one. The same instance is reachable as a block device, and from a program that links the client library and needs no mount point at all. The clients page covers what each of those forms is built of and where each form runs.

Redundancy is per file, and self-restoring​

Redundancy is set on the file. A Storage Class attached to a file says how many copies of the file's data the cluster keeps and on which Chunkservers the copies are placed. The class can instead say that the data is kept as erasure-coded parts. The cluster applies the class to every chunk that the file is cut into. Different files can have different classes. A class can place a file differently once the file stops changing. In this way, data can move from faster to slower media as it ages.

When a disk or a whole Chunkserver fails, the surviving copies keep serving reads and writes. The Master Server has the missing redundancy rebuilt from those copies, with no operator involved (self-healing). Thanks to parallel data distribution, that repair, automatic disk recovery included, runs in the background while the file system stays in service. No single point of failure covers which failures cost access and which cost data.

Capacity grows while the file system stays up​

Adding a Chunkserver, or another disk to a Chunkserver already running, adds the new space to the file system. The Master Server starts using the new space on its own and spreads what is stored across the disks available to it. Hardware is removed the same way: the data on it is moved off first.

The servers are commodity hardware​

Chunkservers are commodity servers with ordinary disks. The server components run on POSIX-compliant operating systems on x86 and ARM processors, in 32-bit and 64-bit variants of both. A client machine additionally needs a working FUSE (Filesystem in Userspace) implementation, which the mount is built on. Supported operating systems covers which systems are supported and tested.

Two editions come from one source tree​

Community Edition is the open-source edition, published under the GNU General Public License version 2. MooseFS PRO runs under a commercial license. MooseFS PRO has capabilities that Community Edition does not have, among them several Master Servers PRO (PRO Edition only) that pass the managing role between themselves without an operator. Editions: Community and PRO states what separates the two editions. Editions & Licensing states what each edition requires of you.

Whether MooseFS suits your data comes next​

MooseFS is ideal for environments requiring distributed storage solutions, such as big data processing, cloud storage, and high-performance computing. This page says what MooseFS is, not whether it is the system for your workload. Where MooseFS fits names the workloads that MooseFS is built for. Where MooseFS does not fit names the workloads that MooseFS is not built for. To try MooseFS rather than read about it, follow quick start, which describes how to build a working cluster from packages. History covers where the system comes from.