Skip to main content

Where MooseFS fits

A property of the system decides whether a workload belongs on MooseFS, not the industry the workload comes from. This page names the properties that make a workload fit and says what each property requires of the data. Where MooseFS does not fit names the properties that rule a workload out.

Applications reach the data through ordinary file calls​

Through a mount, MooseFS behaves like a local file system: a directory tree, the object types and attributes that a Unix program expects, ACLs, and advisory locks that the cluster coordinates across every connected client (POSIX compliance). An application opens, reads, and writes as it did before, with nothing to link against and no protocol to speak.

That behavior makes MooseFS general-purpose storage rather than storage built for a single application. Software that already keeps its data in files needs no changes to store that data in MooseFS. The tools and habits of a local file system still apply. Where a mount does not suit the application, the same instance is reachable as a block device or from a program through a library (client and mount).

Data spreads, metadata stays on one machine​

File data is split into chunks of at most 64 MiB. The chunks of one file are placed across Chunkservers independently of each other. A client asks the Master Server where a chunk is and then exchanges the bytes with that Chunkserver directly, so file data never passes through one machine (the read path, the write path). Capacity and bandwidth are the sum of what the Chunkservers contribute. A server added to a running cluster adds both.

Metadata does not spread. One Master Server answers every lookup, create, and rename, and holds every object in its RAM. A single machine therefore handles the object count of a workload, but all of the Chunkservers handle the bytes of that workload (limits and scale). MooseFS therefore suits workloads whose size is measured in bytes rather than in objects: fewer and larger files rather than very many small ones. Sizing the machines for a particular workload is a calculation. That calculation starts at the workload profile. The limits reference lists the ceilings written into the product.

One cluster holds different data under different rules​

Redundancy is a property of a file, not a setting of the cluster. A Storage Class defines the redundancy. A new object inherits the class from the directory that the object is created in. You can also set a Storage Class on a whole subtree at once. A class has a separate rule for each storage state that a chunk passes through. One policy can therefore keep a file in copies while the file is in use. The same policy can then move the file to Erasure Coding once the file has been left alone for as long as the class says. The move changes nothing in the file's name, permissions, or place in the tree.

A workload that would otherwise need fast storage next to an archive fits in one MooseFS cluster, under one namespace. Copies or Erasure Coding compares what each format costs in raw disk space and how many Chunkservers each format needs.

The cluster serves through failure and repairs itself​

MooseFS is built for mission-critical applications requiring maximum reliability, high availability, and top-tier performance. Uninterrupted access to data is one of the core priorities of MooseFS. That priority makes MooseFS ideal for environments where downtime is not an option.

Copies and parity parts of one chunk are placed on separate machines. When a disk or a whole Chunkserver is lost, the Master Server replicates what remains until the Storage Class is satisfied again, with no operator to start the replication. The cluster keeps answering clients while that replication runs (self-healing).

The editions differ in the Master Server. Community Edition runs exactly one Master Server, so the loss of that server stops the file system until the server is available again. MooseFS PRO runs several Master Servers. A Follower PRO (PRO Edition only) takes the Leader role over without an operator. A workload that cannot pause when one machine fails therefore needs MooseFS PRO. No single point of failure compares the cost of that protection with the cost of the exposure. The rest of what separates the two editions is in the CE and PRO feature matrix.

Hardware mixes and changes under a running cluster​

MooseFS is software that you install on machines that you run, on bare metal, in virtual machines, or in containers (platforms). The machines do not have to match. Chunkservers of different capacities, disks of different sizes, and machines bought years apart work in one cluster, because new chunks are assigned in proportion to what each server has. Rebalancing evens out existing usage (mixing hardware). Machines based on different CPU architectures and hardware generations work together in one cluster as well. An installation can therefore reuse the hardware that it already owns. The platforms that each component runs on are listed under supported operating systems.

This is also where the cost of the whole thing is decided. MooseFS offers a low total cost of ownership thanks to its open-source foundation, ease of deployment, scalability, and vendor support: organizations can reach enterprise-grade storage performance without the heavy financial investment proprietary systems often require.

You can also change the cluster while it serves clients. You add a Chunkserver, add or replace disks in a Chunkserver, or retire a machine permanently without stopping the file system (disks and Chunkservers). A version upgrade works the same way, one component at a time (order of upgrade). The Master Server is again the exception. Only MooseFS PRO upgrades the Master Servers without a pause in service. MooseFS PRO moves the Leader role to a machine that is already up (rolling upgrade). This way of changing the cluster suits an installation that grows in steps and replaces hardware as the hardware ages, rather than an installation bought and retired in one piece.

Solutions describes deployments for workloads that fit​

Each of these workloads depends on one of the properties that this page names. Each workload has a page of its own that describes a design and the reasoning that justifies the choices in that design:

  • Virtualization: a disk image is one large file, so it spreads over many Chunkservers. The throughput of the image is not that of a single disk. Where a guest needs a raw device rather than a file, the block device presents a single stored file as a raw device.
  • CDN and media: copies of one chunk are stored on separate Chunkservers. A client reads from the Chunkserver that matches the client's preferred labels and has the fewest of that client's own operations. Distance decides nothing in that choice (topology mapping and distance). A file that many clients read at once is therefore served by several machines instead of one.
  • Backup target: data written once and kept for years moves to Erasure Coding on its own when its class says so. This lowers what a stored terabyte occupies without moving the files anywhere.
  • Computation on storage nodes: a client running on a Chunkserver is at distance 0 from the copies on that machine's own disks. The Master Server puts those copies first in the list of locations it answers with (topology mapping and distance).
  • Two data rooms: a cluster spread over sites is divided into locations PRO (PRO Edition only). A Storage Class defines placement per location, so a class can keep a full set of copies in every site. The loss of one site then costs no data.