Skip to main content

MooseFS storage in XCP-ng

This procedure gives an XCP-ng pool a storage repository backed by MooseFS. The virtual disks of the pool are then stored in the cluster. Every host of the pool reaches the same disks. XCP-ng includes a storage driver named moosefs. The steps below install MooseFS on the hosts, enable that driver, and create the storage repository.

The XCP-ng hosts take the RPM packages​

An XCP-ng host is prepared like any other machine of the RHEL family. Add the MooseFS package repository for the release that the host is based on. Then install what the host needs to run. Install the server components covers the Master Server and the Chunkservers. Install clients covers the client package that every host of the pool needs. The client reaches the file system through FUSE, so install the FUSE packages on each host as well:

yum install fuse fuse-libs

These are the FUSE 2 packages. XCP-ng 8.3.0 is built on a base that includes that release. On XCP-ng, the client package is linked against that release. The FUSE 3 package names in install clients are for the systems that ship FUSE 3.

Two things belong to the hosts rather than to MooseFS. The name of the Master Server must resolve on every host, through DNS or through /etc/hosts. Each Chunkserver and each client finds the cluster by that name (make the Master Server name resolve on every machine). Also, keep a disk that is meant to hold chunks out of the hypervisor's own storage. Instead, format and mount the disk for the Chunkserver, with one file system per disk. The steps are in give each Chunkserver disk its own file system and mount point.

Enable the moosefs driver in XAPI​

XAPI accepts only the storage drivers on its own list. moosefs is not on that list by default. The list is set by sm-plugins in /etc/xapi.conf. That file is not edited directly. A definition in a file under /etc/xapi.conf.d replaces the list. That file must therefore include every driver that the host already had. Read the list that the host has:

grep sm-plugins /etc/xapi.conf

Then write that list back with moosefs in it, on every host of the pool. The line below is the list of XCP-ng 8.3.0 with moosefs added. Compare this line with what your host printed. Keep any driver that your host has and this line does not:

echo "sm-plugins=ext nfs iscsi lvmoiscsi dummy file hba rawhba udev iso lvm lvmohba lvmofcoe shm smb cephfs glusterfs moosefs xfs zfs largeblock" > /etc/xapi.conf.d/99-enable-moosefs.conf
warning

XAPI reads the files in /etc/xapi.conf.d in alphabetical order and keeps the last definition of sm-plugins it sees. A file already there, added to enable some other driver, is overridden by your definition unless it lists the drivers of that file too.

Restart the toolstack to load the new list:

xe-toolstack-restart

Create the storage repository​

After the driver is enabled, one command creates a storage repository shared by every host of the pool:

xe sr-create type=moosefs name-label=MooseFS-SR content-type=user shared=True \
  device-config:masterhost=mfsmaster.example.com \
  device-config:masterport=9421 \
  device-config:rootpath=/xcp-ng

masterhost and masterport name the Master Server and the port that the Master Server listens on for clients. The port is 9421 unless the cluster was set up otherwise (network ports). rootpath is the directory inside MooseFS that becomes the root of the repository. Point rootpath at a subdirectory rather than at the root of the instance. Create that directory before running the command, from a host that has the file system mounted (first mount and verification). A rootpath that does not exist in the instance makes sr-create fail with SR_BACKEND_FAILURE_111. The repository then keeps its virtual disks in a subtree of its own, next to everything else that the cluster stores. That subtree can have a Storage Class of its own (data placement tasks).