Files in a data directory

Every MooseFS daemon keeps its own files on disk: what it needs to survive a restart, the data for its charts, and its lock. These files are in the directory that DATA_PATH names in that daemon's configuration file. A Chunkserver also keeps a few files on every storage disk. This page names these files and says what each file holds and how long it stays.

The page on changelogs, dumps, and the recovery point covers what the metadata files are for and what a startup does with them. The files that a mount handles itself are listed in Special files in a mount.

The Master Server's data directory

File	What it holds	How long it stays
metadata.mfs	The complete metadata image as of the moment the process stopped. Its presence means that the last stop was clean.	Written at a clean stop. The next start loads it and renames it to metadata.mfs.back, so a running Master Server has no file under this name.
metadata.mfs.back	The image written by the last completed save.	Until the next save renames it to metadata.mfs.back.1.
metadata.mfs.back.1, metadata.mfs.back.2, …	Earlier images, renumbered one step up at every save.	For as many generations as BACK_META_KEEP_PREVIOUS allows.
metadata.mfs.back.tmp	A save in progress. It becomes metadata.mfs.back once the whole structure is in the file.	Until that rename. A file left under this name is a save that broke off. The next start removes it.
metadata_download.tmp	An image the Leader is fetching from a Follower instead of writing one itself (got from other). PRO (PRO Edition only)	Until it passes the check and is renamed to metadata.mfs.back. A transfer that fails, and a clean stop, remove it.
metadata.crc	The checksums of the sections of the image, written by the same save.	Replaced by every save, and by a checksum-only store PRO (PRO Edition only), which rewrites this file alone and leaves the image where it is.
changelog.0.mfs	The operations since the last rotation, appended as text as they happen.	Until the rotation on the hour renames it to changelog.1.mfs.
changelog.1.mfs … changelog.50.mfs	One hour of operations each, the number rising with age.	Until the number passes BACK_LOGS, 50 by default. The file holding that number is overwritten by the next rotation.
stats.mfs	The data behind the Master Server's charts, and what carries them across a restart.	Rewritten every hour and at a clean stop.
metadata.mfs.emergency	A save the Master Server could not write where it belongs. The data directory is only the first place the Master Server tries.	Until you remove it. A start with -a counts it as a candidate image.
metadata.mfs.back_emptystarted	An image that stood in the directory when the process was started empty.	Until you remove it.
metadata.mfs.XXXXXX	The metadata.mfs that stood in the directory when a start with -a found a metadata.mfs.back beside it. The six characters are a unique suffix.	Until you remove it. A later start with -a counts it as a candidate image and refuses to start when its metadata ID differs, unless -i is given.
sessions.mfs	Client sessions as MooseFS 1.x wrote them. MooseFS 5 keeps sessions in the metadata image itself, and never writes this file.	Read at startup when an upgraded installation still carries it.
metadata.mfs.empty	An empty image the packages install. No MooseFS 5 daemon reads it or writes it.	Until you remove it.
.mfsmaster.lock	The lock of the running process.	While the process runs.
.bgwriter.lock	The lock of the mfsmaster (data writer) subprocess: the one that appends the changelog, rotates it, and receives a downloaded image. The image save is made by a separate short-lived subprocess, which locks metadata.mfs.back.tmp instead.	While the process runs.
.mfsmaster_info.txt	The working parameters of the process, written when it receives the INFO signal.	Until the next INFO signal rewrites it.

A Follower keeps the same files as the Master Server PRO (PRO Edition only)

A Follower registers with the Leader the way a Metalogger does, but its data directory holds only the files above, under the Master Server's own names. The Follower keeps its own changelogs and stores its own images when the Leader orders a save. A Follower that falls too far behind the change stream receives a complete copy of the Leader's metadata over its connection, straight into memory, and writes no file for it (Leader-Follower synchronization). The names with _ml in them belong to a Metalogger and are listed under the Metalogger's data directory. With METADATA_DOWNLOAD_FREQ set, the transfer goes the other way: the Leader fetches an image from a Follower into metadata_download.tmp. That option is 0 by default.

The Metalogger's data directory

File	What it holds	How long it stays
metadata_ml.mfs.back	The last complete image downloaded from the Master Server.	Until the next download renames it to metadata_ml.mfs.back.1, for as many generations as BACK_META_KEEP_PREVIOUS allows.
changelog_ml.0.mfs	The changelog lines as the Master Server sends them.	Until the Master Server rotates its own, which moves this file to changelog_ml.1.mfs as well.
changelog_ml.1.mfs … changelog_ml.50.mfs	One hour of lines each, the number rising with age.	Until the number passes BACK_LOGS, 50 by default.
changelog_ml_back.0.mfs, changelog_ml_back.1.mfs	The two newest changelog files of the Master Server, fetched whole with the image.	Replaced by the next download.
.mfsmetalogger.lock	The lock of the running process.	While the process runs.
.mfsmetalogger_info.txt	The working parameters of the process, written when it receives the INFO signal.	Until the next INFO signal rewrites it.

The Chunkserver's data directory

File	What it holds	How long it stays
chunkserverid.mfs	The identity the Master Server gave this Chunkserver, and the metadata ID of the instance this Chunkserver belongs to. A Chunkserver that loses this file asks for a new identity.	Written at the first registration, and kept from then on.
csstats.mfs	The data behind this Chunkserver's charts, and what carries them across a restart.	Rewritten every hour and at a clean stop.
.mfschunkserver.lock	The lock of the running process.	While the process runs.
.mfschunkserver_info.txt	The working parameters of the process, written when it receives the INFO signal.	Until the next INFO signal rewrites it.

What a Chunkserver keeps on every storage disk

In addition to its own data directory, a Chunkserver writes into every path listed in mfshdd.cfg.

Name	What it holds	How long it stays
00 … FF	The 256 directories the chunk files themselves are stored in.	Created when the disk enters service.
.lock	The mark that a Chunkserver has this disk in service. mfschunktool refuses a disk that carries this mark.	While the process runs.
.metaid	The metadata ID of the instance the chunks on this disk belong to. A disk whose value does not match is refused.	Written once the Master Server has confirmed the identifier.
.chunkdb	The index of what the disk holds, written when the disk leaves the cluster so that adding it back does not mean reading every directory again.	Until the disk is scanned back in, which discards it.

A name with .tmp in it is the same file mid-write: metadata_ml.tmp, changelog_ml.tmp, sessions_ml.tmp, and metadata_download.tmp in a data directory, .tmp_chunkdb on a disk. Each file is renamed to its final name once the write is complete.