The Metalogger

The Metalogger is the module that keeps an on-disk copy of the Master Server's metadata on a machine of its own. The module runs as one process, mfsmetalogger, which is configured in mfsmetalogger.cfg. Both editions ship the Metalogger.

The Metalogger runs one connection and serves nothing

A Metalogger consists of a single module: its connection to the Master Server. The Metalogger connects to the port that the Master Server keeps for Metaloggers, supervisors, and Follower Master Servers PRO (PRO Edition only), 9419 by default. The Metalogger then registers as a Metalogger and writes down what arrives. The network ports table lists the port.

Two things come over that connection: each metadata change, sent when the Master Server records the change in its changelog, and a complete metadata dump, downloaded on a schedule. A Master Server that requires an authentication code accepts the registration only when the Metalogger presents the matching AUTH_CODE. Authentication between modules describes the exchange. The Metalogger retries a lost connection every MASTER_RECONNECTION_DELAY seconds, 5 by default. The page on the Metalogger stream covers how registration works on the wire, how the change stream resumes after a disconnection, and when the dump is fetched outside the schedule.

The connection is outgoing only. A Metalogger listens on no port, so no component can ask a running Metalogger for anything. The Metalogger's files are used only when an operator starts a Master Server from them.

The Metalogger's data directory holds three kinds of file

A running Metalogger writes what arrives and serves none of it. The Metalogger reads one thing back: a downloaded dump. The Metalogger checks the header and the end marker of that dump before it replaces the previous one. The directory does not grow without bound either, because both the change stream and the dumps rotate, and the Metalogger drops the oldest generation of each. The directory holds the change stream as it comes in, the metadata dumps that the Metalogger downloads on a schedule, and the Master Server's own recent changelogs that arrive with each dump. The first two rotate under settings of their own. The third is always the same two files, replaced in full with each dump download. The _ml in a name marks the copy as the Metalogger's rather than the Master Server's.

• changelog_ml.0.mfs receives every incoming change. The file rotates together with the Master Server's own changelog, to changelog_ml.1.mfs and up. The Metalogger keeps as many files as BACK_LOGS sets.
• metadata_ml.mfs.back is the dump from the last download. The scheduled download runs every META_DOWNLOAD_FREQ hours. One more download runs whenever the Metalogger registers while it holds no position in the change stream. That happens after a start without a position on disk (the dump downloads on schedule and at start), and after a gap in the change numbering makes the Metalogger drop what it held. Earlier dumps stay as metadata_ml.mfs.back.1 and up, as many as BACK_META_KEEP_PREVIOUS sets.
• changelog_ml_back.0.mfs and changelog_ml_back.1.mfs are the Master Server's two most recent changelog files, downloaded together with each dump.

The mfsmetalogger.cfg reference lists every option of the process together with its default value.

The Metalogger is no server and never becomes one

A Metalogger is a module of the system, not a server: it answers no requests from clients, Chunkservers, or Master Servers. Leader and Follower PRO (PRO Edition only) are roles of a Master Server. A Metalogger never holds either role.

There is no promotion path. When the Master Server is lost together with its metadata, you recover by setting up a Master Server again and starting it from the files that the Metalogger holds. mfsmaster started with its -a option finds the newest usable metadata dump in its data directory, metadata_ml.mfs.back included, and replays the changelog files onto it. The command line tools reference covers mfsmaster.

In the PRO Edition the Metalogger follows the Leader

In a cluster of several Master Servers PRO (PRO Edition only), the stream that a Metalogger receives comes from the Leader. Resolving MASTER_HOST may point the Metalogger at any Master Server of the cluster first. A Master Server that is not the Leader answers the registration with the Leader's address. The Metalogger then reconnects there. A Master Server that changes its role disconnects its Metaloggers, so after a change of Leader the registration repeats and ends at the new Leader.

A Follower Master Server receives the same broadcast of changelog lines, described in Leader-Follower synchronization. The Metalogger's copy is separate from the copy of a Follower: on disk rather than in memory, and on a machine that is not part of the Master Server cluster.