Moving between CE and PRO

An instance changes edition by changing packages. The file system stays where it is. The chunks stay on the disks they are on. The metadata carries over. Both directions are possible. Neither is final: an instance moves from Community Edition to PRO, and back from PRO to Community Edition. Moving to PRO is the usual direction. An organization takes it when it grows and wants more of the system. The move back is taken as insurance by anyone who wants to know that the data stays reachable in a future without access to PRO, for whatever reason. The move back is also taken plainly as a decision to run Community Edition. This page describes what each direction brings with it and what the move back costs. Editions: Community and PRO covers what an edition is and which capabilities each edition includes.

The version string names the edition

Every daemon prints its version without touching the cluster. A PRO build ends the version string with -pro:

mfsmaster -v

version: 5.2.0-1-pro ; build: 2219

A Community Edition build prints the same line without the -pro part. The GUI and mfscli show the same fact as a PRO suffix on the version of each module they list. A cluster with modules of both editions is therefore readable from one screen. Every module of the wrong edition also raises an alert of its own. The messages and their severities are in Alerts.

Moving to PRO swaps packages, adds a license

Each moosefs- package is replaced by the moosefs-pro- package of the same role. The names are listed side by side in Install the server components. The Master Server then reads a license file. Install a license covers it, from the file to the check that every Master Server runs under the license.

The swap leaves a PRO instance with the Master Server it already had. Metadata redundancy across machines comes next. Metadata redundancy needs work outside MooseFS: a Follower PRO (PRO Edition only) is found through the name that the other modules resolve, not through a setting on each of them. Another Master Server therefore joins the cluster once DNS answers with its address as well. Leader election, quorum, and the role of DNS covers what the name has to return and what happens when two modules resolve it differently.

The Chunkservers change edition before the Master Server

A PRO Master Server refuses a Chunkserver built as Community Edition. The Master Server drops the connection at registration and logs the reason:

CSTOMA_REGISTER: chunkserver (<server>) without pro features - master with pro features needs chunkservers with pro features

The Chunkserver logs the same event as a connection that opens and closes again: connected to Master, then connection was reset by Master, then closing connection with master. The lines repeat for as long as the mismatch lasts.

Those two logs also confirm that the order held, so the sequence needs no reading of its own. The Master Server writes the refusal to the system log the moment a Chunkserver built as Community Edition tries to register. The Master Server writes the refusal again on every retry. A Master Server that starts on the PRO build and logs no such line found every Chunkserver already running PRO.

A Community Edition Master Server has no matching check, so a Chunkserver already running PRO registers with that Master Server and keeps serving. Every Chunkserver therefore runs PRO before the Master Server does. This order reverses the sequence that a version upgrade follows in Order of upgrade.

No check of this kind blocks a client or a Metalogger, so their packages can be replaced on their own schedule. Version compatibility covers the mismatches that such a schedule leaves behind and how each mismatch is reported for as long as it lasts.

Verify that every module runs PRO

The version string is the last check of the move. The monitoring tools show it next to every module at once. In the CLI, the version column of these sections covers the Master Servers, the Metaloggers, the Chunkservers, and the mounted clients:

mfscli -SIM
mfscli -SML
mfscli -SCS
mfscli -SMS

Every row has to end its version in PRO. The same columns appear on the matching tabs of the GUI. A row without the suffix is a module still running the build it had before. A Chunkserver is the only module that does not appear that way. The Master Server refuses such a Chunkserver at registration. A Chunkserver left in Community Edition is therefore missing from the connected servers rather than listed at the wrong edition.

The GUI adds a second reading. Alerts appear there and not in the CLI. The tables of the CLI have no alert list. With every module on the PRO build, none of the edition mismatch alerts in Alerts is raised.

The license has a reading of its own. Install a license describes it. The section verify that all masters run under the same license ends with every Master Server reporting the license you purchased and a Master version that reads PRO.

Metadata crosses the boundary without conversion

Both editions write metadata in one format: the same sections, in the same order, under the same header. A PRO Master Server starts from the file that a Community Edition Master Server left. A Community Edition Master Server starts from a PRO file. Neither direction has a conversion step. A move to an older version is a separate matter. Metadata does not always make that move (rollback). An instance that has never run PRO has no location section in its metadata at all. A PRO Master Server creates that section.

The contents of the instance carry over as they are. The move alone adds nothing to them. A PRO Master Server reads a Storage Class written under Community Edition exactly as it is. PRO can then raise the Redundancy Level of that class. The wider set of default classes belongs to instances created under PRO. An instance that changes edition keeps the classes it already has.

Moving back to CE drops the PRO capabilities

The move back is the same package swap in reverse. The check that fixes the order of the move to PRO does not apply to the move back. A Community Edition Master Server accepts a Chunkserver that still runs PRO. Removing the PRO build removes the capabilities it included:

• The Follower PRO (PRO Edition only) role has nothing to run on. Community Edition has no state shared between Master Servers and no connection between them, so the instance runs one Master Server.
• The -e switch starts a Master Server with no metadata of its own and fetches the metadata from the Leader. This switch is not in the Community Edition build. A Community Edition Master Server started with no metadata file of its own refuses to start and logs can't find metadata.mfs - try using option '-a'. The metadata that the remaining machine holds is therefore the metadata that the instance has.
• Locations PRO (PRO Edition only) stop being applied. A Community Edition Master Server works on the first location, reports every location as off, and does not ship mfslocadmin or its links.
• mfspatch PRO (PRO Edition only) with its links and mfslicinfo are not in the Community Edition packages.
• The monitoring tools lose what those capabilities supplied. mfscli drops its location and license sections. The GUI drops the command that takes a Chunkserver out of the cluster temporarily.
• The Licence Info table is shown for a PRO Master Server only. In its place the GUI states that the cluster runs the Community Edition under the GPL v2.
• PRO-only options left in mfsmaster.cfg and mfschunkserver.cfg are ignored. Nothing in the logs says so. Each file is still parsed but no longer describes what the process does.
• The quota that covers the whole instance PRO (PRO Edition only) is the only thing that leaves the metadata. A Community Edition Master Server skips this quota while loading the metadata and does not write the quota back.

The location map itself stays. A Community Edition Master Server writes back the same location section that it read. The locations, their names, and their states are therefore still there when a PRO Master Server reads the file again.

Verify Community Edition everywhere and no PRO capabilities

The same sections verify the first half when they are read the other way round: no version column ends in PRO. mfsmaster -v prints a version string without the -pro at its end. A Chunkserver does appear in the reading this time, because a Community Edition Master Server accepts a Chunkserver that still runs PRO. The alerts are reversed as well: a Chunkserver or a Metalogger left on the PRO build is an error. A client that still uses the PRO mount package is a warning.

The second half is read in the monitoring tools themselves, because what the instance gave up is missing from what they print:

mfscli -SIN

The full master info of a Community Edition installation has no Licence Info table. The missing table is the reading: a PRO Master Server prints the table whatever else is true of the instance. The Info tab of the GUI lacks the same table. In its place, the tab states that the cluster runs the Community Edition under the GPL v2. The Locations table is also missing, but its absence decides nothing on its own. The GUI and mfscli print it only while more than one location is defined. A PRO instance that was never divided into sites therefore shows none either.

The rest of what the move back takes out of the instance reads the same way: the tool is not installed, or the section is not printed. The PRO-only options left in the configuration files are the only entry without a reading. The Community Edition build ignores them and logs nothing about it. The options can therefore be found only in the files themselves.

A Storage Class survives, its Redundancy Level does not

A Storage Class that requires more than one parity part PRO (PRO Edition only) is the case that decides how reversible the move is. Its definition survives: a Community Edition Master Server loads the class as it is and writes it back unchanged. Its behavior does not. Community Edition treats such a class as though it required a single parity part. The surplus parity parts of the chunks under the class are deleted as the cluster works through those chunks. Every place that prints the class shows the reduced level rather than the level in the definition.

The files under such a class therefore have less redundancy from then on. The cluster does this on its own, without anyone asking it to. Erasure Coding itself is not lost. Community Edition stores erasure-coded data with one parity part, so the layout stays. Only its width shrinks. Setting the level to one in PRO before the move makes the definition and the behavior agree from the start. The redundancy that this setting gives up is the redundancy that Community Edition would remove anyway.

Editing the class works differently from reading it. Community Edition refuses to create a Storage Class with a Redundancy Level above one, and refuses to raise an existing class to such a level. Community Edition answers with an invalid argument error and logs the reason. A class that already has the higher level can still be edited. As long as the edit leaves the ARCHIVE and TRASH definitions alone, the stored definition keeps the level it had in PRO. An edit that redefines the ARCHIVE or TRASH state itself removes that level permanently. Community Edition then writes that state as the edit specifies it, within the Community Edition limit of one parity part.

warning

A return to PRO reads the original definition and restores the original Redundancy Level, on one condition: nobody redefined the class's ARCHIVE or TRASH placement with mfsscadmin while the cluster ran Community Edition. Every other edit leaves the recorded level untouched: the description, the priority, admin_only, the archive delay, the labels mode, and the CREATE and KEEP definitions. A redefinition writes that placement in its Community Edition form. The level it had in PRO is then recorded nowhere and cannot be recovered.

A class can also have a separate definition for each location PRO (PRO Edition only). All of those definitions are written to metadata and survive the move. However, Community Edition applies the definition written for the first location and no other.

warning

A class that carries no definition for the first location is the case to settle with MooseFS Support before the move, not after. Take that step whenever a PRO instance that uses several locations is going down to Community Edition.

The two editions are not released in step

Release lines are not shared between the editions. Release 4.58.4 was published for both of them. Releases 5.0.0 and 5.0.2 were published for PRO only. The current Community Edition releases have 4.59.x version numbers. The current PRO releases have 5.x version numbers. The version that an instance runs therefore decides how large a move the change of edition is.

In the move to PRO, the target version is part of the decision rather than a detail of it. A version published for both editions makes the change of edition the only change. A version published for PRO alone makes the move two changes carried out on one instance. A version change has its own conditions on which module is replaced when. Those conditions are in Upgrade.

In the move back, the version that the instance moves to does not follow from the version that the instance runs today. Neither does the route to that version. Agree on both with MooseFS Support before any package is replaced. Agree on them in the same conversation as the Storage Classes. What the classes require decides how much redundancy the instance keeps through the move back.