mfsmaster.cfg
Every option of mfsmaster.cfg, the configuration file of the Master Server. Lines starting with # are comments, and one line sets one option:
OPTION = VALUE
An option the file does not set takes the default value from this table. A dash stands for an option with no default: what it configures stays off until you set it.
A path, a user name or a port number below is the value the packages are built with. If you build MooseFS yourself, configure sets that value. See values set at build time.
warning
In an installation with several Master Servers PRO (PRO Edition only), each Master Server reads its own copy of mfsmaster.cfg. The copies have to agree. Options that differ between the copies lead to behaviour that no single copy describes. Keeping the copies consistent is your work. CONFIGURATION_TEST_PERIOD sets how often the Leader Master Server tests the modules for such differences and reports them.
Startup options
Changes in this section require process restart.
| Option | Default | Description |
|---|---|---|
mfs | User to run daemon as. | |
mfs | Group to run daemon as (optional - if empty then default user group will be used). | |
mfsmaster | Name of process to place in syslog messages. | |
0 | Whether to perform mlockall() to avoid swapping out mfsmaster process (0, i.e. no). | |
1 | Linux only: disable out of memory killer. | |
-19 | Nice level to run daemon with (note: process must be started as root to increase priority, if setting of priority fails, process retains the nice level it started with). | |
027 | Set default umask for group and others (user has always 0, 027 - block write for group and block all for others). | |
/var/lib/mfs | Where to store daemon lock file. |
Runtime options
Changes in this section require only process reload.
| Option | Default | Description |
|---|---|---|
| PRO (PRO Edition only) | /etc/mfs/mfslicense.bin | Alternate location/name of mfslicense.bin file; this file will be re-read on each process reload, regardless if the path was changed. |
/etc/mfs/mfsexports.cfg | Alternate location/name of mfsexports.cfg file; this file will be re-read on each process reload, regardless if the path was changed. | |
/etc/mfs/mfstopology.cfg | Alternate location/name of mfstopology.cfg file; this file will be re-read on each process reload, regardless if the path was changed. | |
| not set | Optional name of MooseFS instance (cluster) that will be displayed in GUI interface. | |
0 | Offset used for options: METADATA_SAVE_FREQ, METADATA_DOWNLOAD_FREQ, METADATA_CRCCHECK_FREQ (minutes) format X:Y is also accepted and means X hours and Y minutes, suffix "L" can be added to mean local(server) time zone instead of UTC. | |
1 | How often single master or follower will store metadata (hours). | |
| PRO (PRO Edition only) | 0 | How often leader will download (instead of store) metadata from followers (hours - 0 which means do not download metadata). |
| PRO (PRO Edition only) | 24 | How often leader will store metadata and check its consistency with followers (hours). |
| PRO (PRO Edition only) | 50 | Metadata downloading limit in mebibytes per second (0 means no limit). |
1 | Number of previous metadata files to be kept. | |
50 | Number of metadata change log files. | |
5000 | How many seconds of change logs have to be preserved in memory (this sets the minimum, actual number may be a bit bigger due to logs being kept in 5k blocks; zero disables extra logs storage; maximum accepted value 100000). | |
500 | How much RAM can be used to preserve change logs in memory in megabytes (this sets the minimum, actual number may be a bit bigger due to logs being kept in 5k blocks; zero disables extra logs storage) master will always preserve the lower number of 5k blocks defined by CHANGELOG_PRESERVE_SECONDS and CHANGELOG_PRESERVE_MB, so be sure to set them appropriately. | |
0 | Changelog save mode, one of the three following behaviours is possible: 0 - write in background by different process (less safe, but doesn't make master stop in case of heavy hdd load) 1 - write in foreground without syncing data (master waits for every changelog to be saved to hdd, but without syncing - a little more safe than the background option, but may cause master to stop and wait for flushing hdd buffers) 2 - write in foreground with fsync after each write (very safe, but may make your master very slow unless you have very sophisticated hardware). | |
| PRO (PRO Edition only) | 0 | How many seconds until master-elect turns into master-deputy (a.k.a master-readonly) if there are not enough chunkservers connected to become master-leader (0 - never). |
| PRO (PRO Edition only) | 0 | If this option is set to 1, this master will try to assume leadership after it has lost it due to a restart or disconnection; it will not attempt this if a user has switched the leadership on purpose; for detailed explanation see man mfsmaster.cfg. |
100000 | How many missing chunks will be stored in master (up to 100*MISSING_LOG_CAPACITY bytes of memory will be allocated). | |
| PRO (PRO Edition only) | 24 | How often master leader should run a test, that checks if configuration options are consistent between modules (hours, max possible value is 168, 0 means no tests are performed). |
INFO | Minimum level of messages that will be reported by master; levels in order of importance: ERROR, WARNING, NOTICE, INFO, DEBUG. | |
NOTICE | Reported messages of level lower than set here will be elevated to this level (i.e. if SYSLOG_MIN_LEVEL is set to DEBUG and SYSLOG_ELEVATE_TO is set to NOTICE, all INFO and DEBUG messages will be sent to syslog as NOTICE). | |
0 | If this option is set to 1, chunkservers and clients connected to this master will be required to use authentication for I/O operations; this option requires the AUTH_CODE (from CHUNKSERVER CONNECTION OPTIONS section) to be set and all clients will only be able to use exports that have passwords defined (see mfsexports.cfg). |
Control connection options
Changes in this section require only process reload.
| Option | Default | Description |
|---|---|---|
* | IP address to listen on for metalogger, masters and supervisors connections (* means any). | |
9419 | Port to listen on for metalogger, masters and supervisors connections. | |
10 | Default timeout in seconds for master-metalogger connection. | |
0 | Forced timeout in seconds for master-metalogger connection (0 - do not force timeouts). |
Master-leader connection options
Changes in this section require only process reload.
| Option | Default | Description |
|---|---|---|
| PRO (PRO Edition only) | mfsmaster | MooseFS master host, IP is allowed only in single-master installations. |
| PRO (PRO Edition only) | 5 | Delay in seconds before next try to reconnect to master-leader if not connected. |
| PRO (PRO Edition only) | 10 | Timeout in seconds for master-leader connections. |
| PRO (PRO Edition only) | * | Local address to use for connecting with master-leader (*, i.e. default local address). |
Chunkserver connection options
Changes in this section require only process reload.
| Option | Default | Description |
|---|---|---|
* | IP address to listen on for chunkservers connections (* means any). | |
9420 | Port to listen on for chunkserver connections. | |
10 | Default timeout in seconds for master-chunkserver connection. | |
0 | Forced timeout in seconds for master-chunkserver connection (0 - do not force timeouts). | |
| not set | Optional authentication string; when defined, only chunkservers, other masters and metaloggers with the same AUTH_CODE are allowed to connect to this master; when not defined (default), all modules are allowed note: if you want to switch on authentication, then first define AUTH_CODE in all your chunkservers, follower masters and metaloggers (and reload/restart them) then uncomment this option and reload/restart master leader after AUTH_CODE change and master reload currently connected modules are disconnected. | |
| not set | Optional IP class remapping: if you want to use class A for communication with master and class B for I/O (communication with chunkservers) then define class A as REMAP_SOURCE_IP_CLASS and class B as REMAP_DESTINATION_IP_CLASS you must also define REMAP_BITS which means how many of first bits of class A should be changed to bits from class B; all chunkserver IP numbers that are not in class A will stay unchanged. | |
| not set | Optional IP class remapping: if you want to use class A for communication with master and class B for I/O (communication with chunkservers) then define class A as REMAP_SOURCE_IP_CLASS and class B as REMAP_DESTINATION_IP_CLASS you must also define REMAP_BITS which means how many of first bits of class A should be changed to bits from class B; all chunkserver IP numbers that are not in class A will stay unchanged. | |
| not set | Optional IP class remapping: if you want to use class A for communication with master and class B for I/O (communication with chunkservers) then define class A as REMAP_SOURCE_IP_CLASS and class B as REMAP_DESTINATION_IP_CLASS you must also define REMAP_BITS which means how many of first bits of class A should be changed to bits from class B; all chunkserver IP numbers that are not in class A will stay unchanged. | |
| not set | Optional LAN IP remapping: if you have clients in multiple LANs and want the clients communication to stay in the class of their LAN, then define those classes and MULTILAN_BITS value; first MULTILAN_BITS bits of each IP sent to the client will be changed to bits from client's IP for all client IPs that match the defined classes; all masters and chunkservers must have valid IPs from all LAN classes. | |
| not set | Optional LAN IP remapping: if you have clients in multiple LANs and want the clients communication to stay in the class of their LAN, then define those classes and MULTILAN_BITS value; first MULTILAN_BITS bits of each IP sent to the client will be changed to bits from client's IP for all client IPs that match the defined classes; all masters and chunkservers must have valid IPs from all LAN classes. | |
/etc/mfs/mfsipmap.cfg | Optional custom IP mapping. |
Chunkserver working options
Changes in this section require only process reload.
| Option | Default | Description |
|---|---|---|
60 | Initial delay in seconds before starting replications. The delay is counted from the end of the startup of the Master Server. In MooseFS PRO the delay is counted from the moment the server last changed its role, so after a failover it runs from the point the new Leader took over. | |
0 | Whether to make undergoal replications respect topology; 0 - do not respect topology 1 - pick destination server at random, but then choose best source server 2 - try to find destination server in the same rack as one of the existing copies and then replicate chunk locally (in the same rack). | |
0 | Whether new chunks should be recorded with respect to topology; 0 - do not respect topology N (N>0) - first try to create new chunks on servers with topological distance LOWER than N from the client; if not possible, for example because of storage class, chunk servers being busy or lacking space, then try servers with distance greater or equal to N. | |
0 | Avoid using same ip/rack for different chunk copies 0 - ignore ip and rackid (standard behaviour) 1 - avoid storing more than one copy on chunkservers using same IP number 2 - avoid storing more than one copy on chunkservers using IP number from the same rack id NOTICE! This parameter is available for backward compatibility purposes and should not be set to a value other than 0. Instead, distinguish feature from storage classes should be used. However, if it is set to a value other than 0, it will override any distinguish definition in storage classes. For more information about storage classes and distinguish feature refer to mfsscadmin manual. | |
100000 | Chunks loop shouldn't check more chunks per seconds than given number. | |
300 | Chunks loop shouldn't be done in less seconds than given number. | |
10 | Soft maximum number of chunks to delete on one chunkserver. | |
30 | Hard maximum number of chunks to delete on one chunkserver. When the option is not set, the limit is three times CHUNKS_SOFT_DEL_LIMIT. | |
2,1,1,4,4 | Maximum number of chunks to replicate to one chunkserver; one number is equal to five same numbers separated by commas first limit is for endangered chunks (chunks with only one copy or no extra EC parts) second limit is for undergoal chunks (chunks with number of copies lower than specified goal or with missing EC parts, also due to active MFR) plus chunks/parts on wrong labeled servers third limit is for rebalance between servers with space usage close to each other (no more than ACCEPTABLE_PERCENTAGE_DIFFERENCE*1.5) fourth limit is for rebalance between other servers fifth limit is for recovery replications caused by I/O operations (read/write) usually first number should be greater than or equal to second, second greater than or equal to third, and fourth greater than or equal to third ( 1st >= 2nd >= 3rd <= 4th ); fifth limit should be equal or greater than any of the other limits. | |
10,5,2,5,10 | Maximum number of chunks to replicate from one chunkserver; one number is equal to five same numbers separated by commas limit groups are the same as in write limit, also relations between numbers should be the same as in write limits ( 1st >= 2nd >= 3rd <= 4th ); fifth limit should be equal or greater than any of the other limits. | |
150 | Threshold for chunkserver load. | |
3.0 | Threshold ratio for chunkserver load (whenever chunkserver load is higher than previously specified threshold and this ratio times higher than average load, then chunkserver is switched into 'grace' mode). | |
900 | Defines how long chunkservers will remain in 'grace' mode (in seconds). | |
1.0 | Acceptable percentage difference between space usage of chunkservers, under that threshold the rebalance between chunkservers slows down significantly expressed in percentage points (valid values: 0.1 - 10.0) deprecates: ACCEPTABLE_DIFFERENCE (ACCEPTABLE_PERCENTAGE_DIFFERENCE = ACCEPTABLE_DIFFERENCE * 100.0). | |
1000000 | Length of priority queues (for endangered, undergoal etc. chunks - chunks that should be processed first). | |
0 | Maximum number of seconds server can be in maintenance mode (0 - which means 'forever'). | |
30m | Maximum time server can be in "temporary" maintenance mode (server is switched to this mode whenever is stopped gracefully, after reconnection server is switched back to normal mode automatically ); time can be defined as a number of seconds (integer) or a time period in one of two possible formats: first format: #.#T where T is one of: s-seconds, m-minutes, h-hours, d-days or w-weeks; fractions of seconds will be rounded to full seconds second format: #w#d#h#m#s, any number of definitions can be omitted, but the remaining definitions must be in order (so #d#m is still a valid definition, but #m#d is not); ranges: s,m: 0 to 59, h: 0 to 23, d: 0 to 6, w is unlimited and the first definition is also always unlimited (i.e. for #d#h#m d will be unlimited). | |
7 | How many days unused (disconnected) chunkserver should be kept in master data structures (valid values: 0 - 365; 0 means indefinitely ). |
Clients connection options
Changes in this section require only process reload.
| Option | Default | Description |
|---|---|---|
* | IP address to listen on for client (mount) connections (* means any). | |
9421 | Port to listen on for client (mount) connections. | |
10 | Default timeout in seconds for master-client connection. | |
0 | Forced timeout in seconds for master-client connection (0 - do not force timeouts). | |
1 | Whether MooseFS should prevent connections from clients that are unable to read all data (especially erasure encoded data); 1 - prevent connections. |
Clients working options
Changes in this section require only process reload.
| Option | Default | Description |
|---|---|---|
1d | How long to sustain a disconnected client session; time can be defined as a number of seconds (integer) or a time period in one of two possible formats: first format: #.#T where T is one of: s-seconds, m-minutes, h-hours, d-days or w-weeks; fractions of seconds will be rounded to full seconds second format: #w#d#h#m#s, any number of definitions can be omitted, but the remaining definitions must be in order (so #d#m is still a valid definition, but #m#d is not); ranges: s,m: 0 to 59, h: 0 to 23, d: 0 to 6, w is unlimited and the first definition is also always unlimited (i.e. for #d#h#m d will be unlimited). |
File system options
Changes in this section require only process reload.
| Option | Default | Description |
|---|---|---|
1w | Default grace period for soft quota; time can be defined as a number of seconds (integer) or a time period in one of two possible formats: first format: #.#T where T is one of: s-seconds, m-minutes, h-hours, d-days or w-weeks; fractions of seconds will be rounded to full seconds second format: #w#d#h#m#s, any number of definitions can be omitted, but the remaining definitions must be in order (so #d#m is still a valid definition, but #m#d is not); ranges: s,m: 0 to 59, h: 0 to 23, d: 0 to 6, w is unlimited and the first definition is also always unlimited (i.e. for #d#h#m d will be unlimited) deprecates: QUOTA_TIME_LIMIT. | |
2 | Mode of atime modification; all modes: 0 - always modify atime for files, folders and symlinks 1 - always modify atime but only in case of files (do not modify atime in case of folders and symlinks) 2 - modify atime only when it is lower than ctime or mtime and when current time is higher than ctime or mtime respectively, also modify atime when current atime is older than 24h; do it for all objects during access (like "relatime" option in Linux) 3 - same as above but only in case of files; in case of folders and symlinks do not modify atime 4 - never modify atime during access (like "noatime" option). | |
0 | Move empty files to trash after unlink? (0 - delete empty files immediately regardless of trash retention settings). | |
0 | Define amount of space reserved for superuser only; #B or # - amount defined in bytes (you can use metric prefixes - SI and IEC binary prefixes) #% or #.#% - amount defined as a percent of total space #U or #.#U - amount in multiplies of 'max bytes currently used by one of chunkservers' #C or #.#C - amount in multiplies of 'max total bytes of chunkserver - the highest capacity of one chunkserver'. | |
32767 | Define limit for number of hardlinks allowed for one object minimum value is 8 (required by POSIX) and maximum value is 65000 (same as in ext4 on Linux). | |
1d | Delay time after which inodes of deleted objects will be reused; time can be defined as a number of seconds (integer) or a time period in one of two possible formats: first format: #.#T where T is one of: s-seconds, m-minutes, h-hours, d-days or w-weeks; fractions of seconds will be rounded to full seconds second format: #w#d#h#m#s, any number of definitions can be omitted, but the remaining definitions must be in order (so #d#m is still a valid definition, but #m#d is not); ranges: s,m: 0 to 59, h: 0 to 23, d: 0 to 6, w is unlimited and the first definition is also always unlimited (i.e. for #d#h#m d will be unlimited) BE AWARE if you change this value below 1 day you MUST ensure that this value is higher than any of the following timeouts in all clients: mfsattrcacheto,mfsxattrcacheto,mfsentrycacheto,mfsdirentrycacheto,mfsnegentrycacheto,mfssymlinkcacheto. | |
8 | How many data parts should system use when old style EC definition is in use (@n instead of @8+n or @4+n). |