Deletion: from unlink to space returned
rm returns at once. The space does not. Between the unlink and free disk space, four independent mechanisms run, each with a clock of its own. The mechanisms are the trash, the sustained area, the Master Server's chunk loop, and the deletion orders that the Chunkservers execute. This page describes what happens to a file in all four. The page ends with the progress that the cluster reports itself, because no fixed number says how long the whole chain takes.
A file with retention 0 that nobody holds open never reaches the trash: the unlink destroys it on the spot.
Destruction frees metadata, not space: the chunk loop finds the unused chunks on its next pass and orders the copies deleted, at CHUNKS_SOFT_DEL_LIMIT per Chunkserver (10, adapting up to the hard limit). Chunkservers unlink at once. The free bytes reach df with the next space report.
A chunk shared with a snapshot keeps its remaining references and is not deleted.
An unlink removes a name, not the file
Unlinking removes one name of a file. Deletion begins only when the last name is removed. Even then, the file's data usually remains after the call:
- A file with a trash retention above zero is not destroyed. Its node becomes an entry in the Global Trash and keeps its chunk table. The Master Server sends no message to any Chunkserver. The Master Server reads the retention first, so a file that a client still holds open goes to the trash as well. The entry, its expiry, the exception that an empty file makes, and what the entry costs in RAM until then are covered in trash retention.
- A file with a retention of zero that a client still holds open is not destroyed either. Its node moves to the sustained area and stays there until no session holds the file open. A trash entry whose retention runs out while a client holds the file open moves to the same area. Trash retention covers both routes into the sustained area.
Neither mechanism converts anything while it holds the file. A waiting file keeps its Storage Class. A deleted file is never archived. Only a trash entry reaches the TRASH storage state. Unless the file's class has a TRASH definition of its own, the chunks of that entry stay under the KEEP or ARCHIVE definition that already held them. This happens because Storage Class evaluation skips a state that the class does not define. The chunks of a sustained file never enter TRASH at all, regardless of what the class defines. A file that passes from the trash into the sustained area leaves that state behind. Converting chunks that are about to disappear would be wasted work. A class that does not define TRASH sets no rule for how deleted files are held.
A file with a retention of zero that nobody holds open is destroyed immediately. Destruction is the same step wherever the file enters it: at the unlink itself, when a trash entry expires, or when the last holder of a sustained file releases that file.
Destroying the file frees metadata, not space
Destruction removes the file's node from the Master Server's RAM. Every chunk of the file drops its reference to that node. The chunk table is freed, and the inode number is released. That is all. No message goes to any Chunkserver, so every copy and every Erasure Coding part of every chunk is still on disk, exactly where it was.
A chunk that no file refers to any longer stays in the Master Server's chunk registry as an unused chunk. A chunk that the file shared, through a snapshot or a duplicated tree, keeps its other references and is not unused. Only the chunks that this file held alone become work for the next mechanism.
The chunk loop orders deletions against a cap
Nothing looks for unused chunks on demand. The chunk loop (the Master Server's periodic pass over every registered chunk) finds them. When the loop reaches a chunk that no file refers to, the loop orders every remaining copy and part of that chunk deleted. Two delays follow from that design.
The first delay is the pass itself. A chunk that became unused right after the loop passed it waits close to a full pass before the loop reaches it again. The loop's fixed budget sets how long a pass takes. Replication priorities describes that budget. The size of the deletion backlog does not shorten the pass.
The second delay is the deletion cap. CHUNKS_SOFT_DEL_LIMIT limits how many deletion orders may be outstanding for one Chunkserver. The default is 10. An unused chunk whose orders do not fit under the cap is left for a later pass. The cap adapts to a growing backlog, up to CHUNKS_HARD_DEL_LIMIT. The adaptation is described in replication priorities. Once the orders are confirmed and no copy remains, the loop removes the chunk's record. The chunk then no longer exists in the registry.
A Chunkserver unlinks at once, reports later
A Chunkserver that receives a deletion order unlinks the chunk file immediately. Nothing waits in a local queue. Chunkserver housekeeping loops describes that behavior. The gradual progress of a large deletion comes entirely from the Master Server's cap, not from the Chunkservers.
The duplicate chunk files that a Chunkserver sets aside for HDD_KEEP_DUPLICATES_HOURS are not part of this chain. These files come from disk scans, not from deletion orders. Chunkserver housekeeping loops covers that mechanism.
The freed bytes reach df through the space reports. The Chunkserver's disk watcher re-reads a disk's free space promptly after a chunk was deleted on that disk. The watcher sends the changed totals to the Master Server on its next one-second check. The Master Server answers a mount's statfs with the sum of those reports. The space is available in df again when the Chunkservers have reported after their unlinks.
The loop's progress is the only estimate offered
No rule converts an amount deleted into minutes, because the chain above has too many independent clocks. Instead, the cluster reports its own progress. The Master Server keeps the start and end times of the last completed chunk loop pass. The Master Server tracks how far the current pass has come and estimates the time to the end of the pass. The Master Server also counts what each pass did. Among those counters are the deletions that the Master Server ordered for chunks that no file needs anymore. mfscli prints all of these numbers in its self-check loops report. The chunk matrix summary of the same tool counts the chunks awaiting deletion as pending deletion. The syntax is in command line tools.
Read those numbers on your own cluster: how long the last pass took, how far the current pass has come, and how many deletions each pass placed under the cap. Together with the retention of the file in question, these numbers give a better bound on the wait than any general figure could.