Ugarit
View Ticket
Login
Ticket Hash: fa1d9a51ad1238935049d759ca8f4273e8bea240
Title: Deletion from snapshots
Status: Open Type: Feature_Request
Severity: UNSPECIFIED Priority: 1_High
Subsystem: Backup_Frontend Resolution: Open
Last Modified: 2020-07-08 09:38:43
Version Found In:
Description:
Perhaps you might want to remove all snapshots older than a given number of days on a given tag. Or just remove X out of Y snapshots older than a given number of days on a given tag, to thin out the frequency of older snapshots.

We have the core support for this; just find a snapshot and unlink-directory! its contents, leaving a dangling pointer from the snapshot, and write the snapshot handling code to expect this.


alaric added on 2013-01-12 11:20:57 UTC:
A more generic approach would be to allow deleting arbitrary vault paths, resolved using globbing (see [444b6b74720b0]). Then one can remove precise subtrees as well as specific snapshots.

We'll need a delete-object! that takes a dirent; it can call delete-directory! on directory nodes, delete-file! on file nodes, etc. Deleting a snapshot should be disallowed as it'll break the chain, but deleting a tag can be allowed by deleting snapshots recursively (being mindful of refcounts, as usual).

It would be nice to detect snapshots with missing contents in the explore interface and mark them somehow, to save having to go into them to see (certainly, the "contents" dirent needs to disappear), if that can be done efficiently.

The use case of more subtle date-based snapshot deletion will need special-casing, sadly - perhaps as an extension to the glob syntax for snapshot history nodes?


alaric added on 2014-01-02 10:32:56 UTC:
There's a possible user-interface "complexity" here; if we, say, delete a snapshot (unlinking its directory tree, while keeping the snapshot node in order to preserve the chain), this should normally cause the directory root node to disappear; users browsing the tag will then see that snapshot be marked as "deleted", or just not see it at all (do we need an "ls -a" option to show deleted snapshots in a tag?), as the system can't find the directory root node.

However, if that directory root node is still referenced from elsewhere (perhaps as an archived object), unlinking it once won't kill it, and the snapshot will seem to continue to exist. But the pointer from the snapshot object is now "dangling" as the refcount has been decremented, so we can then delete the same snapshot again, and it'll really go this time - but we'll also now break the other reference to it!

So perhaps deletion of snapshots by leaving a dangling pointer from the existing snapshot is a bad idea. Do we really need to rebuild the entire snapshot chain to delete a snapshot? That sucks...

User Comments:
anonymous added on 2020-07-08 09:38:43:
I would like to raise the priority of this, because right now one of my Ugarit installations is failing because of too many snapshot entries.

I'm using the S3 backend, and just requesting the list of snapshots takes 15 minutes or so, which makes browsing the archive unusable (especially in a high stress situation of needing to restore backups).

Worse is that the vault can't be locked anymore, which is preventing new backups from being made. I'm not 100% sure but I *think* the locking problem occurs due too many files being present.  I have to debug this.

This happened after about two years of hourly archiving on my server.