Symptoms

When I list the files inside a ploop container's hard drive directory (/vz/private/CT_ID/root.hdd/), there are several ".hds" files that indicate the container has one or more snapshots:

~# du -csh /vz/private/101/root.hdd/*
4.0K    /vz/private/101/root.hdd/DiskDescriptor.xml
0       /vz/private/101/root.hdd/DiskDescriptor.xml.lck
577G    /vz/private/101/root.hdd/root.hds
147G    /vz/private/101/root.hdd/root.hds.{c6fd60f3-6da5-47a8-975a-0e14b06d95ad}
44K     /vz/private/101/root.hdd/templates
724G    total

However, the snapshots of this container cannot be listed:

~# prlctl snapshot-list 101
PARENT_SNAPSHOT_ID                      SNAPSHOT_ID

As a result, it is not possible to delete the snapshots using the command prlctl snapshot-delete.

How do I delete these snapshots?

Cause

The fact that a snapshot is not listed is the result of an empty dump folder inside the container's private area.

Below is an example of usual snapshot creation:

~# prlctl snapshot 101
...
The snapshot with ID {08ddd014-7d57-4b19-9a82-15940f38e7f0} has been successfully created.

A newly created snapshot is saved to the /vz/private/<CT_ID>/dump/<snapshot_ID> file, where <CT_ID> is the Container ID and <snapshot_ID> is a snapshot ID.

In the above example, the snapshot with the ID {08ddd014-7d57-4b19-9a82-15940f38e7f0} is saved to the file /vz/private/101/dump/{08ddd014-7d57-4b19-9a82-15940f38e7f0}.

~# ls /vz/private/101/dump
{08ddd014-7d57-4b19-9a82-15940f38e7f0}

Snapshot IDs are required to list, switch to, and delete snapshots.

As a result, if the /vz/private/<CT_ID>/dump folder is empty for some reason, you will not be able to list the snapshots of this container.

~# ll /vz/private/101/dump/
total 0

Resolution

Solution 1

In order to get rid of these unlisted snapshots, use the prl_disk_tool utility (which works with the "hdd" file directly). prl_disk_tool should be executed on a stopped container.

IMPORTANT: Before deleting a snapshot, please, make sure you have enough free space on the host (this should be greater than the total size of all snapshots). If you do not, the operation will not be completed and this may lead to container corruption.

~# prl_disk_tool merge --hdd /vz/private/101/root.hdd
merge snapshot {5fbaabe3-6958-40ff-92a7-860e329aab41} -> {4ab2f4c8-e459-4355-ab76-e5a69b9f7efb}
Opening delta /vz/private/101/root.hdd/root.hds.{d29fd925-9dc3-4abd-8cb2-53d0a17ccace}
Opening delta /vz/private/101/root.hdd/root.hds
Storing /vz/private/101/root.hdd/DiskDescriptor.xml
Removing /vz/private/101/root.hdd/root.hds.{d29fd925-9dc3-4abd-8cb2-53d0a17ccace}
ploop snapshot {5fbaabe3-6958-40ff-92a7-860e329aab41} has been successfully merged

(prl_disk_tool is not selective by design - it will delete all snapshots of the specified container.)

Solution 2

As an alternative solution, a backup and restore of the container can be used for the same purpose.

The backup operation creates an archive of the current container data and does not include snapshots. Therefore, after restoration, the resulting container will not have any snapshots.

~# prlctl stop 101
~# prlctl backup 101
~# prlctl restore 101

Internal content

In fact, such snapshot can be listed with "ploop" utility:

    [root@vzd188 ~]# ploop snapshot-list /vz/private/115303/root.hdd/DiskDescriptor.xml
    PARENT_UUID                            C UUID                                   FNAME
    {00000000-0000-0000-0000-000000000000}   {4e96ccfc-f5ff-4f1a-8eef-1db0df5be15b} /vz/private/115303/root.hdd/root.hds
    {4e96ccfc-f5ff-4f1a-8eef-1db0df5be15b} * {5fbaabe3-6958-40ff-92a7-860e329aab41} /vz/private/115303/root.hdd/root.hds.{71dec0c0-f8e9-4f93-81e8-7e08239be1c1}

Confusion here is in fact that we have CT snapshots and Ploop snapshots.

'vzbackup' creates CT snapshot (AFAIR), vzabackup and 'prlctl backup' create only ploop snapshots. So if backup has been interrupted somewhat incorrectly, Ploop shapshot can be left unmerged. The fix for orphaned snapshots has been prepared in scope of PSBM-24727 and PSBM-24873

So 'prlctl snaphot-list' will never show snapshots from vzabackup and prlct backup.