Symptoms
Backup creation task fails with the following error message:
-1 : vzquota : (error) Can't open quota file for id 2472, maybe you need to reinitialize quota: No such file or directory
The following error messages can be seen in the log file /var/log/pva/agent/yyyy.mm.dd-vzagent.log on the node with a container around the time of the failed backup:
T=01:38:24:832; L=(err); PID=224142; TID=7f6a991ae760; P=VZLHaulControlAgent [create] [VZLHaulSerializerBaseAgentPrototype::create] endCommandWithAnswer failed (2, 2) Target backup_deserializer11 is not found. . Target backup_deserializer11 is not found.
T=01:38:24:832; L=(wrn); PID=224142; TID=7f6a991ae760; P=VZLHaulControl [start] m_deserializer->create() failed: Unable to perform remote operation. The target host does not have the necessary PVA components installed or is running an incompatible operating system.
T=01:38:24:832; L=(wrn); PID=224142; TID=7f6a991ae760; P=VZLHaulControl [start] Process finished with code 3302. Unable to perform remote operation. The target host does not have the necessary PVA components installed or is running an incompatible operating system.
Or the following messages can appear on the backup node, pointing to lack of free heavy operators:
T=00:45:00:619; L=(wrn); PID=7460; TID=7f0929454760; P=VZLControl [getFreeTransport] [getFreeTransport] backup_deserializer11: Heavy transports exceeded: 1 of 1
T=00:45:00:619; L=(wrn); PID=7460; TID=7f0929454760; P=VZLControl [logDroppedMessage] backup_deserializer11: Message (id: efk26afly1gaa5fvbpum5tjox2g1gaaa) was dropped
T=00:45:00:659; L=(wrn); PID=7460; TID=7f0929454760; P=VZLControl [getFreeTransport] backup_deserializer11: Transports exceeded: 1 of 0
T=00:45:00:659; L=(wrn); PID=7460; TID=7f0929454760; P=VZLControl [logDroppedMessage] backup_deserializer11: Message (id: e3nabxmly1gaa5fvbpuoaiv2gzg1gaaa) was dropped
T=00:45:00:693; L=(inf); PID=614258; TID=7f9248347760; P=VZLOperatorCommon [createCmd] [VZLHaulControlOperatorPrototype::createCmd] Creating local serializer.
T=00:45:00:693; L=(inf); PID=614258; TID=7f924156e700; P=VZLConnection [LoopThreadProc] New Event Loop Thread has been started, see thread ID
T=00:45:00:713; L=(inf); PID=614258; TID=7f9248347760; P=VZLOperatorCommon [initCmd] [VZLHaulControlOperatorPrototype::initCmd] initializing
T=00:45:01:587; L=(inf); PID=614258; TID=7f9248347760; P=VZLOperatorCommon [doStageCmd] [doStageCmd] Processing stage prepareBackup
Cause
One of the following may lead to issue:
PVA Agent on the backup node is outdated
- The limit of heavy operators is reached on the backup node.
In both above mentioned cases during a backup operation, the PVA Agent on the node with the container being backed up uses backup_deserializer11 (the latest version of backup back-end), while the storage node uses the old version of backup back-end, that does not support ploop containers.
- There is not enough of free space on the drive where backups are stored.
Resolution
For correct backup of ploop containers the storage node should run the PVA 6 Agent. The version 6.0-2160 changes the limit of heavy operations to 20 among other fixes and improvements.