Symptoms
There are failed tasks for VM backups in customer CP.
The tasks fail with timeout with no callback received from vm2vf daemon. In /var/log/IM/PACI-im.log
the following error is found:
2015-11-23 15:21:19,589 () ERROR Vm2VfProviderBase [VM2VF callbacks watchdog thread] - No registered callback found: ActiveCallbackId [timestamp=2015-11-22 23:21:19.484, reqId=162737, corbaReqId=927068, info=backup ve [{246853d3-b2e6-4758-afb1-150bbcb3bc9c}] to the node [192.0.2.2], contextId=(95fd9507-9dd8-4ede-9502-631beafb9c85)]
VM is located on a node with Virtuozzo 6.0 Update 10 installed and has a large size (more than 100GB).
The backup operation itself finished successfully in the background, and the backup can be listed on the backup node via prlctl backup-list
command.
Cause
Virtuozzo product issue PSBM-41302: huge VM backups last extremely long.
Resolution
The fix for the Virtuozzo issue is available since Virtuozzo Update 10 hotfix 7.
In case the updates installation had been postponed, as a workaround, the task timeout can be increased per this article.
Note: the backups created in scope of these failed tasks should be cleaned up manually on the backup nodes in order to free up the disk space.