Symptoms

During a backup procedure for a large Parallels Virtuozzo Containers (PVC) container (amount of inodes > 100000) with VZFS file system, the backup process fails in 10 hours with the following error message:

[root@host ~]# vzabackup -F localhost -e 101 
... 
 Backup operation for node "host.server.com" failed: Failed to perform the file backup 
 Acronis Error: Module=0 Code=0, Tag=0, 
 Backup failed 

Backup task in Virtual Automation Management Node fails with the following error message:

 #307 Message processing was terminated because timeout was reached waiting for a response

Cause

The Parallels Virtual Automation (PVA) backup process speed is affected by the number of inodes inside of a container. For containers with 1 million inodes, the backup may take up to 2 days, and if the timeout period is not long enough, the operation will fail.

Resolution

  1. Recommended: Update the PVA Agent on the node to the latest available version. Available versions and update instructions can be found in this article.
  2. Recommended: Move the container to a Parallels Cloud Server hardware node and convert to 'ploop' format. Due to their nature, ploop containers are backed up much faster than VZFS container.

  3. Review container data and remove unnecessary files.

    It is possible that the data, impacting the backup time, is not important and can be excluded from the backup: mail files, PHP sessions, old logs, etc.

    A common scenario inflated /var/lib/php/session directory:

    [root@vz ~]# find /vz/private/101/fs/root/var/lib/php/session | wc -l
    6842890
    

    Check the following article for more information on how to resolve such issues on Plesk containers.

  4. If it is still required to create a backup of such a big amount of files, no matter how much time it takes, backup timeouts can be adjusted:

    • Virtualization Node - the node being backed up.

      1. Edit the PVA Agent configuration file in one of the following locations:

        • /vz/pva/agent/etc/vzagent.conf
        • /var/opt/pva/agent/etc/vzagent.conf
        • /vz/private/1.sve***/vzagent/etc/vzagent.conf (for pre-PVA nodes)
      2. Change the heavy timeout limit value on the line with <heavy_timeout_limit>:

        • vzlin_backup_serializer (for pre-PVA 4.6.4 nodes):

          <vzlin_backup_serializer>
          <configuration>
          <backend>0</backend>
          <enable_snapshot>1</enable_snapshot>
          <custom>
          <queue>0</queue> 
              <heavy_timeout_limit>360000</heavy_timeout_limit>
          </custom> 
          </configuration> 
          </vzlin_backup_serializer> 
          
        • vzlin_backup_serializer11 (for PVA 4.6.4 and later versions):

          <vzlin_backup_serializer11> 
          <configuration> 
          <backend>0</backend> 
          <enable_snapshot>1</enable_snapshot> 
          <custom> 
          <pool>0</pool> 
          <queue>0</queue> 
              <heavy_timeout_limit>360000</heavy_timeout_limit>
          </custom> 
          </configuration> 
          </vzlin_backup_serializer11> 
          
      3. Edit /opt/pva/agent/plugins/VZABackupLibClient/VZABackupLibClient.conf, changing the value for <heavy_timeout_limit>:

        <vzlin_backup_serializer11>
        <configuration>
        <backend>0</backend>
        <enable_snapshot>1</enable_snapshot>
        <custom>
        <pool>0</pool>
        <queue>0</queue>
            <heavy_timeout_limit>360000</heavy_timeout_limit>
        </custom>
        </configuration>
        </vzlin_backup_serializer11>
        
      4. Edit /opt/pva/agent/plugins/VZLBackupManager/VZLBackupManager.conf, increasing the value for <backup>:

        <packet>
        <data>
        <backup_storagem>
        <configuration>
        <timeouts>
        <notify>30</notify>
        </timeouts>
        </configuration>
        </backup_storagem>
        <backupm>
        <configuration>
        <timeouts>
            <backup>360000</backup>
        </timeouts>
        </configuration>
        </backupm>
        </data>
        </packet>
        
      5. Restart PVA Agent:

        ~# pvaagent restart
        
    • Backup Storage Node - the node the backup is stored to.

      1. Edit the PVA Agent configuration file:

        • /vz/pva/agent/etc/vzagent.conf or
        • /var/opt/pva/agent/etc/vzagent.conf.
      2. Change the heavy timeout limit value:

        • backup_deserializer (for pre-PVA 4.6.4 nodes):

          <backup_deserializer> 
          <configuration> 
          <port>4435</port> 
          <custom> 
          <queue>0</queue> 
              <heavy_timeout_limit>360000</heavy_timeout_limit>
          </custom> 
          </configuration> 
          </backup_deserializer>
          
        • backup_deserializer11 (for PVA 4.6.4 and later versions):

          <backup_deserializer11> 
          <configuration> 
          <port>4435</port> 
          <custom> 
          <queue>0</queue> 
              <heavy_timeout_limit>360000</heavy_timeout_limit>
          </custom> 
          </configuration> 
          </backup_deserializer11>
          
      3. Restart PVA Agent:

        ~# pvaagent restart
        
    • Control Node - the node the backup process is initiated from.

    It can be the PVA Management Node, Virtualization Node, or Backup Storage Node.

    1. Edit the PVA Agent configuration file:

      • /var/opt/pva/mn/etc/vzagent.conf or
      • /vz/pva/agent/etc/vzagent.conf or
      • /var/opt/pva/agent/etc/vzagent.conf.
    2. Change the backup timeout limit value:

      • backupm:

        <backupm> 
        <configuration> 
        <timeouts> 
            <backup>360000</backup>
        </timeouts> 
        <backupm_config> 
        
    3. Restart the appropriate PVA service:

      • ~# pvaagent restart (for PVA Agent on a Virtualization Node) or
      • ~# pvamn restart (for PVA MN on a Management Node).
NOTE: If there is a PVA Management Node (PVA MN), and both the Virtualization Node and the Backup Storage Node are registered in the same PVA group, the easiest way to configure timeouts is to run reconfiguration from PVA MN:
~# vzagroup reconfigure /data/backupm/configuration/timeouts/backup 360000
~# vzagroup reconfigure /data/vzwin_backup_serializer/configuration/custom/heavy_timeout_limit 360000
~# vzagroup reconfigure /data/vzlin_backup_serializer/configuration/custom/heavy_timeout_limit 360000
~# vzagroup reconfigure /data/vzlin_backup_serializer11/configuration/custom/heavy_timeout_limit 360000
~# vzagroup reconfigure /data/backup_deserializer/configuration/custom/heavy_timeout_limit 360000
~# vzagroup reconfigure /data/backup_deserializer11/configuration/custom/heavy_timeout_limit 360000

In addition you also need to edit configure limits in /opt/pva/mn/plugins/VZLBackupManager/VZLBackupManager.conf:

<packet>
<data>
<backup_storagem>
<configuration>
<timeouts>
<notify>30</notify>
</timeouts>
</configuration>
</backup_storagem>
<backupm>
<configuration>
<timeouts>
    <backup>36000</backup>
</timeouts>
</configuration>
</backupm>
</data>
</packet>

Also see:

113790 PVA Backup: Servers, roles, and processes.

113801 Unable to back up container with old layout: Acronis Error: Module=0 Code=0, Tag=0.

113802 Unable to back up large container: Acronis Error: Module=0 Code=0, Tag=0.

113794 Unable to back up containers on LVM over iSCSI.

NOTES:

The error can be shown in case when there is no free space on backup server. In such case please free some space and repeat the operation.

Internal content

Link on internal Article