Information
This article describes steps that will help prevent migration issues. Most migration issues can be prevented by keeping OS templates up-to-date and having valid repositories for all applications and OS EZ templates. The solution can be applied on an already migrated container as well.
Symptoms
A container does not start after migration. A similar error is observed:
~# vzctl start 102
Starting the Container ...
Container is mounted
Setting permissions 20002 dev 0x7d00
Set up ioprio: 4
Adding offline management to Container(1): 4643 8443
Adding IP addresses: 10.20.30.40
Error: Unable to execute bash: No such file or directory
Container is unmounted
Failed to start the Container
There are other signs of this issue, such as the inability to show properties of the files in some directory:
CT-102-bash-4.2# ls -al /usr/bin/ | grep '\?'
ls: cannot access /usr/bin/mailq: No such file or directory
ls: cannot access /usr/bin/hoststat: No such file or directory
ls: cannot access /usr/bin/praliases: No such file or directory
ls: cannot access /usr/bin/purgestat: No such file or directory
ls: cannot access /usr/bin/newaliases: No such file or directory
?????????? ? ? ? ? ? hoststat
?????????? ? ? ? ? ? mailq
?????????? ? ? ? ? ? newaliases
?????????? ? ? ? ? ? praliases
?????????? ? ? ? ? ? purgestat
CT-12264-bash-4.2#
This may affect important services start-up, for example SSH:
# /etc/rc.d/init.d/sshd restart
Stopping sshd: [FAILED]
Starting sshd: /usr/sbin/sshd: error while loading shared libraries: libpam.so.0: cannot open shared object file: No such file or directory
[FAILED]
In this case, the following command for the mounted/started container returns some output (like find: ...: No such file or directory):
~# find /vz/root/$CTID -xdev -ls >/dev/null
(Replace "$CTID" with the actual container's ID value.)
Resolution
Before performing a migration, prepare the source and destination Hardware Nodes, as well as the container that is to be migrated.
Set
SCAN_PRIVATEto "yes" to assure that thevzpackagesfile will list all packages (and their versions) installed in the container by Virtuozzo package management tools based on existing links from the container's private area. Edit/etc/vztt/vztt.confon both Nodes:~# grep SCAN_PRIVATE /etc/vztt/vztt.conf #SCAN_PRIVATE="yes" ~# sed -i 's/#SCAN_PRIVATE/SCAN_PRIVATE/' /etc/vztt/vztt.conf ~# grep SCAN_PRIVATE /etc/vztt/vztt.conf SCAN_PRIVATE="yes"NOTE: This option increases the time, required for migration. It is advised to comment it again before the migration attempt.
Sync packages for the container that is to be migrated:
~# vzpkg sync_vzpackages $CTID(Replace "$CTID" with the actual container ID value.)
It is also useful to do
vzpkg repairfor the migrated container if the migration failed before this step. Usually, this command is performed automatically during migration:~# vzpkg repair /vz/private/$CT_ID /etc/sysconfig/vz-scripts/$CTID.confStep 3 may return this error:
Error: The next package(s) are not availableIf this happens, it means that the listed packages are missing in the template area and that these packages are not available on the configured repositories -- for example, because these packages are from an outdated OS release.
For example, let's assume that our container is based on the
centos-5-x86_64template:~# vzpkg list 113 centos-5-x86_64 2012-04-16 15:13:10The error is as follows:
Error: The next package(s) are not available: samba-swat.x86_64 0:3.0.33-3.29.el5_5 samba-common.x86_64 0:3.0.33-3.29.el5_5 glibc-common.x86_64 2.5-49.el5_5.4 openldap.x86_64 2.3.43-12.el5_5.2 krb5-libs.i386 1.6.1-36.el5_5.5 krb5-libs.x86_64 1.6.1-36.el5_5.5 nss.x86_64 3.12.6-2.el5.centos glibc.i686 2.5-49.el5_5.4 nscd.x86_64 2.5-49.el5_5.4 samba.x86_64 0:3.0.33-3.29.el5_5 dhclient.x86_64 12:3.0.5-23.el5_5.1 device-mapper-multipath.x86_64 0.4.7-34.el5_5.4All missing packages above are from the Centos 5.5 release (as seen from the
el5_5part in the package version).Add Centos 5.5 repositories (other repositories for CentOS can be added in the same way, replacing "5.5" with the actual release ID):
~# echo http://vault.centos.org/5.5/os/x86_64 >> /vz/template/centos/5/x86_64/config/os/default/repositories ~# echo http://vault.centos.org/5.5/updates/x86_64 >> /vz/template/centos/5/x86_64/config/os/default/repositoriesNOTE: With many repositories in the list, the operation of metadata update (which is done every time after expiration timeout is reached, in 24 hours) takes much time to complete. It is advised to remove additional repositories when the migration is done.
After adding/removing repositories, update metadata on the server for the EZ OS template:
~# vzpkg update metadata centos-5-x86_64 -rAfter that,
vzpkg repairshould be able to locate the needed packages.Some packages do not contain the release ID. For such packages, it is necessary to check all available repositories manually. Examples of such packages are
httpdandmod_ssl.Digital suffixes, like
.2in.el5_5.2version, mean that such versions are released in updates to the given release.NOTE: The instructions above work for Virtuozzo, Virtuozzo hypervisor 5.0, and Virtuozzo containers for Linux 4.6, 4.7 with containers based on EZ OS templates.
For previous versions, these instructions work only for Virtuozzo hypervisor 4.0 (latest release) and Virtuozzo containers 4.0 starting from 4.0.0-520.
The commandvzpkg repaircan be used only if it is possible to create new containers based on the given OS template.Sometimes it is not possible to find a mirror with missing packages, collected on the third step. In this case, copy the missing files from the source node:
~# rsync -a --ignore-existing --rsh="ssh" root@${SOURCE_NODE}:/vz/template/centos/5/x86_64/ root@${DESTINATION_NODE}:/vz/template/centos/5/x86_64/NOTE: The
ignore-existingoption is essential in this command, since the files, already used by the running containers, must not be updated.