Summary

Operations Automation (formerly POA) 5.5 Upgrade procedure - Known Issues

Issues List

  • Unforeseen error occurs on the attempt to open the task dashboard during VPS upgrade. The only UI working during VZ templates upgrading is UI placed on MN. To get working other UI (placed on slave), one needs to restart pemui service manually. POA-78630.
  • Upgrade of package "myLittleAdminFullVersion" to version 3.7.2 may fail after upgrade with the following output "Service package MyLittleAdmin cannot be upgraded from 2.7 to 3.7 directly". It's safe to cancel the task. POA-73325.
  • WPE host may be rebooted during "WindowsProvisioningEngine" package upgrade. POA-79204.
  • Vendors resources propagation may take hours/days (depending on number of resources owned by vendors).Issues in JIRA: APS-13151, APS-12964. POA 5.5.4 HOTFIX 119995 APS should be installed with 5.5.4.
  • Tasks upgrading Apache subscriptions to new resource model may be running for hours/days (depending on number of apache subscriptions on the installation. Issues in JIRA: POA-79565, APS-13611.
  • Updater may stop execution on 'post' stage if some VZ hosts are inaccessible (the scheduled task fails in this case). All the scheduled tasks "Update List of templates installed to HW Node : NNN" should be processed before upgrade continuing. Operations Automation UI is not running at the moment, one needs to launch it manually if needed. Issues in JIRA: POA-79568 + POA-79625.
  • Unable to open a financial document from Operations Automation CCP in case Operations Automation 5.5.x is integrated with Business Automation 5.4.15 or lower. Issue is fixed as described in _place_holder;KB 116635
  • Task "Install other-whosond-2.05-2 (pkgid = xxx) on whoson.provider.com (host_id = yy)"_ fails with erro_r "Native dependency processing failed; list of dependences: whoson >= 2.05-2:whoson-server >= 2.05-2. Please either install the packages above, or application templates with them. Error: Transaction Check Error: file /etc/init.d/whoson from install of whoson-server-2.05-2.parallels.x86_64 conflicts with file from package whoson-server-2.05-1.parallels.i386". Reason: there are both i386 and x8664 whoson packages installed on this host. So yum could not upgrade whoson rpms. [WA] "rpm -e whoson-server" and restart the task. POA-79147.
  • During Operations Automation 5.5.1 installation action 147070-POA-80261.WinFileManager uninstalls service WinFileManager. New service package WebFileManager should be installed to host with package msii7 after Operations Automation 5.5.2.
  • Post-update task "Grant WebFileManager-service" that applies permissions for Application pool identity of a FileManager site to 'CustomerData' folder on a Windows shared host can fails by timeout. Such task can take much time depending on number of web sites and files inside websites folders. Resolution is to increase task timeout and re-run the task KB 115550
  • Clustered control panel is not accessible because pulse service malfunction. Resolution: service pulse restart on load balancer. POA-83207
  • 13950_POA69479_Postgres_9_0_to_9_1_migration fails with collate error like in precheck POA-81853. W/A: Add UTF-8 locale settings to initdb in /etc/init.d/postgresql-9.1

    $SU -l postgres -c "$PGENGINE/initdb --pgdata='$PGDATA' --auth='ident' $LOCALESTRING --lc-collate='en_US.UTF-8' --lc-ctype='en_US.UTF-8' --encoding='UTF8'" >> "$PGLOG" 2>&1 < /dev/null
    
  • 13950_POA69479_Postgres_9_0_to_9_1_migration fails with error below in case of PostgreSQL DB located on remote host without yum installed:

    2013-12-15 02:12:58.759 ERROR exceptions.Exception Command 'ssh -T root@hostname 'yum -y install postgresql91-server'' exited with non-zero status
    

    W/A is to install and configure yum manually on remote DB host.

  • 13950_POA69479_Postgres_9_0_to_9_1_migration removes all host access records with method trust from pga_hba.conf file. Such behavior can break customer functionality that use access to Operations Automation DB from other hosts (for example see 'Synchronizing Changes Applied in Microsoft Online Services Portal to Service Automation'). As a W/A one may need to backup manually added host access records from pga_hba.conf and restore them after upgrade.
  • Updating of msi packages on W2K3 is incompatible with MS kb2918614 MS14-049. POA-88031. W/A: MS kb2918614 MS14-09 should be uninstalled before update. One could install it back after update completion.
  • 13892-POA-77291-0-2.Apache fails with error "Subscription #nnnnnnn has no resource type #rrrrrrrr". POA-87101 W/A: add required resource to subscription using pem.setResourceTypeLimit.
  • Upgrade action 147902-POA-85130 fails POA-86406:

    2014-06-17 05:14:02.920  INFO [post] ---> executing atomic action 147902-POA-85130 owner WebCluster (r)etry/abort/ignore/(c)onsole:  (r)etry/abort/ignore/(c)onsole: (r)etry/abort/ignore/(c)onsole:  (r)etry/abort/ignore/(c)onsole:  (r)etry/abort/ignore/(c)onsole:  (r)etry/abort/ignore/(c)onsole:  r
    2014-06-17 05:15:45.732  WARN Retrying action executing atomic action 147902-POA-85130 owner WebCluster ...
    (r)etry/abort/ignore/(c)onsole:  r
    

    W/A: skip this action (ignore), run manually commands for all redis config files (redis.*conf) in /etc

    # chmod 0600 /etc/redis.conf
    # chown redis:redis /etc/redis.conf
    
  • Filemanager on webcluster does not work after upgrade from version 5.4 to 5.5. This issue happens if NG cluster was deployed in 5.4; in such situations LB is not configured for processing (forwarding) port (1299) for WebFileManager. W/A:

    1. Perform packet filtering (all commands executed on LB node):

      iptables --list --line-numbers
      Chain INPUT (policy ACCEPT)
      num  target     prot opt source               destination         
      1    ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           state RELATED,ESTABLISHED 
      2    ACCEPT     icmp --  0.0.0.0/0            0.0.0.0/0           icmp type 255 
      3    ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpt:20 
      4    ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpt:21 
      5    ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpt:2021 
      6    ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpt:22 
      7    ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpt:80 
      8    ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpt:9113 
      9    ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpt:9114 
      10   ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpt:443 
      11   ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpts:60000:65535 
      12   ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           udp dpt:8649 
      13   ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpt:8649 
      14   ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           udp dpt:8652 
      15   ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpt:8652 
      16   DROP       udp  --  0.0.0.0/0            0.0.0.0/0           
      17   DROP       tcp  --  0.0.0.0/0            0.0.0.0/0           tcp flags:0x17/0x02
    2. Run the following commands to add the port manually:
      iptables -D INPUT 16
      iptables -D INPUT 17
      iptables -A INPUT -p tcp -m tcp --dport 1299 -j ACCEPT
      iptables -t mangle -A PREROUTING -m state --state NEW -p tcp -m tcp --dport 1299 --dst ! 201.31.6.10 -j MARK --set-mark 100
      iptables -A INPUT -s 0/0 -d 0/0 -p udp -j DROP
      iptables -A INPUT -s 0/0 -d 0/0 -p tcp --syn -j DROP
      service iptables save
  • After upgrade from version 5.4 to 5.5, SaaS web applications and web spaces created by the provider may be affected by an error when the "memory_limit" NG resource for webspaces is not sufficient to execute PHP scripts of the website, which results in "Internal Server Error".

    This happens because resource limits of the provider's subscription are not modified during the upgrade. The apache_h2e_ws_limits table contains old limits. This issue affect the 'Bandwidth limit' and 'Memory consumption limit' resources of the provider's subscription if they are not unlimited. W/A:

    1. Set required value for memory_limit in the apache_h2e_ws_limits table:

      BEGIN;
      update apache_h2e_ws_limits set memory_limit = 524288 where homedir_id=XXXXXXXXX;
      COMMIT;
    2. Update the web hosting settings for the required webspace.

      • In Customer CP, go to Web Hosting > website (homedir_id) > Web Hosting Settings.
      • Click Edit > Submit.

      This will create a task and update the new limits for the webspace.

Internal content

Link on internal Article