CloudBlue Technical Support

Symptoms

Odin Automation has been upgraded to 7.3.

The environment includes Web Hosting Plesk and Virtualization (Virtuozzo Containers) modules.

Task Manager is filled with tasks

Execute operation 'applyNodeChanged' with ID bfbc4427-aa63-4e62-9567-5d8caaf2b5d7 on resource c2899a83-4778-4c3b-a322-24a8f74eeb29. Additional info: Starting attaching of the (10101) Service Node.

Tasks are stuck in Running state for 45 minutes and get rescheduled or failed afterwards with errors like:

The remote server is down, details:
system exception, ID 'IDL:omg.org/CORBA/OBJECT_NOT_EXIST:1.0'
Unknown vendor minor code id (0), minor code = 0, completed = NO

or:

Internal error: SDK::Platform::blob Plesk::SaaS::SaaSManagerTasks_impl::taskCustomOperationAsyncExec(const SDK::Platform::Properties&) : [APSC] Timeout.

Cause

An upgrade action that propagates all OA hosts to APS bus triggers a flood of tasks, resulting in logical deadlocks. The behavior is improved in scope of WHM-10423.

Resolution

A fix for the issue is to raise the amount of Apache workers on the Web Hosting Plesk Management Node:

• Edit /etc/httpd/conf.d/mpm_prefork.conf

• Increase the following parameters to 50:

  ServerLimit       10
  MaxClients        10
  

  Note: the workaround requires that the Web Hosting Plesk node is configured with the recommended amount of RAM (32 GB or more).

Once the parameters are increased, restart the Apache service:

# systemctl restart httpd

It is highly recommended to roll back the changes after all applyNodeChanged tasks are successfully completed.

The permanent fix will be included into one of the future versions of Odin Automation.

Internal content