Symptoms
There are some failed tasks in the background task queue; some POA provisioning operations are stuck indefinitely, and related objects are out of sync (updating, creating, deleting, etc.).
Cause
PSA manages its hosts and services asynchronously using the Task Manager. Almost any operation, such as provisioning, updating the configuration, or removing provisioned services, triggers task scheduling in the Task Manager. Scheduled tasks are grouped by queues and can have dependencies that need to be executed in a certain order.
Sometimes, tasks can fail due to network, hardware, or software issues. In such cases, failed tasks usually need to be rescheduled manually to finish pending operations.
If a task has failed, it blocks dependent tasks. To process all of them, the first blocker tasks must be finished successfully. We do not recommend canceling background tasks, as this can produce various inconsistencies.
It is important to check if there are any failed tasks and then process them in order to avoid stalled operations and out-of-sync statuses.
Resolution
The list of all scheduled tasks can be checked here (this path is available under the provider's control panel only):
Top > System Director > Task Manager > Background Tasks.
The general way to fix a failed task is as follows:
- Check why the task has failed.
- Fix the issues found in step 1.
- Re-run the task.
- Return to the first step if needed.
How to see why a task has failed:
Open Background tasks and click on a failed task name. Check the Last execution output field for an error (the most common errors are described below). If the reason for failure is unclear, it is a good idea to re-run the task to see if it is a permanent error.
How to re-run a task:
Open Background tasks and select failed task(s) by marking the corresponding check-box(es). Click the Run tasks button (only selected tasks will be run).
How to check task dependencies:
Open Background tasks and click on a task name. On the task details screen, two extra tabs are shown: Tasks this task blocks and Tasks this task depends on. Check the corresponding tab to see blocker or dependent tasks.
Common error messages:
Task name:
any
Error message:
PEM node is unmanageable, host_id=_<host_id>_.
Related article: #2258
Task name:
Delete webspace's AD users and groups
Error message:
Exception during execution of MPF request: "The file exists. [<response><errorContext description="The file exists. Object: Removing disk quota for user WEBHOSTING\i00133342" code="0x80070050" executeSeqNo="2"><errorSource namespace="NTFSManager" procedure="clearUserQuota" /></errorContext></response>]"
Related article: #1914
Task name:
any virtuozzo related
Error message:
ExSystem: ERRCODE 5000 ERRCODE_MINOR -4 VZALib error, code: -4, msg: Connection to vzagent was closed
Related article: #3012
Task name:
task for some VPS
Error Message:
ExSystem: ERRCODE 5000 ERRCODE_MINOR 1902 VZALib error, code: 1902, msg: Error parsing processes info for VE $VEID..
Related article: #3022
Task name:
Initialize mailbox "_<Username>_" (id=_<id>_)
Error message:
Cannot fetch URL: "http://192.168.1.123/exchange/", HTTP error: "Unauthorized" (HTTP error code: 401)
Related article: #3073
Task name:
Wait for objects to be replicated to all domain controllers (domain=hosting.local)
Error message:
Some of objects are still not replicated to all active domain controllers. AD replication may take up to several minutes. You have to troubleshoot AD replication only if this task has finally failed (all attempts have failed).
Related article: #3086
Task name:
Operation with FrontPage Server Extentions on host with subds_id=_<id>
Error message:_
FrontPage Server Extentions are not installed on host.
Related article: #3096
Task name:
Create Windows SiteBuilder site "<sitename>"
Error message:
Axis call has failed, message is Server was unable to process request.
---> Exception of type "SWsoft.SiteBuilder.BO.Facade.PlanLimitException" was thrown., code is 71
Related article: #3090
Task name:
Update Apache domain service <ds_id> at <webserver>
Error message:
ExSystem: ERRCODE 0 ERRCODE_MINOR 0 OpMkDir: /usr/local/pem/vhosts/<hd_id>/webspace/httpdocs/<DocumentRoot>: Not a directory
Related article: #3091
Task name:
_Create virtual web server <srv_id> on host "<hostname>" (host_id=<host_id>)
Error message:
Exception during execution of MPF request: "The system cannot find the path specified. [<response><errorContext description="The system cannot find the path specified." code="0x80070003" executeSeqNo="24"><errorSource namespace="File System Provider Ex" procedure="createBinaryFile" /></errorContext></response>]"
Related article: #3088
Task name:
Disable domain service (ds_id=<ds_id>, hd_id=<hd_id>) on host "<hostname>" (host_id=<host_id>)
Error message:
DS not found (id = 1234)
Related article: #3466
Task name:
Create contact "<contact_name>" (id=<contact_id>)
Error Message:
Exception during execution of MPF request: "Unknown error 0x80131600 [<response><errorContext description="This task does not support recipients of this type. The specified recipient hosting.local/Hosting/Provider/S000<sub_id>/<contact_name> is of type MailContact. Please make sure that this recipient matches the required recipient type for this task. Cmdlet="Enable-MailContact"
DomainController="DC01.hosting.local"
Identity="LDAP://CN=<contact_name>,OU=S000<sub_id>,OU=Provider,OU=Hosting,DC=hosting,DC=local" code="0x80131600" executeSeqNo="2"><errorSource namespace="Exchange 2007 Provider" procedure="MailEnableContact" /><errorSource namespace="SW Managed Exchange" procedure="MailEnableContact" /></errorContext></response>]"
Related article_: #3498
Task name:
any BlackBerry related task
Error message:
Exception during execution of MPF request: "Unknown error 0x80131500 [<response><errorContext description="BESUserAdmin Error: "Service busy.", Error Code = -27" code="0x80131500" executeSeqNo="1"><errorSource namespace="BESProvider" procedure="someProcedure" /></errorContext></response>]"
Related article: #1850
Additional Information
Please see #2247 for more information regarding out-of-sync issues.