Symptoms

Task "Synchronize resource usage for instances of APS applications" fails with message:

Internal error: Resource usage calculation has failed at subscription xxxxxx: user exception, ID 'IDL:Plesk/ExSystem:1.0'.

In OA logs the following set of exceptions is captured:

  1. Failure to create a service-user:

    [rest:381630 1:23606:a04f9b90 SAAS 1520209927]: [ APSC] REST Error: POST //aps/2/resources code=500: Provisioning: resource a32eb9cf-148b-490c-b09f-fb2073742aa9 of type 'service-users' (http://parallels.com/aps/types/pa/service-user/1.1) for APS application 'POA-Core-Services-2.1-96': ExSystem: / by zero

  2. Failure to get resources from target application's endpoint:

    [SYSTEM 1:23606:9e4fdb90 lib 1520209927]:
[ APS::APSManager_impl::updateServiceInstanceResourceUsageReportEx] {module_id="APSManager"; code="54"} APS Application Error: 500 Internal Server Error 
[Exception] Internal Server Error: Trying to get property of non-object at /var/www/html/clouddrive/company.php:570.
Body: {         "code": 500,    "error": "Exception",   "message": "Internal Server Error: Trying to get property of non-object at /var/www/html/clouddrive/company.php:570",   "http_request": "GET /clouddrive/company/c7c51d08-e7ad-4667-8fd5-2153f35cd7c9 to '192.168.101.178'" }

In endpoint logs among diagnostic messages one may see the same error message: Poco::Exception: ExSystem: / by zero.

An exception, in scope of which / by zero is thrown in attempt of creating service user with empty password:

[1:23606:a0affb90:405 RequestProcessor-126042 pau 1520209927]: com.parallels.pa.service.acntmgmt.ejb.UserManagerBean creating user: user: { login: 'user.name@example.tld', accountId: 1002858 }
<...>
[1:23606:a0affb90:405 RequestProcessor-126042 pau 1520209927]: com.parallels.pa.service.acntmgmt.ejb.LocalIdentityManagerBean changing password for local identity with id = 74021 new password length: 0
[1:23606:a0affb90:405 RequestProcessor-126042 pau 1520209927]: com.parallels.pa.corba.ServiceInvoker UserManager invocation failedjavax.ejb.EJBTransactionRolledbackException: / by zero  at org.jboss.as.ejb3.
tx.CMTTxInterceptor.handleInCallerTx(CMTTxInterceptor.java:162)

Cause

At least one user was created on external application portal manually, not via OA. Therefore, it does not exist on OA. OA workflow tries to create it automatically, but fails due to combination of software-related issues:

  1. POA-91280: it throws an exception when application tries to create a user with empty password. Service user is created in scope of synchronization of users between external application and OA (application lists one user more - it was created manually on portal, application tries to create in OA in scope of "Synchronize..." task).

  2. APS-18457: causes gathering statistics failure because operation of user creation from (1) is performed in the same transaction as resource gathering (note APS-Transaction-ID header):

    [ HTTP] IN  192.168.101.178  [192] "POST //aps/2/resources/c7c51d08-e7ad-4667-8fd5-2153f35cd7c9/externalusers HTTP/1.1"  [0] "200 OK"  3125ms
...
[ HTTP] IN  192.168.101.178:57640  Begin request: "POST //aps/2/resources"
[ HTTP] IN  Request headers: <...>  'APS-Resource-ID:c7c51d08-e7ad-4667-8fd5-2153f35cd7c9'  'APS-Transaction-ID:1622-828258'

    resource gathering from instance uses the same APS-Transaction-ID:

    [ HTTP] OUT  192.168.101.178  Begin request: "GET /clouddrive/company/c7c51d08-e7ad-4667-8fd5-2153f35cd7c9"
[ HTTP] OUT  Request headers: 'APS-Controller-URI:https://192.168.101.15:6308/'  'APS-Instance-ID:9523bc2f-5e3a-4b85-8cab-f876ac7d4f1d'  'APS-Transaction-ID:1622-828258'

Resolution

Remove user on external application portal, then create it again via Odin Automation CCP and assign necessary services.

Internal content