Symptoms

BA error Could not install Resources to PEMGATE: POA is unavailable while upgrading subscriptions, adding new APS2 resources to subscriptions.

Provisioning order for APS 2.0 application fails with different errors

Service Creation Failed: Parallels Operations Automation error #extype_id #35, module_id #SaaS, APS resource for '/aps/2/application/mail-mailbox' with id '3365583' was NOT registered. Error code: 500..

Service Creation Failed: Parallels Operations Automation error #extype_id #1, module_id #Subscriptions, Transaction TransactionImple < ac, BasicAction: 0:ffffc0a82183:-25c201ed:5581e794:250041 status: ActionStatus.ABORTED > is not in a valid state to be invoking cache operations on..

The remote server is down, details: system exception, ID 'IDL:omg.org/CORBA/OBJECT_NOT_EXIST:1.0' Unknown vendor minor code id (0), minor code 0, completed NO 

APSC: Resource for account 1000100 not found...

User with id 8061195 linked to mailbox 1001001 is not registered in APS..

APS resource for '/aps/2/application/mail-mailbox' with id '1000100' was NOT registered. Error code: 404..

org.hibernate.exception.GenericJDBCException: Could not open connection.
Transaction TransactionImple < ac, BasicAction: 0:ffffc0a82183:-25c201ed:5581e794:209630 status: ActionStatus.ABORTED > is not in a valid state to be invoking cache operations on..

javax.persistence.EntityNotFoundException: Unable to find com.parallels.pa.service.subscriptions.domain.Subscription with id 1000100.

APS resource for '/aps/2/application/users' with id '1000100' was NOT registered. Error code: 500.. 

The issue may also occur during a subscription transfer process, when the target account does not exist in APS2 context.

Cause

When APS2 resources are added to an account for the first time, APS2 propagation process is started, in order to create APS2 resources for account domains, users and mailboxes.

This process can take time for heavy accounts, and exceed both default OA transaction timeout and BA waiting timeout.

Resolution

Perform APS2 propagation beforehand, using saas_ctl script.

On OA MN, perform the following SQL query to find accounts with many resources, which are not propagated to APS2:

select owner_id, count(*) from (select domains.owner_id as owner_id from domains, dns_resource_records dr where dr.domain_id=domains.domain_id union all select scope_id as owner_id from users union all select owner_id from domains union all select users.scope_id as owner_id from hs_user_services, users where users.user_id=hs_user_services.user_id and hs_type='Exchange') resources left join aps_account_context on owner_id=poa_account_id where poa_account_id is null group by owner_id having count(*)>1000 

For each of these accounts, call saas_ctl on OA MN.

On Linux:

/usr/local/pem/bin/saas_ctl sync_account <account_id> yes

On Windows:

"C:\Program Files (x86)\SWsoft\PEM\bin\saas_ctl.exe" sync_account <account_id> yes

Example:

/usr/local/pem/bin/saas_ctl sync_account 1234 yes

You can use the following script to perform these actions (/tmp/ownerid.txt must contain the query result without column names):

#!/bin/bash
while read line;
do {
STRING1='/usr/local/pem/bin/saas_ctl sync_account ';
STRING2="$line";
STRING3=' yes';
STRING4=$STRING1$STRING2$STRING3;
echo $STRING4;
export STRING4=$($STRING1$STRING2$STRING3)
echo $STRING4;
}
done < /tmp/ownerid.txt

Internal content

Link on internal Article