Symptoms
Various REST API requests to APS Controller end with error similar to:
May 2 04:09:06.975 : INF [rest:825406 1:22619:7f9e9eded700 SAAS 871251414]: [ APSC] REST: request transaction is blank
May 2 04:09:06.977 : ERR [rest:825406 1:22619:7f9e9eded700 SAAS 871251414]: [ setCurrentByID] Failed to set current transaction by ID ''. system exception, ID 'IDL:omg.org/CORBA/OBJECT_NOT_EXIST:1.0'
Unknown vendor minor code id (0), minor code = 0, completed = NO
and subsequent errors for random REST API requests contain errors:
Unhandled exception during transaction acquire
This may have an impact on many aspects of Odin Automation:
1. Login to CP:
An attempt to log into the Control Panel fails with the following error:
java.lang.NullPointerException
java.lang.NullPointerException One or more hotfix(es) are available.
On the UI node in the `/var/log/pa/pui/pui.log` the following error can be found:
04:08:32.371 [default task-210] DEBUG com.tbn.app.APSC - 18 ms [GET /packages?active()]
04:08:32.411 [default task-210] DEBUG com.tbn.app.NavApi - com.tbn.app.NavigationExceptionWrapper: java.net.SocketException
The same issue could be represented by these errors:
06:27:24.949 [default task-53] DEBUG c.o.a.s.navigation.ejb.NavApiBean - javax.ejb.EJBException: javax.ws.rs.ProcessingException: RESTEASY004655: Unable to invoke request
at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleExceptionInNoTx(CMTTxInterceptor.java:213)
...
Caused by: org.apache.http.NoHttpResponseException: 172.20.231.40:6308 failed to respond
2. Failing orders:
Random orders may fail, having the following pattern in the problem report:
[18-05-22 14:30:59.439 XMLRPC RQ116768 TH25753 TRC] RESULT: pem.APS.getServiceTemplateToken
[18-05-22 14:30:59.439 XMLRPC RQ116768 TH25753 DBG] { 'result' => { 'aps_token' => '......gzMnc9PQ==', 'controller_uri' => 'https://172.20.231.40:6308/', }, 'status' => 0, }
[18-05-22 14:30:59.439 GenericWork RQ116768 TH25753 TRC] ---[3] virtual APSToken bPemRequest::getServiceTemplateToken(STLRT::Component*, Int)
[18-05-22 14:30:59.439 APS RQ116768 TH25753 DBG] POST http://10.20.80.10:8080//aps/2/navigation:
{
"placeholder": "http:\/\/odin.com\/ui\/vendor-cp#purchase-wizard",
"allViews": "true"
}
[18-05-22 14:30:59.521 APS RQ116768 TH25753 DBG] Response: '<html>
<head>
</head>
<body>
<b>We're sorry, but something went wrong.</b>
<p>Please, contact your support team for assistance.</p>
<p><a href="/">Home</a></p>
</body>
</html>
another kind of order errors:
Registering domain in external source failed. An attempt to call external for name servers failed. An attempt to call external for name servers failed. Parallels Operations Automation error #error_code #-1, extype_id #-1, module_id #, Could not persist APS entity "http://parallels.com/aps/types/pa/subscription/1.0" Could not persist APS entity "http://parallels.com/aps/types/pa/subscription/1.0". Domain status:Prepared
3. Failing APS tasks
Tasks manipulating APS resources may also fail with the similar errors. Web Hosting Plesk specific scenario:
Cause
The issue is acknowledged as APS Controller stability issue APS-45156.
Resolution
The issue is temporary and is known to heal itself automatically within reasonable time (several minutes to a couple of hours).
In case immediate resolution is required, restart OA services.
There is a possibility that there is a repetitive action that causes the issue over and over, e.g. a Cancellation order that needs to remove a service. In such cases it is required to trace the issue to the original first failure and understand the exact origin.
The permanent solution is available in Odin Automation 7.4, so the upgrade is highly recommended.