OA CP is unavailable: java.sql.SQLException: Invalid state, the Connection object is closed. : CloudBlue Technical Support

Symptoms

OA CP is unavailable, it is impossible to login to the system. CP shows the error upon login:
```
An unforeseen error occurred. Please contact your provider. 
Error ID: 1492435491805  
```

UI calls in /var/log/poa.log fail with the following error:

java.sql.SQLException: Invalid state, the Connection object is closed.

There is a DBService error preceding the exceptions in poa.log:

javax.ejb.EJBTransactionRolledbackException: could not extract ResultSet
Caused by: java.sql.SQLException: I/O Error: Read timed out

Cause

As a result of a service restart or an outage of some different kind on the side of MSSQL, PAU ODBC uses the old threads for DB requests without validating the connection, therefore SQL errors are faced. This case has been submitted to the Maintenance Team as internal issue with id POA-101854.

Resolution

Please contact your Account Manager or Pooled Technical Associates Team (pta@odin.com) to trace the status of the issue. In order to avoid similar outages in the future, JBoss configuration should be tuned:

In C:\Program Files (x86)\SWsoft\PEM\jboss-eap-6.2\standalone\configuration\standalone-full.xml file add the following line at the top of <validation>...</validation> section:
```
<check-valid-connection-sql>SELECT 2</check-valid-connection-sql>
```
Please also note recommendations related to standalone-full.xml customization from the following KB article:
- 123177 Operations Automation (formerly POA 6.0) - Known Issues and limitations

Restart services on MN:

net stop pem
net stop pau
net start pau
net start pem