Symptoms
pa-agent service on the Management Node cannot be started with the error in /var/log/messages:
Oct 9 11:20:25.126 pa-agent[4803]: Starting pa-agent: SCMonitor::startAllSC failed: Failed to list active SCs. Is service pau running?
Oct 9 11:20:26.024 pa-agent[4803]: start failed
Service pau starts with error in console.log:
Caused by: org.postgresql.util.PSQLException: Connection to 192.0.2.1:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
PostgreSQL service is down:
[root@osscore ~]# systemctl status postgresql-9.6
● postgresql-9.6.service - PostgreSQL 9.6 database server
Loaded: loaded (/usr/lib/systemd/system/postgresql-9.6.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Mon 2017-10-09 10:55:05 EEST; 44min ago
Process: 191 ExecStart=/usr/pgsql-9.6/bin/postmaster -D ${PGDATA} (code=exited, status=1/FAILURE)
Process: 152 ExecStartPre=/usr/pgsql-9.6/bin/postgresql96-check-db-dir ${PGDATA} (code=exited, status=0/SUCCESS)
Main PID: 191 (code=exited, status=1/FAILURE)
The following error may be observed in /var/log/messages upon service startup:
Oct 9 11:40:44.635 postmaster[5520]: [2017-10-09 11:40:44.635 EEST] p=5520:1@ c=@/: FATAL: could not create lock file "/var/run/postgresql/.s.PGSQL.5432.lock": Permission denied
Cause
Incorrect ownership on the /var/run/postgresql/ directory.
[root@osscore ~]# ll -d /var/run/postgresql/
drwxr-xr-x 2 root root 40 Oct 9 10:55 /var/run/postgresql/
Resolution
Change directory ownership to the postgres user and group:
[root@osscore ~]# chown -R postgres:postgres /var/run/postgresql/
Once it's done, start up services:
[root@osscore ~]# systemctl start postgresql-9.6
[root@osscore ~]# service pau restart
[root@osscore ~]# service pa-agent start