Symptoms
After making changes in Apache configuration via OAP Control panel, one of generated tasks "Configuring service ApacheNG" failed with the following output:
truepath fastcgi: No such file or directory (errno 2).
In core.log one could see failed attempt of file creation:
[task:100587203:...]: [ HCL::HCLProcessor::process] Processing CREATEFILE(group="${defgrp}", overwrite="yes", owner="${defuser}", path="fastcgi", perm="0644"):
[task:100587203:...]: [ OpCreateFile] {module_id="HCL"; code="1101"} truepath fastcgi: No such file or directory (errno 2).
Capturing process calls via strace utility sheds more light on the issue:
9013 send(3, "<135>Dec 15 17:54:10 : DBG [task:100587203:799842 5847:8814:b0af4b70 lib ]: [ HCL::HCLProcessor::process] Processing CREATEFILE(path=\"fastcgi\"...
9013 getcwd(0xb3748374, 4096) = -1 ENOENT (No such file or directory)
9013 send(3, "<135>Dec 15 17:54:10 : DBG [task:100587203:799842 5847:8814:b0af4b70 lib ]: [ OpCreateFile] {module_id=\"HCL\"; code=\"1101\"} truepath fastcgi: No such file or directory (errno 2)."
According to strace output, failure occurs inside of getcwd system call that acquires current working directory of a process. It could be checked the following way (for example, PID of pa-agent is 8814):
# ll /proc/8814/cwd
lrwxrwxrwx 1 root root 0 Dec 15 18:49 /proc/8814/cwd -> (deleted)/var/www/vhosts/108/...
Cause
Under unknown circumstances pa-agent was left in the location that was removed afterwards. Attempt to obtain path of removed directory lead to an error.
Resolution
Restart of agent on the slave host is required:
# service pa-agent restart
Working directory of a new process should point to /usr/local/pem/var/cores.