Symptoms

On a RHEL-7 based node, there are no system logs recorded. All loggers configured via rsyslog service have empty logs.

journalctl shows last lines dated back to several days ago.

Verifying the journal shows FS corruption:

# journalctl --verify
658bd8: Invalid object
File corruption detected at /run/log/journal/51f718164f82479b888ce94ec08c51cd/system.journal:658bd8 (of 8388608 bytes
FAIL: /run/log/journal/51f718164f82479b888ce94ec08c51cd/system.journal (Cannot assign requested address)

Cause

systemd journal corruption

Resolution

Remove the corrupted journal file and restart the service:

# mv /run/log/journal/51f718164f82479b888ce94ec08c51cd/system.journal /run/log/journal/51f718164f82479b888ce94ec08c51cd/system.journal.corrupt
# systemctl restart systemd-journald

Internal content