You will query journald, forward logs to a central rsyslog server, and rotate files so disks never fill. By the end you will have searchable, retained, centralized logs, the foundation for monitoring and the SOC labs.

Learning Objectives

  • Filter and follow logs with journalctl.
  • Forward logs to a central server over rsyslog.
  • Rotate and retain logs with logrotate.
  • Time: ~3 hours · Difficulty: Intermediate · Prereqs: a server (and optionally a second host as collector).

Architecture Overview

Environment Setup

You will need: systemd-journald (default), rsyslog, and logrotate.

Before you begin: if forwarding, have a reachable collector listening on TCP/514.

Step-by-Step Execution

01
Query logs precisely with journalctl

Targeted filters turn a flood of entries into the few that matter for triage.

journalctl -p err -S "2026-06-17 00:00:00" -u ssh --no-pager
Shows SSH errors since midnight at priority error or higher.
02
Forward logs to a central server

Centralized logs survive a compromised host and enable cross-host correlation.

echo '*.* @@192.168.1.60:514' | sudo tee /etc/rsyslog.d/90-forward.conf && sudo systemctl restart rsyslog
03
Rotate logs with logrotate
# logrotate -d /etc/logrotate.d/rsyslog
rotating pattern: /var/log/syslog weekly (4 rotations) considering log /var/log/syslog log does not need rotating

Progress So Far

Testing & Validation

logger "test-event-tyf" && journalctl -n 1 | grep test-event-tyf

You should see the test event in the local journal (and on the collector if forwarding). If so, your logging pipeline is working.

Troubleshooting
  • No logs on collector: confirm the collector listens on 514 and firewall allows it.
  • Journal too large: cap with SystemMaxUse= in journald.conf.
  • logrotate not running: it is triggered by a daily timer; test with logrotate -f.

Extension Ideas

  • Feed centralized logs into the Automated SOC Pipeline.
  • Add structured logging (JSON) for easier parsing.
  • Forward over TLS (RELP) for confidentiality and reliability.

Key Results

  • Queried logs precisely by priority, time, and unit.
  • Centralized logs to a collector for cross-host correlation.
  • Bounded disk usage with rotation and retention.
  • Verified the pipeline end to end with a test event.