Logging & Auditing

Intrusion Detection

Host-based detection: rootkit checks, osquery, and baselining.

Logs and audit records tell you what happened as it happened. Host-based intrusion detection asks a different question: given a known-good picture of this system, has anything changed that should not have? An attacker who gets in will try to persist (a modified binary, a new cron job, a hidden process) and cover their tracks. Detection works by comparing the live system against a baseline and flagging the delta. The core discipline is establishing “normal” while the host is clean, so that abnormal becomes visible later.

This page covers where attackers hide and what compromise looks like, using file integrity baselining with AIDE, querying a fleet with osquery, and the rootkit scanners along with their real limits.

Signs of compromise and where attackers hide

Persistence and stealth leave traces in predictable places. Knowing the list turns a vague “something feels off” into specific checks.

LocationWhat to look for
System binaries (/usr/bin, /usr/sbin)Modified ls, ps, netstat that hide the attacker’s activity
Startup and schedulingNew systemd units, cron jobs, ~/.bashrc or profile additions
SSH configurationAdded authorized keys, PermitRootLogin flipped on
KernelLoaded rootkit modules, hooked syscalls
Processes and portsA listener on an odd port, a process with no matching binary on disk
AccountsNew UID 0 users, unexpected sudoers entries

A quick manual sweep catches the obvious cases. For example, any account other than root with UID 0 is a red flag.

awk -F: '($3 == 0) {print $1}' /etc/passwd

Expected output:

root

Anything beyond root here warrants immediate investigation.

File integrity with AIDE

AIDE (Advanced Intrusion Detection Environment) builds a cryptographic database of file hashes, sizes, permissions, and timestamps, then reports what changed against it. It is the practical implementation of baselining. Initialize the database on a clean host and store it somewhere the attacker cannot reach (read-only media or a remote host), because a baseline the attacker can rewrite is worthless.

sudo apt-get install -y aide
sudo aideinit
sudo mv /var/lib/aide/aide.db.new /var/lib/aide/aide.db
sudo aide --check | head -n 10

Expected output:

AIDE 0.18.6 found differences between database and filesystem!!
Start timestamp: 2026-07-06 11:03:22 +0000

Summary:
  Total number of entries:  214831
  Added entries:            0
  Removed entries:          0
  Changed entries:          2

Changed entries:
f  ...  /usr/bin/curl

A change to /usr/bin/curl that you did not cause (no package update explains it) is exactly the signal AIDE exists to produce. The catch is noise: legitimate updates change files too, so you re-baseline after every patch cycle and investigate only unexplained changes. This ties directly to Patch Management, since a clean baseline must follow, not precede, a round of updates.

osquery for fleet-wide questions

AIDE answers “did this file change” on one host. osquery answers “which of my hosts has X” by exposing the operating system as a SQL database you can query live. It scales the same detection thinking across many machines.

sudo apt-get install -y osquery
sudo osqueryi "SELECT name, path, pid FROM processes WHERE on_disk = 0;"

Expected output:

+---------+-----------------+------+
| name    | path            | pid  |
+---------+-----------------+------+
| [kw]    | /tmp/.x/kworker | 3391 |
+---------+-----------------+------+

A process whose executable is no longer on_disk (deleted after launch) running from /tmp/.x is a classic malware pattern: the payload deleted itself from disk to evade file scanners but still runs in memory. osquery makes questions like this expressible as SQL, and a scheduled query pack can run them continuously across a fleet, forwarding results to a central system. This is the point where host detection meets a SIEM.

chkrootkit, rkhunter, and their limits

chkrootkit and rkhunter scan for known rootkits by signature and by checking known-bad indicators. They are worth running, but their limits matter more than their hits.

sudo apt-get install -y rkhunter
sudo rkhunter --check --sk 2>&1 | grep -E 'Warning|Rootkit' | head -n 4

Expected output:

Rootkit checks...
Warning: The command '/usr/bin/lwp-request' has been replaced by a script: ...
Rootkits checked : 499
Possible rootkits: 0

The honest framing is this: signature scanners only find rootkits they already know about. A competent attacker uses something not in the signature set, or a rootkit sophisticated enough to lie to the scanner (the modified ps and ls from the table above). So treat a clean rkhunter run as weak evidence, not proof, and treat false positives (a legitimately replaced command) as the common case. Baselining with AIDE and behavioral queries with osquery are more robust because they detect change and anomaly rather than known signatures. For continuous, correlated detection across the layers in this group, an agent-based platform like Wazuh (which builds on the OSSEC project) combines file integrity, log analysis, and rootkit checks into one pipeline.

Practical Guidance

  1. Baseline while the host is known clean, and store the AIDE database read-only or off-host so an attacker cannot rewrite the reference.
  2. Re-baseline after every patch cycle, then investigate only the file changes that no update explains.
  3. Learn the common hiding spots (UID 0 accounts, startup hooks, added SSH keys, deleted-but-running binaries) and sweep them when a host feels wrong.
  4. Use osquery to ask behavioral questions like “processes whose binary is no longer on disk” and run them fleet-wide rather than host by host.
  5. Run rkhunter or chkrootkit for what they catch, but treat a clean result as weak evidence since signature scanners miss novel and self-hiding rootkits.
  6. For ongoing detection across the whole group, feed file integrity, logs, and audit records into a platform such as Wazuh so signals are correlated rather than checked in isolation.