Most Linux servers don’t get broken into through some clever kernel exploit. They get broken into because root login was still allowed over SSH, password auth was on, nobody had patched in five months, and a firewall was never turned on because “it’s behind the cloud security group anyway”. Boring stuff. Which is exactly why a checklist works better than a lecture here.
This is the hardening checklist I actually run through on a fresh box, in the order I do it, with the commands to check each item, the fix, and how to prove the fix took. Everything below was run on Ubuntu 24.04 LTS and AlmaLinux 10.2 (RHEL 10 rebuilt bit for bit, so the same commands apply to RHEL, Rocky and Oracle Linux 10). Where the two families differ, and they differ more than you’d think, both versions are shown.
The checklist at a glance
Twelve areas. The first four matter before the box has a public IP. The rest you can do in the first week.
- Patch, then make patching automatic
- Lock down SSH
- Accounts, sudo and password policy
- Turn on the firewall (ufw or firewalld)
- Leave AppArmor or SELinux enforcing
- Kernel and network sysctl settings
- Block filesystem and kernel modules you don’t use
- Mount options, SUID binaries and permissions
- Audit rules with auditd
- Time sync, persistent logs and file integrity
- Cut the service list down
- Score yourself with Lynis and OpenSCAP
Each section below has three parts: how to check, how to fix, how to verify. If a section already checks out on your server, skip it. No points for redoing work.
1. Patch it, then make patching automatic
Unpatched packages are still the number one way in. Not because people don’t know to patch, but because patching is a chore that gets deferred. So the fix isn’t “remember to patch”. It’s “make the machine patch itself for security updates and tell you when it needs a reboot”.
Check. On Ubuntu:
$ sudo apt update && apt list --upgradable
apparmor/noble-updates 4.0.1really4.0.1-0ubuntu0.24.04.7 amd64 [upgradable from: 4.0.1really4.0.1-0ubuntu0.24.04.3]
apport/noble-updates 2.28.3-0ubuntu0.1 all [upgradable from: 2.28.1-0ubuntu3.8]
apt/noble-updates 2.8.3 amd64 [upgradable from: 2.7.14build2]
base-files/noble-updates 13ubuntu10.5 amd64 [upgradable from: 13ubuntu10.1]
...On AlmaLinux and RHEL, dnf can tell you specifically what’s a security fix, which apt can’t:
$ sudo dnf updateinfo summary
Updates Information Summary: available
38 Security notice(s)
13 Important Security notice(s)
22 Moderate Security notice(s)
3 Low Security notice(s)
1 Bugfix notice(s)Thirteen “Important” security notices on a box that was installed a few months ago. That’s normal, and it’s the reason automation matters.
Fix. Patch it now:
# Ubuntu
sudo apt update && sudo apt full-upgrade -y
# RHEL / AlmaLinux
sudo dnf upgrade --security -y # security only
sudo dnf upgrade -y # everythingThen set up automatic security updates. Ubuntu ships unattended-upgrades already enabled on server installs, but confirm it:
$ systemctl is-enabled unattended-upgrades
enabled
$ cat /etc/apt/apt.conf.d/20auto-upgrades
APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Unattended-Upgrade "1";
$ grep -vE "^\s*(//|$)" /etc/apt/apt.conf.d/50unattended-upgrades | head -8
Unattended-Upgrade::Allowed-Origins {
"${distro_id}:${distro_codename}";
"${distro_id}:${distro_codename}-security";
"${distro_id}ESMApps:${distro_codename}-apps-security";
"${distro_id}ESM:${distro_codename}-infra-security";
};If 20auto-upgrades is missing or shows "0", run sudo dpkg-reconfigure -plow unattended-upgrades and answer yes. The Allowed-Origins list is the important bit: by default it only pulls from the security pocket, so you won’t get surprised by a feature update at 3am.
On RHEL the equivalent is dnf-automatic, and it is not installed by default:
sudo dnf install -y dnf-automatic
sudo sed -i 's/^upgrade_type = .*/upgrade_type = security/; s/^apply_updates = .*/apply_updates = yes/' /etc/dnf/automatic.conf
sudo systemctl enable --now dnf-automatic.timerVerify.
$ grep -E "^(upgrade_type|apply_updates|download_updates)" /etc/dnf/automatic.conf
upgrade_type = security
download_updates = yes
apply_updates = yes
$ systemctl list-timers 'dnf-automatic*' --no-pager
NEXT LEFT LAST PASSED UNIT ACTIVATES
Mon 2026-01-05 06:20:06 PKT 3h 32min - - dnf-automatic.timer dnf-automatic.serviceAnd on Ubuntu, a dry run tells you what it would have done:
$ sudo unattended-upgrades --dry-run --debug 2>&1 | grep -E "Allowed origins|All upgrades"
Allowed origins are: o=Ubuntu,a=noble, o=Ubuntu,a=noble-security, o=UbuntuESMApps,a=noble-apps-security, o=UbuntuESM,a=noble-infra-security
All upgrades installedOne more thing people forget: a patched kernel does nothing until you reboot into it. sudo dnf needs-restarting -r on RHEL, or cat /var/run/reboot-required on Ubuntu, tells you whether a reboot is pending. Put it in your monitoring.
2. Lock down SSH
SSH is the one service every server exposes, so it gets the most attention. The defaults on both distros are worse than you’d expect for 2026:
$ sudo sshd -T | grep -Ei "^(permitrootlogin|passwordauthentication|maxauthtries|x11forwarding|clientaliveinterval|logingracetime) "
clientaliveinterval 0
logingracetime 120
maxauthtries 6
passwordauthentication yes
permitrootlogin without-password
x11forwarding yesThat output is from Ubuntu 24.04. AlmaLinux 10 is identical apart from the ordering. Password auth on, root allowed in with a key, six guesses per connection, X11 forwarding on a headless server. sshd -T is the command to remember here, by the way. It prints the effective config after every include and match block has been applied, which is the only thing that matters.
Fix. Don’t edit /etc/ssh/sshd_config directly. Both distros now ship an Include line near the top, and the package manager will happily overwrite the main file on upgrade:
$ sudo grep -n "^Include" /etc/ssh/sshd_config
15:Include /etc/ssh/sshd_config.d/*.confDrop-ins are read in filename order, and for most keywords in sshd, the first value wins. So name yours to sort first. Before you write it, make sure your own key works and you have a second session open. Everyone locks themselves out of a server once. Try to make it a test box.
sudo groupadd -f sshusers
sudo usermod -aG sshusers yourname
sudo tee /etc/ssh/sshd_config.d/00-hardening.conf > /dev/null <<'EOF'
PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
PermitEmptyPasswords no
MaxAuthTries 3
MaxSessions 4
LoginGraceTime 30
ClientAliveInterval 300
ClientAliveCountMax 2
X11Forwarding no
AllowAgentForwarding no
AllowTcpForwarding no
LogLevel VERBOSE
AllowGroups sshusers
Banner /etc/issue.net
EOF
sudo sshd -t && sudo systemctl restart ssh # sshd on RHELAllowGroups is the underrated one. It means a new account someone creates for a contractor can’t SSH in until you deliberately add it to the group. AllowTcpForwarding no stops the server being used as a pivot; turn it back on for the specific hosts that need tunnels. LogLevel VERBOSE logs the key fingerprint of every login, which you’ll want the day you’re trying to figure out whose key that was.
If you still need to generate and copy keys, the passwordless SSH guide covers that end to end.
Verify. Always sshd -t first, then check the effective values:
$ sudo sshd -t && echo "config OK"
config OK
$ sudo sshd -T | grep -Ei "^(permitrootlogin|passwordauthentication|maxauthtries|allowgroups|x11forwarding|loglevel|clientaliveinterval) "
allowgroups sshusers
clientaliveinterval 300
loglevel VERBOSE
maxauthtries 3
passwordauthentication no
permitrootlogin no
x11forwarding noThen actually try to get in as root and watch it fail:
$ ssh -o BatchMode=yes root@localhost
Authorised access only. All activity is monitored and logged.
root@localhost: Permission denied (publickey).
$ sudo journalctl -u ssh -n 3 --no-pager
Jan 5 02:45:35 web1 sshd[56069]: Connection from 127.0.0.1 port 52930 on 127.0.0.1 port 22 rdomain ""
Jan 5 02:45:35 web1 sshd[56069]: User root from 127.0.0.1 not allowed because none of user's groups are listed in AllowGroups
Jan 5 02:45:35 web1 sshd[56069]: Connection closed by invalid user root 127.0.0.1 port 52930 [preauth]That’s the Ubuntu journal. Same test on AlmaLinux 10, which ships OpenSSH 9.9, gives you a bonus line:
$ sudo journalctl -u sshd -n 3 --no-pager
Jan 5 02:55:49 app1 sshd-session[28428]: User root from 127.0.0.1 not allowed because none of user's groups are listed in AllowGroups
Jan 5 02:55:49 app1 sshd-session[28428]: Connection closed by invalid user root 127.0.0.1 port 44900 [preauth]
Jan 5 02:55:49 app1 sshd[28398]: srclimit_penalise: ipv4: new 127.0.0.1/32 deferred penalty of 1 seconds for penalty: connections without attempting authenticationThat srclimit_penalise line is the new PerSourcePenalties feature. OpenSSH 9.8 and later slows down clients that keep failing, on its own, without fail2ban. It’s on by default on RHEL 10. Ubuntu 24.04 is on 9.6 so doesn’t have it yet.
Three things that differ between the distros here:
- The unit is
sshon Ubuntu andsshdon RHEL. Same daemon, different name. Get it wrong and systemctl tells you the unit doesn’t exist. - Ubuntu 24.04 uses socket activation.
systemctl stop sshstops the daemon butssh.socketkeeps port 22 open and respawns it on the next connection. To change the listening port on Ubuntu you edit the socket unit, notsshd_config, and stopping SSH for real means stopping both. - RHEL puts its own drop-ins in place:
40-redhat-crypto-policies.confand50-redhat.conf. The crypto one means you don’t setCiphersorKexAlgorithmsin sshd at all on RHEL. You useupdate-crypto-policiessystem-wide instead, and every service picks it up.update-crypto-policies --showshould sayDEFAULTorFUTURE, neverLEGACY.
And a small one: on RHEL /etc/ssh/sshd_config is mode 600 so you need sudo just to read it. On Ubuntu it’s 644. Neither is wrong, but it catches people writing scripts for both.
3. Accounts, sudo and password policy
Five quick checks that take a minute and catch the most common mess:
$ sudo passwd -S root
root L 2025-06-05 0 99999 7 -1
$ awk -F: '$3==0' /etc/passwd
root:x:0:0:root:/root:/bin/bash
$ sudo awk -F: '$2==""' /etc/shadow
$ awk -F: '$3>=1000 && $7!~/nologin|false/' /etc/passwd
asif:x:1000:1000::/home/asif:/bin/bash
$ sudo grep -rE "NOPASSWD" /etc/sudoers /etc/sudoers.d/
/etc/sudoers:asif ALL=(ALL) NOPASSWD: ALLIn order: root’s password is locked (L), good. Only one account has UID 0, good. No accounts with an empty password field, good. One human account with a real shell, fine. And one NOPASSWD sudo rule, which was put there by a cloud image and should not survive on a production box. Remove it with visudo, never by editing the file directly, because a syntax error in sudoers locks everyone out of sudo at once.
The sudo group itself is sudo on Ubuntu and wheel on RHEL. Check who’s in it:
$ getent group sudo # Ubuntu
sudo:x:27:asif
$ getent group wheel # RHEL
wheel:x:10:asifPassword ageing. The defaults say passwords never expire:
$ grep -E "^(PASS_MAX_DAYS|PASS_MIN_DAYS|PASS_WARN_AGE|UMASK|ENCRYPT_METHOD)" /etc/login.defs
UMASK 022
PASS_MAX_DAYS 99999
PASS_MIN_DAYS 0
PASS_WARN_AGE 7
ENCRYPT_METHOD SHA512That’s Ubuntu. AlmaLinux 10 shows ENCRYPT_METHOD YESCRYPT, which is stronger and a nice free upgrade. Set sane values, and note that login.defs only affects new accounts. Existing users need chage:
sudo sed -i 's/^PASS_MAX_DAYS.*/PASS_MAX_DAYS\t365/; s/^PASS_MIN_DAYS.*/PASS_MIN_DAYS\t1/; s/^PASS_WARN_AGE.*/PASS_WARN_AGE\t14/; s/^UMASK.*/UMASK\t\t027/' /etc/login.defs
sudo chage -M 365 -m 1 -W 14 asif$ sudo chage -l asif
Last password change : Nov 05, 2025
Password expires : Nov 05, 2026
Password inactive : never
Account expires : never
Minimum number of days between password change : 1365 days rather than 90. CIS moved to 365 and NIST says don’t force rotation at all if you have MFA and breach monitoring. Pick your compliance framework and follow it; don’t invent a number.
Password quality. Both distros use pam_pwquality. Ubuntu needs the package installed (libpam-pwquality, and it wires itself into PAM on install); RHEL has it already. The config goes in a drop-in:
sudo mkdir -p /etc/security/pwquality.conf.d
sudo tee /etc/security/pwquality.conf.d/50-local.conf > /dev/null <<'EOF'
minlen = 14
minclass = 3
maxrepeat = 3
dictcheck = 1
enforce_for_root
EOFYou can test the policy without changing anyone’s password using pwscore (package libpwquality-tools on Ubuntu, included on RHEL):
$ echo "Summer2026!" | pwscore
Password quality check failed:
The password is shorter than 14 characters
$ echo "correct-horse-Battery-9" | pwscore
94Lockout after failed attempts. This is where the distros really split. On RHEL, pam_faillock is one command away:
$ sudo authselect enable-feature with-faillock
$ authselect current
Profile ID: local
Enabled features:
- with-faillock
$ grep faillock /etc/pam.d/system-auth
auth required pam_faillock.so preauth silent
auth required pam_faillock.so authfail
account required pam_faillock.soThen set the thresholds in /etc/security/faillock.conf:
deny = 5
fail_interval = 900
unlock_time = 900
even_deny_root
root_unlock_time = 60And prove it works by getting a password wrong three times and reading the tally:
$ sudo faillock --user asif
asif:
When Type Source Valid
2026-01-05 02:55:50 TTY /dev/pts/4 V
2026-01-05 02:55:54 TTY /dev/pts/4 V
2026-01-05 02:55:57 TTY /dev/pts/4 V
$ sudo faillock --user asif --resetOn Ubuntu 24.04, faillock.conf exists but nothing reads it. There’s no pam-auth-update profile for faillock, so you wire it into /etc/pam.d/common-auth by hand. Put these around the existing pam_unix.so line:
auth required pam_faillock.so preauth
auth [success=1 default=ignore] pam_unix.so nullok
auth [default=die] pam_faillock.so authfail
auth sufficient pam_faillock.so authsuccand add account required pam_faillock.so to common-account. Test it in a second session before you log out of the first. Getting PAM wrong can lock you out of sudo and login at the same time.
Honestly, if password authentication is off on SSH (it should be, see above) and the box has no console users, faillock is belt and braces. It’s still worth doing because CIS checks for it and because su and sudo prompts go through PAM too.
4. Turn on the firewall
“It’s behind a security group” is not a firewall. The host firewall is what still protects you when someone opens 0.0.0.0/0 on the cloud side for a quick test and forgets.
Ubuntu: ufw. It’s installed but inactive on a fresh server. Default deny inbound, allow outbound, rate-limit SSH, open only what the box serves:
$ sudo ufw status
Status: inactive
$ sudo ufw default deny incoming
Default incoming policy changed to 'deny'
$ sudo ufw default allow outgoing
Default outgoing policy changed to 'allow'
$ sudo ufw limit 22/tcp comment "SSH rate-limited"
Rules updated
Rules updated (v6)
$ sudo ufw allow 443/tcp comment HTTPS
Rules updated
Rules updated (v6)
$ sudo ufw enable
Firewall is active and enabled on system startuplimit instead of allow on port 22 means an IP that opens more than six connections in 30 seconds gets dropped. Cheap brute force protection with no extra software. Verify:
$ sudo ufw status verbose
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), deny (routed)
New profiles: skip
To Action From
-- ------ ----
22/tcp LIMIT IN Anywhere # SSH rate-limited
443/tcp ALLOW IN Anywhere # HTTPS
22/tcp (v6) LIMIT IN Anywhere (v6) # SSH rate-limited
443/tcp (v6) ALLOW IN Anywhere (v6) # HTTPSIf you can, restrict SSH to your own ranges instead of “Anywhere”: sudo ufw limit from 203.0.113.0/24 to any port 22 proto tcp. And if you run Docker on the box, be aware Docker writes its own iptables rules that bypass ufw entirely for published ports. That’s a whole separate article, but you should know it’s a thing.
RHEL: firewalld. Installed and running by default on a normal install. The default public zone allows ssh, dhcpv6-client and cockpit, and the last two probably shouldn’t be there on a server:
sudo firewall-cmd --permanent --remove-service=cockpit
sudo firewall-cmd --permanent --remove-service=dhcpv6-client
sudo firewall-cmd --permanent --add-service=https
sudo firewall-cmd --permanent --add-rich-rule='rule service name="ssh" limit value="4/m" accept'
sudo firewall-cmd --reloadThe rich rule is firewalld’s equivalent of ufw’s limit: four new SSH connections a minute per source, then drop. After --reload the permanent config is what’s live, and it lives in a zone file you can read and put in version control:
$ sudo cat /etc/firewalld/zones/public.xml
<?xml version="1.0" encoding="utf-8"?>
<zone>
<short>Public</short>
<description>For use in public areas. ...</description>
<service name="ssh"/>
<service name="https"/>
<rule>
<service name="ssh"/>
<accept>
<limit value="4/m" burst="0"/>
</accept>
</rule>
<forward/>
</zone>Verify with sudo firewall-cmd --list-all and check the services: line says only what you expect. The classic firewalld mistake is forgetting --permanent (change disappears at reboot) or forgetting --reload after using it (change isn’t live yet). Do both, every time.
Whichever firewall you use, confirm from outside with nmap from another host. What the firewall says it does and what a port scan sees are occasionally different things.
5. Leave AppArmor or SELinux enforcing
This is the shortest section because the advice is one line: do not turn it off. Ubuntu ships AppArmor enforcing, RHEL ships SELinux enforcing, and the single worst hardening decision people make is running setenforce 0 because a web app threw a permission error.
Check it’s on:
# Ubuntu
sudo aa-status | head -3 # "apparmor module is loaded" and a profile count
systemctl is-enabled apparmor
# RHEL
getenforce # must say Enforcing
grep ^SELINUX= /etc/selinux/configIf getenforce says Permissive or Disabled, set SELINUX=enforcing in /etc/selinux/config, touch /.autorelabel, and reboot. The relabel takes a few minutes on a big filesystem and is not optional after a period of being disabled.
When something genuinely is blocked, fix the label or the boolean, not the enforcement mode:
# what did SELinux deny recently, in English
sudo ausearch -m AVC -ts recent | audit2why
# common fixes
sudo setsebool -P httpd_can_network_connect on
sudo restorecon -Rv /var/www/html
# AppArmor: put one profile in complain mode while you debug, not the whole system
sudo aa-complain /etc/apparmor.d/usr.sbin.nginxOn RHEL, install setroubleshoot-server on anything you’ll be debugging on; it turns raw AVC denials into a sentence with a suggested fix. On Ubuntu, apparmor-utils gives you aa-status, aa-complain and aa-enforce.
6. Kernel and network sysctl settings
The kernel defaults are tuned for a machine that might be a router or a desktop. A server isn’t either. Here’s what a fresh Ubuntu 24.04 looks like:
$ sysctl net.ipv4.conf.all.send_redirects kernel.dmesg_restrict kernel.kptr_restrict net.ipv4.conf.all.log_martians
net.ipv4.conf.all.send_redirects = 1
kernel.dmesg_restrict = 0
kernel.kptr_restrict = 1
net.ipv4.conf.all.log_martians = 0Sending ICMP redirects (a server never should), any user can read the kernel log, kernel pointers partially exposed, spoofed packets not logged. Write a drop-in. Never edit /etc/sysctl.conf itself:
sudo tee /etc/sysctl.d/99-hardening.conf > /dev/null <<'EOF'
# Network
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv4.conf.all.secure_redirects = 0
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.all.log_martians = 1
net.ipv4.conf.default.log_martians = 1
net.ipv4.conf.all.rp_filter = 1
net.ipv4.icmp_echo_ignore_broadcasts = 1
net.ipv4.icmp_ignore_bogus_error_responses = 1
net.ipv4.tcp_syncookies = 1
net.ipv6.conf.all.accept_ra = 0
net.ipv6.conf.all.accept_redirects = 0
# Kernel
kernel.randomize_va_space = 2
kernel.kptr_restrict = 2
kernel.dmesg_restrict = 1
kernel.yama.ptrace_scope = 1
kernel.sysrq = 0
kernel.unprivileged_bpf_disabled = 1
fs.protected_hardlinks = 1
fs.protected_symlinks = 1
fs.protected_fifos = 2
fs.protected_regular = 2
fs.suid_dumpable = 0
EOF
sudo sysctl --systemTwo of those need a note. net.ipv4.ip_forward isn’t in the list because if you run Docker, Kubernetes or anything that routes, it has to be 1, and those tools set it themselves. Add net.ipv4.ip_forward = 0 only on a box that genuinely doesn’t forward. And net.ipv6.conf.all.accept_ra = 0 assumes static IPv6 or no IPv6; if you get your v6 address from router advertisements, leave that line out.
Verify. sysctl --system prints every file it applies in order, which also shows you why the drop-in is named 99-: it needs to load after the distro’s own files.
$ sudo sysctl --system 2>&1 | grep Applying
* Applying /usr/lib/sysctl.d/10-default-yama-scope.conf ...
* Applying /usr/lib/sysctl.d/50-redhat.conf ...
* Applying /etc/sysctl.d/99-hardening.conf ...
$ sysctl kernel.dmesg_restrict kernel.kptr_restrict net.ipv4.conf.all.send_redirects
kernel.dmesg_restrict = 1
kernel.kptr_restrict = 2
net.ipv4.conf.all.send_redirects = 0That’s AlmaLinux. On a fresh RHEL 10 minimal install /etc/sysctl.d/ doesn’t even exist yet, so mkdir it first. Ubuntu ships a dozen 10-*.conf files in there already, and the same 99- file overrides them cleanly.
7. Block filesystem and kernel modules you don’t use
Every kernel module that can be auto-loaded is attack surface, and a server has no business mounting a HFS+ volume or a USB stick. The install ... /bin/false trick makes modprobe fail instead of loading:
sudo tee /etc/modprobe.d/hardening.conf > /dev/null <<'EOF'
install cramfs /bin/false
install freevxfs /bin/false
install jffs2 /bin/false
install hfs /bin/false
install hfsplus /bin/false
install udf /bin/false
install usb-storage /bin/false
blacklist cramfs
blacklist usb-storage
EOFVerify. Dry-run modprobe shows what it would do, and a real attempt fails loudly:
$ modprobe -n -v cramfs
install /bin/false
$ sudo modprobe cramfs
modprobe: ERROR: ../libkmod/libkmod-module.c:1084 command_do() Error running install command '/bin/false' for module cramfs: retcode 1
modprobe: ERROR: could not insert 'cramfs': Invalid argument
$ lsmod | grep cramfsEmpty lsmod is the result you want. Add squashfs to the list on RHEL if you like; on Ubuntu, leave it, because snapd needs it. Same for usb-storage on a physical box where someone might legitimately need a USB drive for recovery. Think before you copy-paste.
8. Mount options, SUID binaries and permissions
Mount options. World-writable temp locations should not allow executables or device files:
$ findmnt /dev/shm
TARGET SOURCE FSTYPE OPTIONS
/dev/shm none tmpfs rw,nosuid,nodev,noatimeMissing noexec. Add to /etc/fstab:
tmpfs /dev/shm tmpfs defaults,nodev,nosuid,noexec 0 0
tmpfs /tmp tmpfs defaults,nodev,nosuid,noexec,size=2G 0 0then sudo mount -o remount /dev/shm. On both distros you can enable tmp.mount instead of writing the /tmp line by hand (sudo systemctl unmask tmp.mount && sudo systemctl enable --now tmp.mount), and then override its options with a drop-in. Be careful with noexec on /tmp: some installers and a few badly behaved apps unpack and run things there. Test before you roll it out fleet-wide.
If /home, /var, /var/log and /var/log/audit are on separate partitions, give them nodev and, where nothing executes from them, nosuid,noexec too. CIS wants them separate mostly so a full /var/log can’t take down /. That’s a build-time decision, so it’s here as a reminder for the next image you make.
SUID and SGID binaries. These run as their owner regardless of who calls them, so you want to know exactly which ones exist:
$ sudo find / -xdev -type f -perm /6000 2>/dev/null | sort
/usr/bin/chage
/usr/bin/chfn
/usr/bin/chsh
/usr/bin/gpasswd
/usr/bin/mount
/usr/bin/newgrp
/usr/bin/passwd
/usr/bin/su
/usr/bin/sudo
/usr/bin/umount
/usr/bin/write
/usr/libexec/utempter/utempter
/usr/sbin/pam_timestamp_check
/usr/sbin/unix_chkpwdFourteen on AlmaLinux 10, all of them expected. Ubuntu 24.04 has 22 including snap-confine, ssh-agent and the polkit helper. Save this list. Diff it next month. A new entry you don’t recognise is worth a very close look. The find command guide has more on these permission searches if the syntax looks alien.
World-writable and unowned files.
$ sudo find / -xdev -type d -perm -0002 -not -perm -1000 2>/dev/null
/tmp/.X11-unix
$ sudo find / -xdev \( -nouser -o -nogroup \) 2>/dev/nullWorld-writable directories without the sticky bit let one user delete another user’s files. Unowned files usually mean a package was removed and left something behind, or a UID was reused. Both lists should be empty or explainable.
Critical file permissions. Worth a glance because they occasionally get clobbered by a careless chmod -R:
$ ls -l /etc/passwd /etc/shadow /etc/gshadow
-rw-r--r-- 1 root root 936 Jan 5 02:36 /etc/passwd
---------- 1 root root 547 Jan 5 02:47 /etc/shadow
---------- 1 root root 383 Jan 5 02:47 /etc/gshadowThat’s RHEL, where shadow files are mode 000 and only root’s capabilities get through. Ubuntu uses 640 with group shadow. Both fine. What’s not fine is /etc/shadow readable by anyone else, ever.
And umask. The default 022 makes new files world-readable. 027 is the CIS value; set it in /etc/login.defs as above and in /etc/profile.d/umask.sh so shells pick it up too. For files that need finer control than owner/group/other, ACLs are the right tool rather than loosening the umask.
9. Audit rules with auditd
auditd records who did what at the syscall level, which is a different thing from application logs. When you’re reconstructing an incident, it’s the difference between “the sudoers file changed on Tuesday” and “UID 1002 ran vim on /etc/sudoers at 14:32 from an SSH session originating at 10.0.4.19”.
Installed by default on RHEL, one apt install auditd on Ubuntu. A starter rule set covering the things you’d actually want to know about:
sudo tee /etc/audit/rules.d/50-hardening.rules > /dev/null <<'EOF'
-D
-b 8192
-f 1
# identity and privilege
-w /etc/passwd -p wa -k identity
-w /etc/shadow -p wa -k identity
-w /etc/group -p wa -k identity
-w /etc/sudoers -p wa -k sudoers
-w /etc/sudoers.d/ -p wa -k sudoers
# sshd config
-w /etc/ssh/sshd_config -p wa -k sshd
-w /etc/ssh/sshd_config.d/ -p wa -k sshd
# every command run as root by a real user
-a always,exit -F arch=b64 -S execve -F euid=0 -F auid>=1000 -F auid!=unset -k root_cmds
# kernel module loading
-a always,exit -F arch=b64 -S init_module,delete_module,finit_module -k modules
-w /sbin/insmod -p x -k modules
-w /sbin/modprobe -p x -k modules
EOF
sudo augenrules --load
sudo systemctl enable --now auditdOn RHEL add -w /etc/selinux/ -p wa -k mac_policy. RHEL also ships a full set of pre-written rule files in /usr/share/audit-rules/ (the 30-ospp-* and 30-pci-dss-* ones map straight onto those frameworks) that you can copy into rules.d rather than writing your own.
Verify with sudo auditctl -l to see loaded rules and sudo auditctl -s for daemon status. Then touch /etc/passwd and run sudo ausearch -k identity -i to see the event, complete with the auid of whoever did it.
The -e 2 line, which locks the rules until reboot so an attacker with root can’t quietly remove them, is deliberately not in the file above. Add it as the last line once you’ve tested, because with it in place every rule change means a reboot.
Two distro notes. RHEL 10 ships audit 4.0, where rule loading has been split into its own audit-rules.service; Ubuntu 24.04 is on 3.1.2 where auditd loads them itself. And don’t bother trying any of this under WSL: the kernel there doesn’t expose the audit netlink interface and auditd just fails to start.
10. Time sync, persistent logs and file integrity
Time. Wrong time means log timestamps you can’t correlate and TLS certificates that fail validation. Both distros use chrony (Ubuntu server may have systemd-timesyncd instead, which is fine for a client but chrony is the better server choice):
$ chronyc tracking
Reference ID : B97DBE3A (prod-ntp-5.ntp4.ps5.canonical.com)
Stratum : 3
Ref time (UTC) : Mon Jan 05 01:43:17 2026
System time : 0.667504311 seconds slow of NTP timeSynced, stratum 3, fine. Point pool in /etc/chrony.conf (RHEL) or /etc/chrony/chrony.conf (Ubuntu) at an internal NTP server if you have one, and block outbound 123/udp to anything else.
Persistent journal. Ubuntu keeps the systemd journal on disk. RHEL, out of the box, keeps it in RAM and loses it on reboot:
$ ls -ld /var/log/journal # AlmaLinux 10
ls: cannot access '/var/log/journal': No such file or directory
$ journalctl --disk-usage
Archived and active journals take up 8M in the file system.8 megabytes and a missing directory. Compare that with 214M on the Ubuntu box. Fix on RHEL:
sudo mkdir -p /var/log/journal
sudo systemd-tmpfiles --create --prefix /var/log/journal
sudo systemctl restart systemd-journaldThen cap it with SystemMaxUse=1G in /etc/systemd/journald.conf.d/size.conf. The journalctl guide goes into this in more depth, including why journalctl -b -1 is empty on a default RHEL box.
Ship logs off the box. An attacker who gets root will clean up after themselves. Logs that already left the machine can’t be cleaned. Whether that’s rsyslog to a central server, journald’s ForwardToSyslog, or an agent shipping to Loki or Elastic doesn’t matter; what matters is that /var/log/auth.log (Ubuntu) or /var/log/secure (RHEL) and the audit log exist somewhere the server can’t reach.
File integrity with AIDE. A daily diff of every file’s hash, permissions and owner against a known-good baseline. Package aide on both. On RHEL:
$ sudo aide --init
...
End timestamp: 2026-01-05 02:51:24 +0500 (run time: 1m 39s)
$ sudo mv /var/lib/aide/aide.db.new.gz /var/lib/aide/aide.db.gzOn Ubuntu it’s sudo aideinit and the database lands at /var/lib/aide/aide.db. Then create a file somewhere AIDE watches and run a check:
$ sudo touch /etc/testfile.conf
$ sudo aide --check
AIDE found differences between database and filesystem!!
Total number of entries: 25835
Added entries: 1
Removed entries: 0
Changed entries: 0Run it daily from a systemd timer (or a cron job if you prefer) and mail or ship the output. Ubuntu’s package installs a daily cron for you. And copy the baseline database somewhere off the box; a baseline that lives on the server it’s protecting can be regenerated by the same person you’re trying to catch. Files that must never change, like the AIDE config itself, can also be made immutable with chattr +i.
11. Cut the service list down
Every running service is code that can have a bug. Start by seeing what’s listening on the network, because that’s what an attacker sees:
$ sudo ss -tlnp | grep -v 127.0.0
State Recv-Q Send-Q Local Address:Port Peer Address:PortProcess
LISTEN 0 128 0.0.0.0:22 0.0.0.0:* users:(("sshd",pid=28398,fd=7))
LISTEN 0 128 [::]:22 [::]:* users:(("sshd",pid=28398,fd=8))SSH and nothing else, which is what a fresh server should look like before you install your application. Anything bound to 0.0.0.0 or * that isn’t on your list gets either removed or rebound to 127.0.0.1. The ss command guide covers the flags if you want to slice this differently.
Then the enabled services. AlmaLinux 10 minimal is impressively lean:
$ systemctl list-unit-files --state=enabled --type=service --no-legend | awk '{print $1}'
auditd.service chronyd.service dbus-broker.service firewalld.service getty@.service sshd.service
systemd-network-generator.service ...Ubuntu 24.04 server has around 35 enabled, including snapd, apport (crash reporting), landscape-client and the whole cloud-init stack. Disable what you don’t use; apport in particular has no place on a production server. The systemd guide has the mask/disable distinction if you’re not sure which to use.
Legacy packages that should never be present on anything built this decade:
# Ubuntu
dpkg -l | grep -E "telnetd|rsh-server|talkd|ypbind|tftpd|xinetd"
# RHEL
rpm -qa | grep -E "telnet-server|rsh-server|ypbind|tftp-server|xinetd|talk-server"Empty output is correct. Also worth removing on a server: cockpit unless you use it, avahi-daemon, cups, and any compiler you’re not actively building with.
Finally, systemd can score how sandboxed each service is:
$ systemd-analyze security --no-pager | head -8
UNIT EXPOSURE PREDICATE HAPPY
chrony.service 3.5 OK 🙂
cron.service 9.6 UNSAFE 😨
dbus.service 9.5 UNSAFE 😨
polkit.service 1.6 OK 🙂
ssh.service 9.6 UNSAFE 😨Don’t panic about the sad faces. ssh.service scoring 9.6 just means it runs as root with no sandboxing directives, which is normal for sshd. The score is useful for your own services: a drop-in with ProtectSystem=strict, PrivateTmp=yes, NoNewPrivileges=yes and ProtectHome=yes takes an app from 9.6 down to about 4 with zero code changes.
12. Score yourself with Lynis and OpenSCAP
Two tools, different jobs. Lynis is quick, opinionated and gives you a number. OpenSCAP checks you against a specific published benchmark and can generate the remediation script.
Lynis. Package lynis on Ubuntu; on RHEL it’s in EPEL. It ran on the Ubuntu box before and after the steps above:
$ sudo lynis audit system --quick
...
Hardening index : 69 [############# ] # before
Hardening index : 77 [############### ] # after
Tests performed : 258The number isn’t the point; the suggestions list in /var/log/lynis-report.dat is. Each one has a test ID you can look up. Sample from this run: install libpam-tmpdir, set password hashing rounds in login.defs, consider a stricter umask, put /var and /tmp on their own partitions. Some of those you’ll do, some you’ll consciously skip. Either way, get above 70 before the box goes live and don’t chase 100.
OpenSCAP and the CIS benchmark. This is where RHEL has a real advantage. The SCAP Security Guide ships in the distro and includes the CIS profiles, so a full CIS Level 1 Server scan is two packages and one command:
sudo dnf install -y openscap-scanner scap-security-guide
oscap info /usr/share/xml/scap/ssg/content/ssg-almalinux10-ds.xml | grep "profile_cis"
sudo oscap xccdf eval \
--profile xccdf_org.ssgproject.content_profile_cis_server_l1 \
--report /root/cis-report.html \
--results /root/cis-results.xml \
/usr/share/xml/scap/ssg/content/ssg-almalinux10-ds.xmlUse ssg-rhel10-ds.xml on RHEL proper, ssg-rl10-ds.xml on Rocky. It takes a couple of minutes and prints every rule as it goes:
Title
Ensure /tmp Located On Separate Partition
Rule
xccdf_org.ssgproject.content_rule_partition_for_tmp
Result
fail
Title
Ensure gpgcheck Enabled In Main dnf Configuration
Rule
xccdf_org.ssgproject.content_rule_ensure_gpgcheck_globally_activated
Result
passAfter the changes in this article, the AlmaLinux 10 box came out at 75 pass, 12 fail, with 236 rules not applicable to a minimal install. The twelve failures, pulled from the results:
- /tmp not on a separate partition (build-time decision, see section 8)
- /dev/shm missing
nodev,nosuid,noexecin fstab (section 8, the fstab line fixes it) - umask not set in
/etc/bashrcand/etc/profile(I’d only set it inlogin.defs) pam_wheelnot restrictingsuto the wheel group- a CIS-specific crypto policy module not applied
- world-writable directory without the sticky bit (that’s
/tmp/.X11-unix, which on a headless box you can just remove)
Every one of those is actionable and specific, which is what makes OpenSCAP worth the two minutes. And if you’d rather not fix them by hand:
$ sudo oscap xccdf generate fix --fix-type bash \
--profile xccdf_org.ssgproject.content_profile_cis_server_l1 \
--output /root/cis-remediate.sh \
/usr/share/xml/scap/ssg/content/ssg-almalinux10-ds.xml
$ wc -l /root/cis-remediate.sh
11468 /root/cis-remediate.shEleven thousand lines of bash. Read it before you run it. It will, among other things, set a GRUB password and disable things you may need. You can also generate --fix-type ansible and get a playbook instead, which fits far better if you already manage servers with Ansible.
On Ubuntu the official CIS tooling is usg (Ubuntu Security Guide), which needs an Ubuntu Pro subscription. Pro is free for up to five machines, so for a small estate: sudo pro attach <token>, sudo pro enable usg, sudo apt install usg, then sudo usg audit cis_level1_server. Beyond five machines, either pay or use the community ComplianceAsCode content, which builds the same ssg-ubuntu2404-ds.xml file that oscap consumes.
Ubuntu versus RHEL: the differences that catch people
Same checklist, different muscle memory. The table is the stuff that bit me while running both side by side.
| Item | Ubuntu 24.04 | RHEL 10 / AlmaLinux 10 |
|---|---|---|
| SSH unit | ssh (plus ssh.socket) |
sshd |
| SSH log | journalctl -u ssh, /var/log/auth.log |
journalctl -u sshd, /var/log/secure |
| SSH ciphers | Set in sshd_config.d |
Set via update-crypto-policies, not sshd |
| Admin group | sudo |
wheel |
| Auto updates | unattended-upgrades, on by default |
dnf-automatic, must install |
| Firewall | ufw (installed, inactive) | firewalld (installed, active) |
| Mandatory access control | AppArmor | SELinux |
| faillock | Manual PAM edit | authselect enable-feature with-faillock |
| Password hash | SHA512 | yescrypt |
| Persistent journal | Default | Must create /var/log/journal |
| auditd | Install; audit 3.1 | Installed; audit 4.0, separate audit-rules.service |
| CIS scanning | usg (Ubuntu Pro) |
oscap + scap-security-guide, in-distro |
| AIDE init | aideinit |
aide --init then rename db |
The checklist
The whole thing in one place, for the runbook. Tick them off.
- All packages updated; automatic security updates enabled and verified with a dry run
- Reboot-required check in monitoring
- SSH: root login off, password auth off,
MaxAuthTries 3,AllowGroups, forwarding off, verbose logging, banner - SSH config in a
sshd_config.ddrop-in, validated withsshd -t, verified withsshd -T - Root password locked; exactly one UID 0; no empty passwords; no stray
NOPASSWD - Password ageing and umask set in
login.defs; existing users updated withchage pam_pwqualityconfigured and tested withpwscorepam_faillockwired in and tested- Host firewall active, default deny inbound, SSH rate-limited, only serving ports open
- AppArmor or SELinux enforcing; never disabled to “fix” an app
- sysctl drop-in applied: no redirects, martians logged, syncookies on, dmesg and kptr restricted, ptrace scoped
- Unused filesystem modules and usb-storage blocked via modprobe.d
/tmpand/dev/shmmountednodev,nosuid,noexec- SUID/SGID list saved; world-writable and unowned file searches clean
- auditd running with identity, sudoers, sshd, root-command and module rules
- chrony synced; journal persistent and size-capped; logs shipped off-box
- AIDE baseline built, checked daily, database copied off-box
- Listening ports reviewed with
ss; unneeded services disabled; legacy packages absent - Lynis index above 70; CIS scan run and failures triaged
- GRUB password set on physical hosts (a console gives anyone root in two minutes otherwise)
What this checklist doesn’t cover
Deliberately. Application hardening (nginx, PostgreSQL, whatever you run) is its own topic per application. Disk encryption is a build-time choice. Fail2ban gets a lot of mentions in guides like this and it’s useful, but it’s a band-aid for the SSH password auth you’ve already turned off, and it has enough configuration gotchas to deserve its own article. Container security is a different world again.
And the most important thing not on the list: doing it more than once. A checklist you run by hand on one server is a good afternoon. The same checklist as an Ansible role, applied to every server at build time and re-applied weekly so drift gets corrected, is a security posture. The devsec.hardening collection on Ansible Galaxy implements most of what’s above for both distro families and is a reasonable place to start if you’d rather not write it yourself.
Start with the first four sections today. The rest can wait until tomorrow, but not much longer than that.
