Every internet-facing Linux server gets scanned by automated bots within minutes of going online. Attackers don’t need to target you specifically — they sweep entire IP ranges looking for weak SSH configurations, open ports, and unpatched software. As a network and cybersecurity consultant, I see the same avoidable mistakes on client servers again and again. This checklist walks through the 15 hardening steps that matter most, with the commands you need to apply them today.

Table of Contents

1. Update Your System and Automate Patching
2. Create a Non-Root Admin User
3. Switch to SSH Key Authentication
4. Harden the SSH Daemon
5. Configure a Default-Deny Firewall
6. Install Fail2ban
7. Enable SELinux or AppArmor
8. Reduce Your Attack Surface
9. Enforce Strong Password and Account Policies
10. Lock Down File Permissions
11. Audit SUID and SGID Binaries
12. Centralize and Protect Your Logs
13. Set Up File Integrity Monitoring
14. Harden Kernel Network Parameters
15. Test Backups and Disaster Recovery
16. Conclusion
17. FAQ

1. Update Your System and Automate Patching

Unpatched software remains the single biggest attack vector on Linux servers. Vulnerability scanners find outdated kernels and packages within hours of a server going live. You need patching to happen automatically, not whenever you remember to log in.

On Debian/Ubuntu, enable unattended upgrades:

sudo apt update && sudo apt install unattended-upgrades
sudo dpkg-reconfigure --priority=low unattended-upgrades

On RHEL/AlmaLinux/Rocky, use dnf-automatic:

sudo dnf install dnf-automatic
sudo systemctl enable --now dnf-automatic.timer

Review /etc/apt/apt.conf.d/50unattended-upgrades (or /etc/dnf/automatic.conf) to confirm security updates install automatically and reboots happen on a schedule you control.

2. Create a Non-Root Admin User

Never operate a server directly as root. A dedicated sudo user gives you an audit trail and limits the damage from a stolen credential or a mistyped command.

sudo adduser deploy
sudo usermod -aG sudo deploy    # Debian/Ubuntu
sudo usermod -aG wheel deploy   # RHEL-based

Once the new account works with sudo, disable direct root logins entirely — covered in the SSH section below. Keep sudo access limited to people who genuinely need it, and review group membership quarterly.

3. Switch to SSH Key Authentication

Password-based SSH invites brute-force attacks around the clock. Key-based authentication removes that entire attack class. Generate an Ed25519 key, which is faster and more secure than older RSA keys:

ssh-keygen -t ed25519 -C "deploy@yourserver"
ssh-copy-id -i ~/.ssh/id_ed25519.pub deploy@your-server-ip

Test that key login works in a second terminal window before touching password settings — locking yourself out of a remote server is a painful mistake to fix.

4. Harden the SSH Daemon

Once key login is confirmed, edit /etc/ssh/sshd_config:

PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
MaxAuthTries 3
ClientAliveInterval 300
ClientAliveCountMax 2
AllowUsers deploy

Changing the default port (22) to something non-standard cuts down noisy automated scans, though it’s a minor deterrent rather than real security:

Port 2222

Restart the service and confirm your session survives:

sudo systemctl restart sshd

5. Configure a Default-Deny Firewall

A server should reject every connection it doesn’t explicitly need. On Ubuntu, UFW makes this simple:

sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow 2222/tcp   # your SSH port
sudo ufw allow 80,443/tcp # web traffic if applicable
sudo ufw enable

On RHEL-based systems, use firewalld:

sudo firewall-cmd --permanent --add-port=2222/tcp
sudo firewall-cmd --permanent --add-service=https
sudo firewall-cmd --reload

Review open ports periodically with ss -tulpn and close anything you can’t account for.

6. Install Fail2ban

Fail2ban watches your logs and temporarily bans IPs that show brute-force behavior. It’s a lightweight but effective layer on top of your firewall.

sudo apt install fail2ban   # or: sudo dnf install fail2ban

Create /etc/fail2ban/jail.local:

[sshd]
enabled = true
port = 2222
filter = sshd
logpath = /var/log/auth.log
maxretry = 3
findtime = 300
bantime = 86400

sudo systemctl enable --now fail2ban
sudo fail2ban-client status sshd

Add ignoreip entries for your own trusted IPs so you don’t accidentally lock yourself out during testing.

7. Enable SELinux or AppArmor

Mandatory access control (MAC) restricts what a compromised process can do, even if an attacker gets code execution. RHEL-based distributions ship SELinux; Ubuntu and Debian favor AppArmor. Neither should be disabled “to make troubleshooting easier” — that habit is how a minor exploit becomes a full compromise.

Check SELinux status:

sestatus
sudo setenforce 1   # enforcing mode

Check AppArmor status

sudo aa-status
sudo systemctl enable --now apparmor

If an application breaks under enforcing mode, write a targeted policy exception rather than disabling MAC system-wide.

8. Reduce Your Attack Surface

Every running service is a potential entry point. Audit what’s listening and remove what you don’t use.

sudo systemctl list-units --type=service --state=running
ss -tulpn

Disable anything you don’t need — old mail daemons, print services, or unused database engines are common offenders:

sudo systemctl disable --now cups

A minimal install is easier to patch, easier to monitor, and gives an attacker fewer places to hide.

9. Enforce Strong Password and Account Policies

Even with SSH keys enforced, local accounts and sudo escalation still rely on passwords. Install libpam-pwquality (Debian/Ubuntu) or pam_pwquality (RHEL) to require strong passwords:

sudo apt install libpam-pwquality

Edit /etc/security/pwquality.conf:

minlen = 14
dcredit = -1
ucredit = -1
ocredit = -1

Set account lockout after repeated failures with pam_faillock, and review /etc/passwd for accounts that no longer need shell access — set their shell to /usr/sbin/nologin.

10. Lock Down File Permissions

Weak file permissions are a common cause of internal privilege escalation. Sensitive files should never be world-readable:

sudo chmod 600 /etc/shadow
sudo chmod 644 /etc/passwd
sudo chown root:root /etc/ssh/sshd_config
sudo chmod 600 /etc/ssh/sshd_config

For web application directories, avoid chmod 777 entirely — it’s a shortcut that turns a small misconfiguration into a full compromise. Scope permissions to the specific user and group that need access.

11. Audit SUID and SGID Binaries

Programs with the SUID bit run with the file owner’s privileges, usually root. Attackers look for misconfigured or forgotten SUID binaries to escalate privileges. Find them:

sudo find / -perm -4000 -type f 2>/dev/null

Compare the output against a known-good baseline and remove the SUID bit from anything that doesn’t need it:

sudo chmod u-s /path/to/binary

Re-run this audit after every major software installation, since packages sometimes set SUID bits you didn’t ask for.

12. Centralize and Protect Your Logs

Local logs can be deleted by an attacker who gains root. Shipping logs to a separate, append-only destination gives you a forensic record that survives a compromise.

sudo apt install rsyslog

Configure /etc/rsyslog.conf to forward logs to a remote syslog server or a SIEM. At minimum, enable auditd for kernel-level event tracking:

sudo apt install auditd
sudo systemctl enable --now auditd
sudo auditctl -w /etc/passwd -p wa -k passwd_changes

Review logs on a schedule rather than only after something breaks.

13. Set Up File Integrity Monitoring

AIDE (Advanced Intrusion Detection Environment) creates a cryptographic baseline of your filesystem and alerts you when files change unexpectedly — a strong signal of compromise.

sudo apt install aide
sudo aideinit
sudo cp /var/lib/aide/aide.db.new /var/lib/aide/aide.db

Schedule regular checks with cron:

0 5 * * * /usr/bin/aide.wrapper --check

Store the baseline database somewhere the server itself can’t modify, such as a separate host or read-only media.

14. Harden Kernel Network Parameters

A handful of sysctl settings close off common network-based attacks. Add these to /etc/sysctl.d/99-hardening.conf:

net.ipv4.tcp_syncookies = 1
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.icmp_echo_ignore_broadcasts = 1

Apply the changes:

sudo sysctl -p /etc/sysctl.d/99-hardening.conf

These settings reduce your exposure to SYN flood attacks, IP spoofing, and ICMP-based reconnaissance — small changes with real impact.

15. Test Backups and Disaster Recovery

Hardening reduces risk; it doesn’t eliminate it. When something does go wrong — ransomware, hardware failure, or a bad update — a tested backup is what actually saves you.

sudo rsync -avz --delete /etc /var/www /home backup-server:/backups/$(hostname)/

A backup you’ve never restored from isn’t a backup — it’s a hope. Schedule a quarterly restore test to a spare environment and confirm the data comes back intact and the application actually boots.

Conclusion

Securing a Linux server isn’t a single action you complete and forget — it’s a layered process. SSH hardening and a default-deny firewall stop the most common automated attacks. Fail2ban and mandatory access control add friction for anyone who gets past the front door. Logging, file integrity monitoring, and tested backups make sure that if something does slip through, you notice quickly and can recover cleanly. Work through this checklist one item at a time, automate what you can with configuration management tools like Ansible, and revisit it every time you provision a new server.

FAQ

Do I need antivirus software on a Linux server? Most production Linux servers don’t need traditional antivirus. Proper hardening, timely patching, and intrusion detection tools like fail2ban and AIDE provide better protection for server workloads than signature-based antivirus.

Is changing the default SSH port actually useful? It reduces log noise from automated scanners but isn’t real security on its own. Combine it with key-only authentication and fail2ban for meaningful protection.

How often should I re-run this checklist? Review firewall rules and running services monthly, rotate SSH keys and audit sudo access quarterly, and re-run the full checklist whenever you provision a new server.

Should I disable SELinux if it’s blocking my application? No. Write a targeted policy exception with audit2allow instead of disabling SELinux system-wide, which removes an entire layer of protection.

What’s the fastest single change to make right now? Disabling SSH password authentication in favor of key-based login closes off the most commonly exploited attack path with almost no downside.