Server hardening: a complete checklist
Step by step, with commands: from a fresh VPS to a server you can trust in production.
In 30 seconds
A fresh server on the internet is scanned within minutes. Hardening means shrinking what an attacker can reach and limiting what they can do if they get in: keys-only SSH, a firewall that denies by default, automatic security updates, separate users per app, private databases, HTTPS everywhere, a WAF in front, and backups you have actually restored. No single step is enough; together they make a small server genuinely hard to break.
Why I use it
- I apply a written security baseline to every server I manage, because consistency beats heroics: the same eight checks everywhere.
- Most real breaches I hear about are not clever exploits but an open database port, a password login or a forgotten update.
- I hold my own tooling to the same standard: when I work on servers with an AI agent, it is read-only by default and every change needs my approval.
- A hardened, documented server is easier to hand over, audit and recover, which is exactly what clients pay for.
How it works
01The mindset: layers and least privilege
Hardening follows two principles. Defence in depth: assume any single layer can fail, so stack several (edge WAF, firewall, SSH policy, app permissions, backups). Least privilege: every user, process and key gets only the access it needs, so a compromise stays small. Work in a safe order, because the classic hardening accident is locking yourself out. Create your own user and key first, confirm you can log in with it in a second terminal, and only then disable root and passwords. Allow SSH in the firewall before enabling it. Keep the provider's web console as an emergency door. Write each step into a short runbook or script, so the next server gets exactly the same treatment and you can prove to a client what was done.
- Reduce attack surface: fewer open ports, services and packages.
- Limit blast radius: separate users, private networks, scoped credentials.
- Assume breach: backups off-site, logs and alerts you will actually read.
021. Users and SSH: keys only, no root
Create a personal admin user with sudo and install your public key (an Ed25519 key is the modern default). Then disable root login and password authentication with a drop-in file in /etc/ssh/sshd_config.d/, which survives package updates better than editing the main file. On Ubuntu the service is called ssh. Validate the config with sshd -t before reloading, and keep your current session open while you test a new login in another terminal. Changing the SSH port only reduces log noise; it is not a security control, so do it only if you want quieter logs. For teams, give each person their own user and key so access can be revoked individually, and remove keys of people who leave.
# On the server, as root (one time)
adduser your_user
usermod -aG sudo your_user
# From your laptop
ssh-copy-id -i ~/.ssh/id_ed25519.pub [email protected]
# /etc/ssh/sshd_config.d/99-hardening.conf
PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
MaxAuthTries 3
AllowUsers your_user
# Validate, then reload (keep the current session open!)
sudo sshd -t && sudo systemctl reload ssh
032. Firewall, updates and fail2ban
Deny all incoming traffic by default and open only what the server serves: SSH, HTTP and HTTPS. Mirror the same rules in your cloud provider's firewall, so traffic is filtered before it even reaches the machine. One important trap: Docker publishes container ports by writing its own iptables rules, which bypass ufw. Bind published ports to 127.0.0.1 in Compose files unless they truly must be public. Next, enable unattended-upgrades for security patches; most kernel updates still need a reboot, so schedule a maintenance window or allow automatic reboots at a quiet hour. Install fail2ban to ban brute-force sources. Finally, remove services you do not use: every listening daemon is one more thing to patch.
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow OpenSSH
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enable
sudo ufw status verbose
sudo apt update && sudo apt install -y unattended-upgrades fail2ban
sudo dpkg-reconfigure -plow unattended-upgrades
# Optional: /etc/apt/apt.conf.d/50unattended-upgrades
# Unattended-Upgrade::Automatic-Reboot "true";
# Unattended-Upgrade::Automatic-Reboot-Time "04:00";
sudo ss -tulpn # everything listening; nothing unexpected on 0.0.0.0
043. Apps: separate users, permissions, private databases, secrets
Run each site or app as its own system user with its own PHP-FPM pool or process manager entry. If one app is compromised, it cannot read another app's code, uploads or environment file. Code should be owned by the deploy user and not writable by the web process; only folders that must change at runtime (Laravel's storage and bootstrap/cache, upload folders) are writable. Environment files are readable only by the app user. Databases and Redis listen on localhost or a private network and every app has its own least-privilege database user. Secrets live in environment files on the server or in a secrets manager, never in Git, and are rotated when someone leaves or a leak is suspected. Run a secret scanner on the repository, because leaked keys in old commits are a common source of breaches.
# One system user per app, no login shell
sudo adduser --system --group --home /var/www/shop --shell /usr/sbin/nologin shop
# Code: owned by the deploy user, readable by the app group
sudo chown -R deploy:shop /var/www/shop
sudo find /var/www/shop -type d -exec chmod 750 {} +
sudo find /var/www/shop -type f -exec chmod 640 {} +
# Only runtime folders are writable by the app
sudo chmod -R 770 /var/www/shop/storage /var/www/shop/bootstrap/cache
# Secrets: readable by the app only
sudo chown deploy:shop /var/www/shop/.env && sudo chmod 640 /var/www/shop/.env
054. HTTPS everywhere and a WAF in front
Every site gets a certificate, usually from Let's Encrypt through certbot, with automatic renewal (certbot installs a systemd timer; check it with a dry run). Redirect HTTP to HTTPS, use TLS 1.2 and 1.3 only, and add HSTS once you are sure every subdomain works over HTTPS. Putting Cloudflare or a similar CDN/WAF in front hides the origin IP, absorbs basic DDoS traffic, and lets you block bad bots and apply rate limits at the edge. Use the SSL mode Full (strict), so the connection from Cloudflare to your server is encrypted and the certificate is validated; Flexible sends traffic unencrypted to the origin. For extra protection, allow 80 and 443 only from Cloudflare's IP ranges or use a tunnel, so attackers cannot bypass the edge by hitting the origin IP directly.
sudo apt install -y certbot python3-certbot-nginx
sudo certbot --nginx -d example.com -d www.example.com
sudo certbot renew --dry-run
systemctl list-timers | grep certbot
# nginx: modern TLS and HSTS (add HSTS only when all subdomains use HTTPS)
ssl_protocols TLSv1.2 TLSv1.3;
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
add_header X-Content-Type-Options "nosniff" always;
add_header Referrer-Policy "strict-origin-when-cross-origin" always;
server_tokens off;
065. Backups, monitoring and audits
Hardening also means surviving what you could not prevent. Take provider snapshots and, separately, application-level backups of databases and uploads copied to another account or provider. Test restores on a schedule; an untested backup is a hope, not a plan. Monitor what usually kills small servers: full disks, memory exhaustion, expired certificates and a site that is down. An external uptime checker (Uptime Kuma or the provider's alerts) plus disk and load alerts covers most of it. Keep logs long enough to investigate an incident. Then audit: Lynis scans the system and lists concrete hardening suggestions with a score, which is a good way to catch what a checklist missed. Re-run it after changes and keep the report.
# Audit the system and read the suggestions
sudo apt install -y lynis
sudo lynis audit system
sudo less /var/log/lynis-report.dat
# Quick health checks
df -h # disk usage
free -h # memory
sudo last -n 20 # recent logins
sudo journalctl -p err -b # errors since boot
sudo fail2ban-client status sshd
Use it when
- On every new server, before the first application is deployed.
- When you inherit a server: audit first, then apply the checklist step by step.
- After any incident or near miss, as a review of what layer failed.
- Periodically (for example quarterly), because servers drift as people make quick fixes.
Skip it when
- Do not apply aggressive kernel or SSH changes on a production server without a second session and a console fallback.
- Do not chase a perfect audit score at the expense of a working, maintainable system; prioritise real risks.
Alternatives
- Managed platforms (PaaS)
The provider hardens and patches the OS; you secure the app and its configuration.
- Ansible or cloud-init
Automate the same checklist so every server is hardened identically and repeatably.
- CIS Benchmarks
Detailed, formal hardening standards; useful for compliance, heavier than most small teams need.
- Cloudflare Tunnel or a VPN
Removes public ports on the origin entirely; access goes through an authenticated tunnel.
Best practices
- Keys-only SSH, no root login, personal accounts per admin.
- Default-deny firewall on the host and at the provider; open only 22, 80 and 443.
- Automatic security updates and a regular reboot window.
- One system user per app, least-privilege file permissions and database users.
- Databases and Redis on localhost or a private network only.
- HTTPS everywhere with auto-renewal, Cloudflare Full (strict) and edge WAF rules.
- Off-site backups, scheduled restore tests, uptime and disk alerts, and a periodic Lynis audit.
Common mistakes
- Disabling password login before confirming key login works, then needing the provider console to get back in.
- Trusting ufw while Docker publishes database ports to the whole internet.
- Running every site as www-data, so one vulnerable plugin exposes every site on the server.
- Using Cloudflare Flexible SSL, leaving the origin connection unencrypted.
- Keeping backups only on the same server, where ransomware or a deleted droplet takes them too.
Deep dive
07SSH internals on modern Ubuntu: socket activation and crypto
Since Ubuntu 24.04, SSH uses systemd socket activation: ssh.socket listens on the port and starts ssh.service on the first connection. A systemd generator reads Port and ListenAddress from sshd_config to build the socket, so changing the port means daemon-reload plus a restart of ssh.socket, not just reloading the service. Many people have locked themselves out by changing Port, reloading ssh, and closing the session while the socket still listened on 22 and the firewall only allowed the new port. On cryptography, current OpenSSH defaults are good: recent releases negotiate a hybrid post-quantum key exchange automatically, and Ed25519 keys are the standard. Do not paste old cipher lists from blog posts; they often weaken modern defaults. sshd -T prints the effective configuration after all drop-ins, and the ssh-audit tool reports weak algorithms. For the highest assurance, use hardware-backed keys (ed25519-sk) on a security key, so a stolen laptop file is useless.
# /etc/ssh/sshd_config.d/99-hardening.conf
Port 2222
# Open the new port first, then rebuild the socket
sudo ufw allow 2222/tcp
sudo sshd -t
sudo systemctl daemon-reload
sudo systemctl restart ssh.socket
sudo ss -tlnp | grep -E ':22 |:2222 '
# Effective settings after all drop-ins
sudo sshd -T | grep -E 'permitrootlogin|passwordauthentication|port'
08Sandboxing services with systemd
A separate Unix user limits which files a compromised app can read, but the process can still see most of the filesystem, write to /tmp, load arbitrary binaries and open any network socket. systemd can confine each service much further with a few lines in a drop-in: a read-only view of the operating system, no access to home directories, a private /tmp, no privilege escalation through setuid binaries, and only the address families it needs. systemd-analyze security scores every unit from 0 (well protected) to 10 (exposed) and lists which options are missing, which makes this measurable. Apply options gradually and test, because an over-tight sandbox shows up as permission errors at runtime. MemoryMax and CPUQuota also stop one runaway process from taking the whole server down. The same ideas exist for containers: run as non-root, read-only root filesystem, dropped capabilities.
# /etc/systemd/system/web.service.d/hardening.conf
[Service]
User=web
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
PrivateDevices=yes
ReadWritePaths=/var/www/web/shared
ProtectKernelTunables=yes
ProtectKernelModules=yes
ProtectControlGroups=yes
RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX
CapabilityBoundingSet=
MemoryMax=1G
# Then: sudo systemctl daemon-reload && systemd-analyze security web.service
09Kernel and network settings
sysctl settings tune the kernel's behaviour. Ubuntu already ships sensible defaults in /etc/sysctl.d/10-*.conf: SYN cookies, protected symlinks and hardlinks, restricted kernel pointers, and AppArmor restrictions on unprivileged user namespaces since 24.04. A small extra file hardens the network stack further: ignore ICMP redirects and source-routed packets that could reroute traffic, do not send redirects since the server is not a router, and log packets with impossible source addresses. Two cautions. Docker, Kubernetes and VPN servers need net.ipv4.ip_forward = 1, so never disable forwarding blindly on such hosts. And strict reverse-path filtering (rp_filter = 1) can break servers with several interfaces and asymmetric routing; Ubuntu defaults to loose mode (2) for that reason. Apply with sysctl --system and verify that networking and containers still work before rolling out.
# /etc/sysctl.d/99-hardening.conf
net.ipv4.conf.all.accept_redirects = 0
net.ipv6.conf.all.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.all.accept_source_route = 0
net.ipv6.conf.all.accept_source_route = 0
net.ipv4.conf.all.log_martians = 1
net.ipv4.tcp_syncookies = 1
kernel.dmesg_restrict = 1
kernel.kptr_restrict = 2
# Apply and check one value
sudo sysctl --system
sysctl net.ipv4.conf.all.accept_redirects
10Detection: knowing when a layer failed
Prevention eventually fails, so plan to notice. Ship logs off the server (journald forwarding, Loki, or a hosted service), because an attacker with root edits local logs first. auditd records security-relevant events such as changes to /etc/passwd, sudoers or SSH keys. A file integrity checker such as AIDE builds a database of hashes for system files and reports changes; run it from a timer and keep the baseline database off the host. Patching has a detection side too. unattended-upgrades installs fixes, but running processes keep old libraries in memory until restarted: needrestart lists them, and /var/run/reboot-required tells you a new kernel waits for a reboot. Ubuntu Pro, free for a small number of personal machines, adds Livepatch for kernel fixes without reboots and the Ubuntu Security Guide for automated CIS or DISA-STIG profiles. Alert on new sudo users, new authorized_keys entries and unexpected listening ports.
# Watch changes to identity and SSH files
sudo apt install -y auditd
sudo auditctl -w /etc/passwd -p wa -k identity
sudo auditctl -w /etc/sudoers.d -p wa -k sudoers
sudo auditctl -w /home/your_user/.ssh/authorized_keys -p wa -k sshkeys
sudo ausearch -k sshkeys --start today
# Services still running old libraries, and pending reboot
sudo needrestart -r l
[ -f /var/run/reboot-required ] && cat /var/run/reboot-required.pkgs
Real-world scenarios
You inherit a server with password SSH and an open database
Problem
A client hands over a server that has run for years. Root logs in with a password, MySQL listens on 0.0.0.0, nobody knows which cron jobs matter, and the site must stay online.
How to solve it
Audit before changing anything: take a provider snapshot, run ss -tulpn, list users, sudoers, authorized_keys, crontabs and enabled services, and run Lynis for a baseline. Then fix the highest risk with the smallest change: bind MySQL to 127.0.0.1 (check that no external app depends on it first) and block 3306 in the provider firewall. Next, create your personal sudo user with a key, test it in a second session, then disable root and password login. Enable ufw with SSH allowed first. Rotate the database and root passwords, because they may have leaked years ago. Document every step for the client.
CPU at 100 percent from an unknown process
Problem
Monitoring shows constant full CPU. top reveals a process with a random name running as the web user, connecting to an unknown address. The site still works.
How to solve it
Treat it as a compromise, not a performance issue. Snapshot the disk first for evidence, then isolate: restrict outbound traffic in the provider firewall and put the site in maintenance mode. Collect what you can (ps auxf, ss -tupn, the binary's path via /proc/PID/exe, crontabs of the web user, recently modified files under the web root). Find the entry point, usually an outdated plugin or an upload form that accepts PHP. Do not just kill the process and continue: rebuild a clean server from your baseline, deploy code from Git, restore data from a backup taken before the breach, rotate every secret the server held, and patch the hole before going live.
Redis in Docker reachable from the internet
Problem
An external port scan shows 6379 open, although ufw denies everything except 22, 80 and 443. The Compose file publishes '6379:6379' so a developer could connect from home.
How to solve it
Docker's published ports bypass ufw, so the scan is right. Immediately change the mapping to 127.0.0.1:6379:6379, or remove ports entirely and let other containers reach Redis over the Compose network, then docker compose up -d. Set a Redis password or ACL user anyway. For the developer, use an SSH tunnel: ssh -L 6379:127.0.0.1:6379 [email protected]. Check Redis for signs of abuse, such as unknown keys or a changed dir and dbfilename, which attackers use to write files. Add the provider firewall as a second layer and schedule an external port scan, so the next mistake is caught by monitoring, not by an attacker.
Cheat sheet
- Effective SSH settings
sudo sshd -T | grep -Ei 'permitrootlogin|passwordauthentication'- Apply an SSH port change on Ubuntu 24.04+
sudo systemctl daemon-reload && sudo systemctl restart ssh.socket- Firewall rules with numbers (to delete one)
sudo ufw status numbered && sudo ufw delete 3- Everything listening, with process
sudo ss -tulpn- Scan your server from outside
nmap -Pn -p- 203.0.113.10- Preview automatic updates
sudo unattended-upgrade --dry-run --debug- Pending reboot after updates
cat /var/run/reboot-required.pkgs- Score a service's sandbox
systemd-analyze security nginx.service- Recent failed logins
sudo journalctl -u ssh --since today | grep -i 'invalid user'- Short-lived SSH certificate for a user
ssh-keygen -s user_ca -I alice -n alice -V +8h id_ed25519.pub
In my real projects
- I am rolling out an eight-point baseline to every server I manage: keys-only SSH, firewall with only 22/80/443, private databases, fail2ban with unattended-upgrades, a user per app, snapshots with tested restores, monitoring, and HTTPS with Cloudflare in front.
- I manage servers over key-based SSH and work with an AI agent under strict rules: read-only by default, every change approved by me.
- On one server I brought disk usage down from about 91% to 66% by cleaning logs, old builds and oversized node_modules, a reminder that disk monitoring is part of security.
Questions & answers
Q1What are the first things you do on a new Linux server?
Update packages, create a personal sudo user and install my SSH key, confirm key login in a second session, then disable root login and password authentication. Next, ufw with default deny incoming, allowing only SSH, 80 and 443, mirrored in the cloud firewall. Then unattended-upgrades and fail2ban, timezone and hostname, and a check with ss that nothing unexpected is listening. Before deploying apps I create a system user per app, keep databases on localhost, set up certbot with auto-renewal, put Cloudflare in front in Full (strict) mode, and configure backups and uptime monitoring. Ideally all of this is a script or Ansible role, so every server is identical.
Q2Is changing the SSH port a good security measure?
It reduces noise, not risk. Moving SSH off port 22 cuts the number of automated login attempts in the logs, because many bots only try the default port, but any targeted scan finds the new port in seconds. Real protection comes from keys-only authentication, no root login, a firewall that ideally limits SSH to known IPs or a VPN, and fail2ban. If a team changes the port, I make sure the firewall rule is updated before the SSH change, so nobody is locked out, and that it is documented. I see it as optional cosmetics on top of the real controls.
Q3Why can Docker bypass ufw, and how do you prevent it?
ufw manages its own iptables chains, but when a container publishes a port, Docker inserts its NAT and forwarding rules into chains that are evaluated before ufw's input rules. So a Compose file with ports 5432:5432 makes PostgreSQL reachable from the internet even though ufw says deny. The simplest fix is to bind published ports to localhost, for example 127.0.0.1:5432:5432, or not publish internal services at all and let containers talk over a Docker network. The cloud provider firewall is a second safety net, because it filters traffic before it reaches the host. I always verify from outside with a port scan.
Q4How do you manage secrets on a server?
Secrets never go into Git. On a simple server they live in environment files owned by the deploy user, readable only by the app's group, with permissions like 640, outside the web root. For larger setups, a secrets manager such as Vault, AWS Secrets Manager, Doppler or encrypted files with SOPS injects them at deploy or run time. Each app and environment gets its own credentials with minimal scope, so staging keys cannot touch production. I rotate secrets when people leave or after any suspected leak, and run secret scanning in CI, because leaked keys in old commits are a frequent breach source.
Q5What does Cloudflare add to server security, and what SSL mode should we use?
Cloudflare sits between visitors and your server: it hides the origin IP, absorbs volumetric DDoS, provides a WAF, bot protection and rate limiting, and caches static content. The SSL mode should be Full (strict): visitors connect to Cloudflare over HTTPS, and Cloudflare connects to your server over HTTPS while validating the origin certificate, either Let's Encrypt or a Cloudflare origin certificate. Flexible mode leaves the origin connection unencrypted and often causes redirect loops. To prevent attackers from bypassing Cloudflare, restrict ports 80 and 443 to Cloudflare's IP ranges, use authenticated origin pulls, or use a tunnel so no web ports are open at all.
Q6How do you know a server is actually hardened, not just configured once?
By checking continuously, not trusting memory. I keep the baseline as a checklist or automation and re-run it, use Lynis to audit the system and compare scores over time, and scan the server's public IP from outside to see which ports are really open. Monitoring alerts on disk, uptime and certificate expiry, and fail2ban and auth logs show login activity. I review user accounts and SSH keys periodically and remove stale ones. And I test restores, because recovery is part of security. Configuration drifts as people make quick fixes, so hardening is a recurring process with evidence, not a one-time task.
Senior-level questions
S1Patch servers in place or rebuild them from images?
Patching in place is simple and fine for a few long-lived servers: unattended security updates, a reboot window, and configuration management to prevent drift. The weakness is that each server slowly becomes unique, and after an incident you cannot be sure what is on it. Immutable infrastructure builds a fresh image or container for every change, with Packer, cloud-init or Docker, and replaces servers instead of modifying them. Drift disappears, rollbacks are a redeploy, and a compromised host is simply destroyed. It requires stateless servers, automation and a pipeline, which is a real investment. My pragmatic path: in-place patching with scripted baselines for small fleets, and immutable images once state lives in managed services and there are several identical app servers.
S2How do you manage SSH access for a growing team?
Copying public keys into authorized_keys does not scale: offboarding means editing every server, and keys never expire. Better options, in increasing maturity: manage keys centrally with Ansible so adding or removing a person is one change; put a bastion or VPN in front so servers do not expose SSH at all; then use SSH certificates. With certificates, servers trust one CA key through TrustedUserCAKeys, and people get certificates signed for a few hours, ideally issued after single sign-on with MFA by a tool such as an identity-aware proxy. Access expires automatically, logs show who connected, and offboarding happens in the identity provider. Production write access should also be rarer than read access, with sudo logged.
S3Cloudflare Tunnel or firewall allowlists for protecting the origin?
An allowlist keeps ports 80 and 443 open but accepts only Cloudflare's published ranges. It is simple and fast, but the list changes occasionally and must be updated, and anyone else using Cloudflare could in theory route through it to your origin unless you also use authenticated origin pulls. A tunnel runs a small daemon on the server that connects outbound to Cloudflare, so the origin has no inbound web ports at all, and it can even carry SSH with Zero Trust access policies. The costs are one more process to monitor, tighter coupling to one vendor, and debugging through the tunnel. For a small fleet behind Cloudflare, I favour a tunnel or allowlist plus origin certificates; both beat leaving the origin open.
Glossary
- Attack surface
- Everything an attacker can reach and try: open ports, services, logins, uploaded files and exposed APIs.
- Blast radius
- How much damage one compromised account, key or service can cause before other controls stop it.
- Defence in depth
- Stacking independent security layers so the failure of one does not expose the whole system.
- Drop-in file
- A small config file in a .d directory that overrides the main config and survives package updates.
- Socket activation
- systemd listening on a port itself and starting the service only when a connection arrives.
- AppArmor
- Ubuntu's mandatory access control system, which limits what specific programs may read, write or execute.
- sysctl
- The interface for reading and setting kernel parameters at runtime, persisted in /etc/sysctl.d files.
- HSTS
- A response header telling browsers to use only HTTPS for a domain for a set period.
- Bastion host
- A single hardened server used as the only SSH entry point into a private network.
- SSH certificate
- A user or host key signed by a trusted CA, with built-in expiry, replacing scattered authorized_keys entries.
Keep learning
Have a product in mind?Let's build it.
Tell me about your idea, your product or your team. I usually reply within one working day.