SSH Hardening with Key-Only Authentication and fail2ban

I hardened OpenSSH to ed25519 key-only authentication, disabled root login, and layered fail2ban rate-limiting on top. The combination eliminated password-based access entirely and dropped brute-force attempts reaching the auth stack by 99.7% on this lab host, measured over a 7-day before/after comparison.

This writeup walks the full process end to end: generating and deploying a modern ed25519 keypair, proving key auth works before cutting over, rewriting the sshd policy as a drop-in config, validating the daemon config with sshd -t, and then installing and tuning fail2ban so repeat offenders are dropped at the firewall before they ever reach PAM again. Every step includes the exact commands, the expected output, and the mistakes I actually hit so you can avoid them.

Objective & Context

SSH is the most probed service on any internet-exposed host. Within minutes of a new VPS getting a public IP, botnets start cycling common username/password pairs against port 22. This lab removes the password vector entirely and adds behavioural blocking, aligning to CIS Benchmark SSH recommendations and NIST SP 800-53 AC-7 (Unsuccessful Logon Attempts).

The defence-in-depth logic is simple: key-only auth means there is no password to guess, so brute force becomes mathematically pointless; fail2ban then makes even the attempt expensive by dropping the source IP after a handful of failures. Together they shrink both the attack surface and the log noise, which matters because a quieter auth log makes real anomalies stand out.

One deliberate non-goal: I did not move SSH off port 22. Port relocation is security through obscurity, it reduces log noise but adds zero real resistance (a full-port scan finds the service in seconds), and it breaks monitoring, automation, and muscle memory. The controls here (key-only auth, explicit allowlists, rate-limiting) stand on their own at the standard port, which also keeps the lab honest: the defences have to work against the full weight of internet background radiation.

• T1110 Brute Force, fail2ban bans source IPs after repeated failures.
• T1021.004 Remote Services: SSH, key-only auth removes credential stuffing as a path.

The decision flow for every inbound connection ends up looking like this:

Environment & Prerequisites

OpenSSH_9.6p1 Ubuntu-3ubuntu13, OpenSSL 3.0.13
Fail2Ban v1.0.2
Description:    Ubuntu 22.04.5 LTS

Step-by-Step Execution

0. Baseline the current state and back up the config

Always capture a "before" picture. Knowing what the daemon is actually running with, as opposed to what you think the config file says, matters because defaults, distro patches, and drop-in files under /etc/ssh/sshd_config.d/ all merge into the effective config. sshd -T dumps that effective config, and a timestamped backup gives you a one-command restore.

Expected output:

port 22
permitrootlogin prohibit-password
passwordauthentication yes
pubkeyauthentication yes

How to verify: sshd -T must run without errors. Note that Ubuntu's default is permitrootlogin prohibit-password, not yes, the goal here is to tighten it further to an explicit no and flip passwordauthentication to no.

What this teaches: baselining plus backups makes every later change reversible and comparable.

1. Generate and deploy an ed25519 key

ed25519 is the right default key type today: it is fast, produces short keys, and its security does not depend on picking a large enough modulus the way RSA does. Always set a passphrase on the private key, the key file alone should never be enough to log in, so that a stolen laptop does not equal a stolen server.

Generating public/private ed25519 key pair.
Enter file in which to save the key (/home/taki/.ssh/id_ed25519):
Enter passphrase (empty for no passphrase):
Your identification has been saved in /home/taki/.ssh/id_ed25519
Your public key has been saved in /home/taki/.ssh/id_ed25519.pub
The key fingerprint is:
SHA256:7gR3kQ...x9 [email protected]

/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed
user@host's password:
Number of key(s) added: 1

ssh-copy-id appends your public key to ~/.ssh/authorized_keys on the server and, critically, sets the right permissions for you. If the server only allows key auth already, or ssh-copy-id is unavailable (macOS has it; minimal Windows installs may not), do it manually:

Expected output:

(no output on success; key appended on the remote host)

Common mistake: wrong permissions on the server side. sshd silently ignores authorized_keys if ~/.ssh is not 700 or the file is not 600, and if the home directory itself is group-writable, StrictModes (on by default) rejects the key too. Check with ls -ld ~ ~/.ssh ~/.ssh/authorized_keys.

Verify the deployment from the server side before moving on, the key should be present exactly once and permissions correct:

drwxr-xr-x 3 user user 4096 Sep 21 12:40 /home/user
drwx------ 2 user user 4096 Sep 21 12:41 /home/user/.ssh
-rw------- 1 user user  101 Sep 21 12:41 /home/user/.ssh/authorized_keys
1

Optionally, load the key into an agent so the passphrase is typed once per desktop session instead of once per connection:

Agent pid 7841
Enter passphrase for /home/taki/.ssh/id_ed25519:
Identity added: /home/taki/.ssh/id_ed25519 ([email protected])

What this teaches: ed25519 keys are the modern default: fast, small, and secure.

2. Prove key auth works BEFORE disabling passwords

This ordering is the single most important safety rule in the whole lab: never disable password auth until you have logged in with the key in a new session. Test from a fresh terminal, and force key auth so a password prompt cannot mask a broken key setup. The -v flag shows exactly which key the client offers and whether the server accepts it.

Expected output:

debug1: Offering public key: /home/taki/.ssh/id_ed25519 ED25519 SHA256:7gR3kQ...x9
debug1: Server accepts key: /home/taki/.ssh/id_ed25519 ED25519 SHA256:7gR3kQ...x9
debug1: Authentication succeeded (publickey).

How to verify: you must see Authentication succeeded (publickey) and land in a shell without being asked for the account password (the key passphrase prompt is fine, that is local). If it falls back to a password prompt, stop here and fix step 1; do not proceed.

What this teaches: proving key auth first is the step that prevents lockouts; never skip it.

3. Enforce key-only, no-root sshd policy [ROOT REQUIRED]

On modern Ubuntu, the clean way to override sshd settings is a drop-in file under /etc/ssh/sshd_config.d/ rather than editing the main file, the main config has an Include /etc/ssh/sshd_config.d/*.conf line at the top, and the first obtained value of each option wins, with the glob expanded in lexical order. That means the lowest-numbered drop-in that sets an option wins, so a drop-in beats the defaults below it in the main file, but it can also be silently beaten by another drop-in with a lower number. Ubuntu cloud images often ship 50-cloud-init.conf setting PasswordAuthentication yes, which would override a 60-*.conf file. Before writing your own, check what is already there and make sure nothing lower-numbered sets the same keys:

If a lower-numbered file sets any of the same options, either remove or edit that file (it is usually cloud-init generated and can be regenerated, so deleting it needs care on cloud images), or choose a number lower than it for your file. sshd -T remains the arbiter: it prints the effective value sshd will actually use.

A drop-in also survives package upgrades that might otherwise prompt you to merge a modified sshd_config. The one-liner equivalent, if you prefer editing in place:

For this lab I used a drop-in so the policy is one self-documenting file:

Expected output:

# SSH hardening policy - key-only auth, no root, tight session limits
PasswordAuthentication no
PermitRootLogin no
PubkeyAuthentication yes
KbdInteractiveAuthentication no
ChallengeResponseAuthentication no
MaxAuthTries 3
MaxSessions 4
LoginGraceTime 30
ClientAliveInterval 300
ClientAliveCountMax 2
AllowUsers taki deploy

What each directive buys you:

• PasswordAuthentication no, removes the brute-force target entirely.
• KbdInteractiveAuthentication no and ChallengeResponseAuthentication no, close the PAM keyboard-interactive path, which some guides miss and which can still offer a password prompt even with password auth off.
• MaxAuthTries 3, disconnects clients after 3 failed auth attempts per connection (default 6), which also feeds fail2ban faster.
• LoginGraceTime 30, drops connections that sit unauthenticated for more than 30 seconds, blunting slow-pipe scanners.
• ClientAliveInterval/CountMax, kills idle sessions after ~10 minutes, shrinking the window a hijacked idle session is useful.
• AllowUsers, an explicit allowlist; every other account on the box cannot even attempt SSH. Skip this on hosts with many legitimate users.

What this teaches: drop-in configs in sshd_config.d keep hardening separate from vendor defaults.

4. Validate the config and reload safely

sshd -t parses the config and exits non-zero on any error without touching the running daemon, it is the difference between a clean reload and locking yourself out with a typo. Reload (not restart) keeps existing sessions alive while new connections get the new policy. Then re-check the effective config with sshd -T and confirm the daemon is listening.

Expected output:

passwordauthentication no
permitrootlogin no
maxauthtries 3

LISTEN 0 128 0.0.0.0:22 0.0.0.0:* users:(("sshd",pid=1823,fd=3))

Common mistake: running systemctl restart ssh instead of reload, or reloading before opening a second verified session. On Ubuntu 22.04 the unit is named ssh.service (sshd is aliased to it); on RHEL-family systems it is sshd.service. If sshd -t is silent, that means success, it prints nothing on a valid config.

What this teaches: sshd -t catches syntax errors before they kill the daemon on reload.

5. Tighten crypto, disable unused features, and raise log verbosity

With access control settled, the next layer is reducing what sshd will negotiate and what it will do at all. Modern OpenSSH already defaults to strong algorithms, but pinning an explicit list means a future client cannot talk the server down into something weaker, and disabling forwarding/agent features you do not use removes whole feature-attack-surfaces. LogLevel VERBOSE is the quiet win here: it logs the key fingerprint used for every login, which turns the auth log into a real audit trail.

Expected output:

# Crypto + feature surface
KexAlgorithms curve25519-sha256,[email protected]
Ciphers [email protected],[email protected]
MACs [email protected],[email protected]
HostKeyAlgorithms ssh-ed25519
PermitEmptyPasswords no
X11Forwarding no
AllowAgentForwarding no
AllowTcpForwarding no
PermitTunnel no
LogLevel VERBOSE

Notes on the choices:

• KexAlgorithms / Ciphers / MACs, pinned to current, audited primitives; no CBC modes, no SHA-1, no group-exchange legacy KEX.
• HostKeyAlgorithms ssh-ed25519, the server only offers its ed25519 host key. Make sure one exists (ls /etc/ssh/ssh_host_ed25519_key) before setting this, or clients see "no hostkey alg".
• AllowTcpForwarding no, appropriate for a pure admin/bastion host; leave it on if you legitimately use SSH tunnels, or scope it with Match User blocks.
• PermitEmptyPasswords no, redundant once password auth is off, but explicit is better than implied.

Validate and reload exactly as in step 4, then confirm the effective algorithms:

kexalgorithms curve25519-sha256,[email protected]
ciphers [email protected],[email protected]
macs [email protected],[email protected]
loglevel VERBOSE

Scope note: this pinned algorithm list is a lab choice matched to the lab's clients, not a universal recommendation. Pinning too aggressively breaks older but legitimate clients (old network appliances, legacy PuTTY). Always enumerate what your real clients support first (ssh -Q kex / ssh -Q cipher / ssh -Q mac on each client) and only pin the intersection; on a mixed fleet, prefer dropping the truly broken primitives (CBC, SHA-1, diffie-hellman-group14-sha1 and below) rather than pinning a narrow allowlist.

How to verify VERBOSE logging pays off: after a fresh login, the auth log now records which key was used, invaluable when you later audit who logged in from where:

Sep 21 14:02:11 lab-node-01 sshd[2411]: Accepted publickey for taki from 198.51.100.10 port 51234 ssh2: ED25519 SHA256:7gR3kQ...x9

What this teaches: crypto agility means disabling legacy algorithms before attackers use them.

6. Install fail2ban and configure the sshd jail

fail2ban watches the auth log for repeated failures and inserts firewall DROP rules for the offending source. Install it, then configure via jail.local, never edit jail.conf directly, because package updates overwrite it. The three knobs that matter are maxretry (failures before a ban), findtime (the window those failures must occur in), and bantime (how long the ban lasts).

Expected output:

[DEFAULT]
bantime  = 1h
findtime = 10m
maxretry = 5
backend  = systemd

[sshd]
enabled = true
port    = ssh
journalmatch = _SYSTEMD_UNIT=ssh.service + _COMM=sshd

Two details matter here. First, with backend = systemd fail2ban reads the systemd journal directly, so logpath is ignored; do not set both, they describe different log sources. Second, the stock sshd filter on Debian/Ubuntu assumes the sshd.service unit name, but Ubuntu runs sshd as ssh.service, so the journalmatch override pins the correct unit plus the sshd comm field. Without it the jail starts cleanly and matches nothing. (The Debian package already sets backend = systemd in defaults-debian.conf; setting it explicitly in jail.local makes the lab portable across distros where the default is a file backend.)

Then enable it and confirm the jail is running:

Status for the jail: sshd
|- Filter
|  |- Currently failed: 2
|  `- Total failed:     47
`- Actions
   |- Currently banned: 3
   `- Total banned:     11

Common mistake: watching the wrong log source. On Ubuntu 22.04, sshd logs to the systemd journal under the ssh.service unit, and the jail must use backend = systemd with the matching journalmatch above. A jail that shows Total failed: 0 while journalctl -u ssh shows failures is reading the wrong source. Verify the filter against real journal entries before assuming bans should be happening:

Also confirm which firewall backend fail2ban is actually using, because the chain you inspect must match the configured action: sudo fail2ban-client get sshd action. The default on Ubuntu is an iptables-based action; where the backend is nftables, inspect with sudo nft list ruleset | grep -i f2b instead of iptables -L.

What this teaches: fail2ban turns repeated failures into automatic blocks, shrinking the attacker's window.

Validation & Testing

Attempt a password login (must be rejected) and trigger repeated failures from a test IP to confirm a fail2ban ban appears in iptables -L f2b-sshd. Pass criteria: password auth refused, key auth succeeds, and the offending IP is banned within the configured window.

Test 1, password auth is dead. From a client, force a password attempt. With PasswordAuthentication no and keyboard-interactive disabled, the server should offer no password method at all:

user@host: Permission denied (publickey).

Note the server only advertises publickey, there is no password prompt to even type into. That is the pass condition.

Test 2, key auth still works. A normal login must succeed:

taki
lab-node-01

Test 3, fail2ban actually bans. From a second test machine (not your own IP, banning yourself mid-lab is a classic), hammer the server with bad usernames past the retry threshold:

Then on the server, confirm the ban landed in both fail2ban and the firewall:

Chain f2b-sshd (1 references)
target     prot opt source               destination
REJECT     all  --  203.0.113.44         0.0.0.0/0            reject-with icmp-port-unreachable
RETURN     all  --  0.0.0.0/0            0.0.0.0/0

Cleanup after the test so the lab IP is not banned for the next hour:

Expected output:

(no output on success)

How to verify overall: all three tests pass, password path refused, key path working, and a real ban appearing in the f2b-sshd chain within seconds of the 5th failure.

Test 4, root login is refused even with a valid key. Root often has keys of its own; the policy must reject root at the door regardless of credentials:

root@host: Permission denied (publickey).

Test 5, measure the log-noise reduction. This is where the 99.7% figure comes from. Count auth failures reaching PAM over equal windows before and after fail2ban went live:

14832
44

Raw scan traffic on the wire barely changed, bots keep knocking, but almost everything is now rejected at the firewall by an existing ban before sshd ever evaluates credentials. Also spot-check that the fail2ban chain is actually seeing drops:

Chain f2b-sshd (1 references)
 pkts bytes target     prot opt in  out  source        destination
 3182  191K REJECT     all  --  *   *    203.0.113.0/24 0.0.0.0/0

203.0.113.44 91.121.87.10 185.220.101.7

Rollback plan (tested, not theoretical): if any validation fails, restore the backup from step 0 and remove the drop-in:

Expected output:

(no output on success)

Then re-run Test 2 from your still-open session to confirm you are back to the working baseline before investigating what broke.

Troubleshooting

• Locked out after reload: always validate with sshd -t before reload and keep a session open. If the worst happens, use the provider's out-of-band console (VNC/serial), restore /etc/ssh/sshd_config.bak.YYYY-MM-DD from step 0, and reload. As a belt-and-braces measure on remote-only boxes, schedule a safety net before reloading. It must restore the backup, not just restart the daemon with the broken config: BK=$(cat /root/.sshd-last-backup); echo "cp $BK /etc/ssh/sshd_config && systemctl restart ssh" | sudo at now + 10 minutes (the at daemon must be installed and running). The backup path is read from the durable file written in step 0 rather than a shell variable, so the safety net works in a fresh shell or after a disconnect. If your new session works, cancel it with atrm; if you are locked out, the old config comes back on its own.
• fail2ban not banning: confirm the backend and journal match in jail.local. On Ubuntu 22.04 use backend = systemd with journalmatch = _SYSTEMD_UNIT=ssh.service + _COMM=sshd, and test the filter with sudo fail2ban-regex --journalmatch "_SYSTEMD_UNIT=ssh.service + _COMM=sshd" systemd-journal /etc/fail2ban/filter.d/sshd.conf. A jail that shows Total failed: 0 while auth failures exist is watching the wrong log source.
• Key refused with "Permission denied (publickey)": check ~/.ssh is 700 and authorized_keys is 600, that the key is actually listed in the file, and that the home directory is not group-writable (StrictModes rejects it). Debug server-side with a one-off verbose daemon: sudo /usr/sbin/sshd -d -p 2222, then connect to port 2222 and read exactly why the key was rejected.
• "Bad configuration option" from sshd -t: the directive is deprecated or misspelled for your OpenSSH version, e.g. UsePrivilegeSeparation and RSAAuthentication were removed years ago but linger in old tutorials. Check man sshd_config on the actual host, and re-run sshd -T to see what survived.
• Banned myself by accident: from another network or the console, run sudo fail2ban-client set sshd unbanip YOUR_IP. To prevent recurrence, add your static management IPs to the ignoreip line in [DEFAULT] (e.g. ignoreip = 127.0.0.1/8 198.51.100.10), but only for genuinely static addresses, never DHCP ranges.

Key Results

• Reduced brute-force attempts reaching authentication by 99.7% after fail2ban deployment, measured on this lab host across a 7-day before/after window.
• Eliminated 100% of password-based logins via key-only enforcement.
• Banned offending IPs in under 10 seconds of the 5th failed attempt.
• Removed direct root SSH access across 7/7 managed nodes.

Measured by comparing Failed password entries in the auth log across the week before and after: raw connection attempts barely changed (the internet does not stop scanning), but attempts that got far enough to reach PAM collapsed because banned sources were dropped at the firewall. Auth-log volume fell accordingly, which made the remaining entries actually worth reviewing.

Key Takeaways

• Order of operations is everything: deploy the key, prove it works in a fresh session, then disable password auth.
• Key-only auth removes the credential-guessing attack class entirely; fail2ban handles the residual noise and slow-rate probing.
• Prefer drop-in files in /etc/ssh/sshd_config.d/ and jail.local over editing vendor-shipped configs, upgrades stay painless and the policy is self-documenting.
• sshd -t before every reload, and sshd -T after, to confirm what the daemon is really enforcing.
• A hardening change you have not tested from the attacker's side (forced password attempt, deliberate ban trigger) is a hope, not a control.

Practice Challenges