Why This Matters
If your server allows SSH logins with a password, automated bots are already trying to get in. Run this on any internet-facing server and look at the output:
journalctl -u sshd --since '1 hour ago' | grep 'Failed password'
You'll see hundreds or thousands of brute-force attempts from IPs all over the world, trying common usernames (root, admin, ubuntu, pi) with common passwords.
Even on a LAN-only server, password auth is weaker than key-based auth because:
- Passwords can be guessed, keys cannot (an ED25519 key has 128 bits of entropy)
- Passwords can be phished or shoulder-surfed
- Passwords get reused across systems
- Key-based auth is immune to brute force — there's nothing to guess
After this guide, the only way to SSH into your server is with the private key file. No password prompt, no brute-force vector.
What We're Doing
1. Generate an ED25519 SSH key pair on your local machine
2. Copy the public key to the server
3. Verify you can log in with the key (before changing anything)
4. Disable password authentication in sshd_config
5. Disable root login
6. Disable challenge-response authentication (another password vector)
7. Restart SSH and verify everything works
CRITICAL SAFETY RULE: We test key-based login BEFORE disabling passwords. You will have two SSH sessions open the entire time — one as a safety net. If you lock yourself out, you'll need physical console access or a VM console to fix it.
Step 1: Check for Existing Keys (On Your Local Machine)
Run this on your laptop/desktop — the machine you SSH FROM, not the server:
ls -la ~/.ssh/id_ed25519 ~/.ssh/id_ed25519.pub 2>&1Interpret the Output
If you see two files listed (id_ed25519 and id_ed25519.pub), you already have a key pair. Skip to Step 3.
If you see "No such file or directory" for both files, you need to generate a key pair. Continue to Step 2.
If you see only ONE of the two files, something is wrong. The .pub file is the public key you share, the other is the private key you never share. If you're missing one, generate a new pair in Step 2.
Step 2: Generate an ED25519 Key Pair
Run this on your local machine (not the server). ED25519 is the modern standard — it's faster and more secure than RSA.
The -C flag adds a comment to help you identify this key later. Use something meaningful like your email or username@machine:
ssh-keygen -t ed25519 -C "yourname@your-local-machine"What to Expect
You'll be prompted for:
1. File location — press Enter to accept the default (~/.ssh/id_ed25519)
2. Passphrase — STRONGLY recommended. This encrypts the private key at rest. If someone steals the key file, they can't use it without the passphrase. Choose something strong that you'll remember.
3. Confirm passphrase — type it again.
After generation, you'll see a randomart image and the key fingerprint. Verify the files were created:
ls -la ~/.ssh/id_ed25519 ~/.ssh/id_ed25519.pubStep 3: Copy Your Public Key to the Server
ssh-copy-id reads your public key and appends it to the ~/.ssh/authorized_keys file on the remote server. Replace 'user' with your actual username and the IP with your server's address.
You'll be prompted for your PASSWORD one last time — this is the last time you'll need it:
ssh-copy-id [email protected]If ssh-copy-id Isn't Available
On some systems (older macOS, minimal installs), ssh-copy-id might not exist. Do it manually:
cat ~/.ssh/id_ed25519.pub | ssh [email protected] 'mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys'
This does exactly what ssh-copy-id does: creates the .ssh directory if needed, appends your public key, and sets the correct permissions. The permissions matter — SSH refuses to use authorized_keys if the file or directory permissions are too open.
Step 4: Test Key-Based Login (BEFORE Changing Anything)
Open a NEW terminal window. Do NOT close your existing session. Try to SSH in with key auth:
ssh [email protected]Validate: What You Should See
If key auth is working, one of two things happens:
- You're logged in immediately with no password prompt
- You're prompted for your KEY PASSPHRASE (the one you set during keygen, not the server password)
If you're still getting a PASSWORD prompt, key auth isn't working. Common causes:
1. Wrong permissions on the server. SSH in with your password and fix them:
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
2. The key wasn't copied correctly. Check the file on the server:
cat ~/.ssh/authorized_keys
You should see a line starting with 'ssh-ed25519' followed by a long string.
3. SELinux interference (RHEL/CentOS):
restorecon -Rv ~/.ssh
DO NOT proceed to Step 5 until key-based login works. You'll lock yourself out.
Step 5: Harden sshd_config
Now we modify the SSH server configuration. We're going to:
- Disable root login entirely (PermitRootLogin no)
- Disable password authentication (PasswordAuthentication no)
- Disable challenge-response auth (KbdInteractiveAuthentication no)
- Disable PAM password fallback
Keep your existing SSH session open as a safety net. SSH into the server in a SECOND terminal and make the changes:
First, back up the current config so you can restore it if needed:
sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.bak.$(date +%Y%m%d)Step 5b: Apply the Changes
These sed commands find the relevant lines in sshd_config (whether they're commented out or not) and set them to the secure values.
Note: Modern Ubuntu/Debian uses KbdInteractiveAuthentication instead of the older ChallengeResponseAuthentication. We set both to cover all versions:
sudo sed -i 's/^#\?PermitRootLogin.*/PermitRootLogin no/' /etc/ssh/sshd_config
sudo sed -i 's/^#\?PasswordAuthentication.*/PasswordAuthentication no/' /etc/ssh/sshd_config
sudo sed -i 's/^#\?KbdInteractiveAuthentication.*/KbdInteractiveAuthentication no/' /etc/ssh/sshd_config
sudo sed -i 's/^#\?ChallengeResponseAuthentication.*/ChallengeResponseAuthentication no/' /etc/ssh/sshd_config
sudo sed -i 's/^#\?UsePAM.*/UsePAM no/' /etc/ssh/sshd_configStep 5c: Verify the Config Changes
Before restarting SSH, verify the changes were applied correctly. Every line below should show the setting WITHOUT a # prefix:
grep -E '^(PermitRootLogin|PasswordAuthentication|KbdInteractiveAuthentication|ChallengeResponseAuthentication|UsePAM)' /etc/ssh/sshd_configExpected Output
You should see exactly:
PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
ChallengeResponseAuthentication no
UsePAM no
If any line is missing, the sed command didn't match — the line might not exist in your config at all. Add it manually:
echo 'PasswordAuthentication no' | sudo tee -a /etc/ssh/sshd_config
IMPORTANT: Also check for override files in /etc/ssh/sshd_config.d/ — Ubuntu 22.04+ often has a file there that re-enables PasswordAuthentication:
ls /etc/ssh/sshd_config.d/
cat /etc/ssh/sshd_config.d/*.conf
If you see PasswordAuthentication yes in any of those files, change it to no or delete the file.
Step 5d: Check for Override Files
This is the step most guides miss. Ubuntu 22.04+ ships with /etc/ssh/sshd_config.d/50-cloud-init.conf which often contains 'PasswordAuthentication yes' and OVERRIDES your sshd_config changes. Check and fix it:
ls /etc/ssh/sshd_config.d/ 2>/dev/null && grep -r 'PasswordAuthentication' /etc/ssh/sshd_config.d/ 2>/dev/null || echo 'No override directory or no overrides found'Fix Any Overrides
If you found PasswordAuthentication yes in any file under sshd_config.d/, fix it:
sudo sed -i 's/^PasswordAuthentication yes/PasswordAuthentication no/' /etc/ssh/sshd_config.d/*.conf
Or if you want to be thorough, remove the override file entirely:
sudo rm /etc/ssh/sshd_config.d/50-cloud-init.conf
This is the #1 reason people think they've disabled password auth but haven't. The include directive at the top of sshd_config loads everything in sshd_config.d/ and those settings take precedence.
Step 6: Test the Config Before Restarting
SSH has a config test mode that checks for syntax errors without restarting the service. Always run this before restarting — a syntax error in sshd_config will prevent SSH from starting and you'll be locked out:
sudo sshd -tValidate: What You Should See
If the config is valid, this command produces NO OUTPUT and exits with code 0. That's good — no output means no errors.
If you see any error messages, fix them before proceeding. Common errors:
- "Deprecated option" — you can usually ignore these, they're warnings not errors
- "Bad configuration option" — you have a typo in a directive name
- "/etc/ssh/sshd_config line XX: Missing argument" — a directive has no value
Step 7: Restart SSH
KEEP YOUR EXISTING SSH SESSION OPEN. This is your safety net. If the restart breaks something, you're still logged in and can fix it.
Restart the SSH service:
sudo systemctl restart sshdStep 7b: Verify SSH Is Running
Check that the service restarted successfully:
sudo systemctl status sshd | head -5Expected Output
You should see "active (running)" in green. If you see "failed" or "inactive", restore the backup immediately:
sudo cp /etc/ssh/sshd_config.bak.$(date +%Y%m%d) /etc/ssh/sshd_config
sudo systemctl restart sshd
Your existing session stays alive even if sshd crashes — it only affects NEW connections.
Step 8: Test From a New Terminal
Open a THIRD terminal window (keep both existing sessions open). Test key-based login:
ssh [email protected]Step 8b: Verify Password Auth Is Rejected
Test that password authentication is actually disabled. The -o flag temporarily overrides your local SSH config to force password auth. This should FAIL:
ssh -o PubkeyAuthentication=no [email protected]Expected Output
You should see:
[email protected]: Permission denied (publickey).
This confirms password auth is disabled. The only way in is with a valid SSH key.
If you still get a password prompt, the override files in /etc/ssh/sshd_config.d/ are probably the cause. Go back to the override check step.
If key-based login also stopped working, use your safety-net session to restore the backup:
sudo cp /etc/ssh/sshd_config.bak.$(date +%Y%m%d) /etc/ssh/sshd_config
sudo systemctl restart sshd
Step 9: Add a Firewall Rule (Recommended)
If you're running UFW (Ubuntu's default firewall), restrict SSH to your LAN only. This means even if someone discovers your SSH port, they can't reach it from outside your network:
sudo ufw allow from 192.168.1.0/24 to any port 22 proto tcp comment 'SSH from LAN only'
sudo ufw status | grep 22Validate: What You Should See
You should see a rule like:
22/tcp ALLOW 192.168.1.0/24 # SSH from LAN only
If UFW isn't active, that's fine — this step is optional but recommended. Enable it with: sudo ufw enable (make sure you have an allow rule for SSH FIRST or you'll lock yourself out).
Multiple Users?
If other people SSH into this server, you MUST copy their public keys over BEFORE disabling password auth. For each user:
1. Have them generate a key pair on their machine: ssh-keygen -t ed25519
2. Get their public key (the .pub file contents)
3. Add it to their authorized_keys on the server:
sudo mkdir -p /home/otheruser/.ssh
echo 'their-public-key-string' | sudo tee -a /home/otheruser/.ssh/authorized_keys
sudo chown -R otheruser:otheruser /home/otheruser/.ssh
sudo chmod 700 /home/otheruser/.ssh
sudo chmod 600 /home/otheruser/.ssh/authorized_keys
If you disable password auth before doing this, those users are locked out.
Back Up Your Private Key
Your private key (~/.ssh/id_ed25519) is now the ONLY way into your server. If you lose it — laptop dies, disk corrupts, accidental delete — you're locked out.
Back it up:
- Copy it to a USB drive and store it securely
- Store it in your password manager (Vaultwarden/Bitwarden supports file attachments)
- Print it as a QR code for disaster recovery (extreme but effective)
NEVER email it, never put it in cloud storage unencrypted, never commit it to git.
How to Undo Everything
If you need to re-enable password auth (new user without a key, emergency access, etc.):
1. SSH in with your key
2. Restore the backup:
sudo cp /etc/ssh/sshd_config.bak.$(date +%Y%m%d) /etc/ssh/sshd_config
3. Also check the override directory:
sudo sed -i 's/^PasswordAuthentication no/PasswordAuthentication yes/' /etc/ssh/sshd_config.d/*.conf 2>/dev/null
4. Restart SSH:
sudo systemctl restart sshd
5. Verify password auth works:
ssh -o PubkeyAuthentication=no [email protected]
Or if you CANNOT SSH in at all:
- Use the Proxmox/VMware console to access the VM directly
- Boot into recovery mode
- Mount the filesystem and restore sshd_config from the backup