How to Secure SSH Access on Linux Servers

SSH (Secure Shell) is the backbone of remote server administration, but an improperly configured SSH service can become your greatest security vulnerability. Every day, automated bots scan the internet for poorly secured SSH servers, attempting to gain unauthorized access through brute force attacks and credential stuffing.

In this comprehensive guide, you’ll learn practical steps to harden your SSH configuration and protect your Linux servers from common attack vectors. Whether you’re managing a single VPS or an entire infrastructure, these security measures are essential for maintaining a robust defense.

Table of Contents

• Why SSH Security Matters
• Change the Default SSH Port
• Disable Root Login
• Implement Key-Based Authentication
• Configure SSH Idle Timeout
• Limit User Access
• Install and Configure Fail2Ban
• Use Two-Factor Authentication
• Monitor SSH Logs
• Additional Security Measures

Why SSH Security Matters

SSH provides encrypted communication channels for remote server management, but it’s also a primary target for attackers. A compromised SSH service grants attackers complete control over your server, allowing them to steal data, install malware, or use your resources for malicious purposes.

The default SSH configuration prioritizes accessibility over security, which is why hardening your SSH setup should be among your first tasks after deploying a new server. If you’re running cloud instances on platforms like Kamatera, implementing these security measures becomes even more critical due to the public-facing nature of cloud infrastructure.

Change the Default SSH Port

While security through obscurity shouldn’t be your only defense, changing the default SSH port from 22 to a non-standard port significantly reduces automated attack attempts. Most bot scans target port 22 exclusively.

Steps to Change SSH Port

Edit the SSH configuration file:

sudo nano /etc/ssh/sshd_config

Find the line that reads #Port 22 and change it to:

Port 2849

Choose any port number between 1024 and 65535 that isn’t already in use. Restart the SSH service:

sudo systemctl restart sshd

Important: Before closing your current session, test the new configuration by opening a new terminal and connecting with the new port to ensure you don’t lock yourself out.

Disable Root Login

Allowing direct root login via SSH is one of the most dangerous misconfigurations. Attackers know the root username exists on every Linux system, so they only need to crack the password.

Create a Sudo User First

Before disabling root login, ensure you have a regular user with sudo privileges:

sudo adduser adminuser
sudo usermod -aG sudo adminuser

Disable Root Login

Edit the SSH configuration:

sudo nano /etc/ssh/sshd_config

Find and modify this line:

PermitRootLogin no

Restart SSH to apply changes:

sudo systemctl restart sshd

Implement Key-Based Authentication

Password-based authentication is vulnerable to brute force attacks. SSH key pairs provide a cryptographically secure alternative that’s nearly impossible to crack.

Generate SSH Key Pair

On your local machine (not the server), generate a key pair:

ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

Copy Public Key to Server

ssh-copy-id -i ~/.ssh/id_rsa.pub username@server_ip

Disable Password Authentication

After confirming key-based login works, disable password authentication entirely:

sudo nano /etc/ssh/sshd_config

Set these parameters:

PasswordAuthentication no
ChallengeResponseAuthentication no
UsePAM no

Restart SSH:

sudo systemctl restart sshd

Configure SSH Idle Timeout

Idle SSH sessions pose a security risk, especially if someone walks away from an unlocked workstation. Configure automatic disconnection for inactive sessions.

Edit the SSH configuration:

sudo nano /etc/ssh/sshd_config

Add or modify these lines:

ClientAliveInterval 300
ClientAliveCountMax 2

This configuration disconnects idle sessions after 10 minutes (300 seconds × 2).

Limit User Access

Restrict SSH access to specific users or groups rather than allowing all system users to connect remotely.

Edit the SSH configuration:

sudo nano /etc/ssh/sshd_config

Add one of these directives:

AllowUsers adminuser deployuser

Or limit by group:

AllowGroups sshusers

Restart SSH to apply:

sudo systemctl restart sshd

Install and Configure Fail2Ban

Fail2Ban monitors log files for repeated failed login attempts and automatically blocks offending IP addresses, providing excellent protection against brute force attacks.

Install Fail2Ban

sudo apt update
sudo apt install fail2ban -y

Configure SSH Protection

Create a local configuration file:

sudo nano /etc/fail2ban/jail.local

Add this configuration:

[sshd]
enabled = true
port = 2849
filter = sshd
logpath = /var/log/auth.log
maxretry = 3
bantime = 3600

Start and enable Fail2Ban:

sudo systemctl start fail2ban
sudo systemctl enable fail2ban

Use Two-Factor Authentication

Adding two-factor authentication (2FA) creates an additional security layer beyond SSH keys. Google Authenticator provides a free, reliable solution for Linux servers.

Install Google Authenticator

sudo apt install libpam-google-authenticator -y

Configure for Your User

Run the initialization:

google-authenticator

Follow the prompts and scan the QR code with your authenticator app. Edit PAM configuration:

sudo nano /etc/pam.d/sshd

Add this line at the top:

auth required pam_google_authenticator.so

Edit SSH configuration to enable challenge-response:

sudo nano /etc/ssh/sshd_config

ChallengeResponseAuthentication yes

Restart SSH:

sudo systemctl restart sshd

Monitor SSH Logs

Regular log monitoring helps you identify attack patterns and potential security breaches. SSH authentication attempts are logged in /var/log/auth.log on Debian-based systems.

View recent SSH login attempts:

sudo grep 'sshd' /var/log/auth.log | tail -50

Check for failed login attempts:

sudo grep 'Failed password' /var/log/auth.log

Consider implementing centralized logging solutions for easier monitoring across multiple servers.

Additional Security Measures

Use a VPN for Additional Protection

For enhanced security, especially when accessing servers from untrusted networks, consider routing your SSH connections through a VPN service like NordVPN. This adds encryption and masks your actual IP address, providing an additional security layer.

Implement IP Whitelisting

If you access your server from known IP addresses, implement firewall rules to allow SSH connections only from those addresses:

sudo ufw allow from 203.0.113.5 to any port 2849
sudo ufw enable

Keep Your System Updated

Regularly update your system to patch security vulnerabilities:

sudo apt update && sudo apt upgrade -y

Use SSH Protocol 2 Only

Ensure you’re using only SSH protocol 2, as protocol 1 has known vulnerabilities:

Protocol 2

This should already be the default on modern systems, but verification never hurts.

Conclusion

Securing SSH access on your Linux servers is not optional—it’s a fundamental requirement for responsible server administration. By implementing these security measures, you’ve significantly hardened your server against common attack vectors and reduced the risk of unauthorized access.

Remember that security is an ongoing process, not a one-time setup. Regularly review your security configurations, monitor logs for suspicious activity, and stay informed about emerging threats and best practices. The time you invest in properly securing SSH access today will save you from potentially devastating security breaches tomorrow.

Start with the basics like disabling root login and implementing key-based authentication, then progressively add layers like Fail2Ban, two-factor authentication, and IP whitelisting. Each additional measure strengthens your security posture and makes your servers increasingly resilient against attacks.