Secure your servers by blocking off sshd
Wednesday, 30 November 2022
Creating a computer in the cloud is easy, securing it is not. The general approach is to block all inbound access to all ports except the ones that your webserver and
sshd are running on. This approach can get you quite far, but you’re still going to have lots of requests from bad actors trying to find a way in. Tools exist to help you by monitoring auth logs and applying filters, but they’re reactive, and have limited utility as a preventative measure.
Instead, let’s just block off the
sshd port too. But how do we log in if
sshd isn’t visible? Enter Tailscale. Here’s the general idea: install the Tailscale daemon on the machine in question. This lets you access it directly (and securely) from all other machines that you’ve installed Tailscale on.
Quick note: I don’t work for Tailscale and have no real affiliation. Just a happy user.
Start by installing and logging into Tailscale on your main dev machine. You should have a Tailscale icon in your menubar, and it should say you’re Connected.
Once that’s done, let’s do the same thing on the server:
# 1. Install Tailscale # Granular instructions here: https://tailscale.com/kb/1031/install-linux/ $ curl -fsSL https://tailscale.com/install.sh | sh # 2. Start Tailscale $ sudo tailscale up # 3. Look for other machines in your Tailscale network # (You should see your main dev machine here) $ tailscale status 100.100.100.100 formulate.dev tim@ linux - 100.100.100.101 nudges.fyi tim@ linux - 100.100.100.102 m1-mba tim@ macOS active; direct 184.108.40.206:41641, tx 11464 rx 12760 100.100.100.103 tims-iphone tim@ iOS offline # 4. Find the server's Tailscale IP address $ tailscale ip -4 100.100.100.100
Now from your dev machine, try SSH-ing to the server. Make sure Tailscale is running!
# Use the Tailscale IP $ ssh 100.100.100.100 uptime 12:46:44 up 28 days, 15:57, 2 users, load average: 0.15, 0.05, 0.01 # Or hostname (this requires MagicDNS: https://tailscale.com/kb/1081/magicdns) $ ssh formulate.dev 'uname -a' Linux nudges 5.10.0-11-amd64 #1 SMP Debian 5.10.92-1 (2022-01-18) x86_64 GNU/Linux
Now for the fun part!
sshd is accessible via the Tailscale IP, so you can disallow access via the regular public IP without losing access to the server. To start, let’s install a firewall:
$ sudo apt update $ sudo apt install ufw
We can’t enable
ufw yet because it defaults to disallowing all incoming connections, which would lock us out. Let’s first allow all connections via the Tailscale interface:
$ sudo ufw allow in on tailscale0
And then turn on the firewall:
$ sudo ufw enable
At this point you should be able to SSH in with your server’s Tailscale IP address but not its public IP address. Problem solved!
There’s one other thing you’ll have to do to avoid being locked out of your server. Tailscale periodically expires client keys and requires manual reauthentication. If this happens on your server you’ll be locked out, because you can’t get in to reauth.
Make sure you disable key expiry for all servers you’ve set up this way.
Finally, poke holes in your firewall for legitimate incoming traffic. For example, you may want to allow traffic on port 443 solely from Cloudflare IP addresses:
curl https://www.cloudflare.com/ips-v4 > /tmp/ipv4 curl https://www.cloudflare.com/ips-v6 > /tmp/ipv6 for cfip in `cat /tmp/ipv4`; do ufw allow proto tcp from "$cfip" to any port 443; done for cfip in `cat /tmp/ipv6`; do ufw allow proto tcp from "$cfip" to any port 443; done
Finally, your firewall will look something like this:
$ ufw status Status: active To Action From -- ------ ---- Anywhere on tailscale0 ALLOW Anywhere 443/tcp ALLOW 220.127.116.11/20 443/tcp ALLOW 18.104.22.168/22 443/tcp ALLOW 22.214.171.124/22 443/tcp ALLOW 126.96.36.199/22 443/tcp ALLOW 188.8.131.52/18 443/tcp ALLOW 184.108.40.206/18 443/tcp ALLOW 220.127.116.11/20 443/tcp ALLOW 18.104.22.168/20 443/tcp ALLOW 22.214.171.124/22 443/tcp ALLOW 126.96.36.199/17 443/tcp ALLOW 188.8.131.52/15 443/tcp ALLOW 184.108.40.206/13 443/tcp ALLOW 220.127.116.11/14 443/tcp ALLOW 18.104.22.168/13 443/tcp ALLOW 22.214.171.124/22 Anywhere (v6) on tailscale0 ALLOW Anywhere (v6) 443/tcp (v6) ALLOW 2400:cb00::/32 443/tcp (v6) ALLOW 2606:4700::/32 443/tcp (v6) ALLOW 2803:f800::/32 443/tcp (v6) ALLOW 2405:b500::/32 443/tcp (v6) ALLOW 2405:8100::/32 443/tcp (v6) ALLOW 2a06:98c0::/29 443/tcp (v6) ALLOW 2c0f:f248::/32
Finally, let’s make sure this works (I’ve hidden my server’s actual Tailscale and public IP addresses behind the
ssh.formulate.dev hostnames respectively):
This is a great way to quickly and reliably secure personal servers without the hassle of maintaining something like fail2ban. You can even block off all non-Tailscale access if you want to host something for yourself – you can’t really beat that for security!