How to set up SSH and Tailscale on a GPU Container Job

Configure SSH and Tailscale inside a GPU Container Job for persistent, secure access over Tailscale.

Configure SSH and Tailscale inside a running GPU Container Job. You can then reach it securely from your local machine over the Tailscale network. Run the setup steps inside the container as the root user.

Prerequisites

You need the following before you start.

A running GPU Container Job. See Create a GPU Container Job.
Root access inside the container, with sudo -s or a direct root shell.
Your local SSH public key, for example ~/.ssh/id_ed25519.pub on your laptop.
Tailscale admin access to approve the node, if your access control lists (ACLs) require it.

Access the GPU Container Job

Open a shell into the running container. See How to access a GPU Container Job.

Install the required packages

Update the system, then install the base tools and the Tailscale client.

sudo apt update -y
sudo apt install -y sudo openssh-server nano
curl -fsSL https://tailscale.com/install.sh | sudo sh

Create the SSH directory.

mkdir -p /root/.ssh
chmod 700 /root/.ssh

Add your local machine's public key to authorized_keys, then lock down the permissions.

nano /root/.ssh/authorized_keys   # paste the contents of ~/.ssh/id_ed25519.pub, then save
chmod 600 /root/.ssh/authorized_keys

Harden SSH

Allow key-only root login and disable password authentication.

sed -i \
  -e 's/^#\?PermitRootLogin.*/PermitRootLogin yes/' \
  -e 's/^#\?PasswordAuthentication.*/PasswordAuthentication no/' \
  -e 's/^#\?PubkeyAuthentication.*/PubkeyAuthentication yes/' \
  -e 's/^#\?UsePAM.*/UsePAM no/' \
  /etc/ssh/sshd_config

Confirm the settings.

grep -E 'PermitRootLogin|PasswordAuthentication|PubkeyAuthentication|UsePAM' /etc/ssh/sshd_config

The output should be the following.

PermitRootLogin yes
PubkeyAuthentication yes
PasswordAuthentication no
UsePAM no

Root login is enabled because these containers are accessed as root over Tailscale with key-only authentication. Do not enable password authentication.

Start Tailscale and SSH

Create the runtime directories, start tailscaled in the background, then start sshd.

mkdir -p /var/run/sshd
mkdir -p /var/run/tailscale

nohup tailscaled \
  --state=/tmp/tailscale.state \
  --socket=/var/run/tailscale/tailscaled.sock \
  > /var/log/tailscaled.log 2>&1 & disown

/usr/sbin/sshd

Confirm both processes are running.

ps ax | grep -E 'tailscaled|sshd'

Join Tailscale

Bring the node up with a unique hostname, for example gpu-container-05, and any tags your Tailscale ACLs require.

tailscale up --hostname=<hostname> --advertise-tags=tag:<your-tag>

For example.

tailscale up --hostname=gpu-container-05 --advertise-tags=tag:gpu-containers

If the node is not authenticated yet, Tailscale prints a login URL. Open it and approve the device.

Verify

From inside the container, check the processes and the Tailscale status.

ps ax | grep -E 'tailscaled|sshd'
tailscale status
tailscale ip -4

From your local machine, once the node appears in the Tailscale admin console, connect over SSH.

ssh root@gpu-container-05

You connect without a password prompt, using key-based authentication only.

Help and troubleshooting

SSH fails with "Permission denied (publickey)"

Confirm the public key is in /root/.ssh/authorized_keys, with permissions 600 on the file and 700 on .ssh.

SSH connects but asks for a password

Re-run the SSH hardening sed command, then restart sshd.

tailscale up fails

Confirm tailscaled is running with ps ax | grep tailscaled, then check /var/log/tailscaled.log.

Node not visible in Tailscale

Run tailscale login, or approve the device in the admin console.

sshd won't start

Confirm /var/run/sshd exists, then check journalctl or /var/log/auth.log.

Restart SSH or Tailscale

Restart SSH after configuration changes.

pkill sshd
/usr/sbin/sshd

Restart Tailscale.

pkill tailscaled
nohup tailscaled \
  --state=/tmp/tailscale.state \
  --socket=/var/run/tailscale/tailscaled.sock \
  > /var/log/tailscaled.log 2>&1 & disown
tailscale up --hostname=<hostname> --advertise-tags=tag:<your-tag>