smy-skills/linux-ssh-operator-skill/references/ssh-playbook.md

SSH playbook (Linux server ops)

Defaults and conventions

• Prefer SSH keys (ed25519) and ~/.ssh/config aliases for repeatable runs.
• Avoid putting passwords in files, prompts, or chat logs. If password auth is required, use an interactive terminal/TTY.
• Start with read-only inspection, then apply changes, then verify.

Recommended env vars for wrappers:

• REMOTE_USER: default SSH user
• REMOTE_PORT: default SSH port (usually 22)
• REMOTE_KEY: path to identity file (private key)
• REMOTE_CONNECT_TIMEOUT: connect timeout seconds

SSH key setup (recommended)

Generate a new key:

ssh-keygen -t ed25519 -C "codex" -f ~/.ssh/id_ed25519

Copy the public key to the server:

ssh-copy-id -i ~/.ssh/id_ed25519.pub -p 22 USER@SERVER_IP

Add a host alias:

Host my-server
  HostName SERVER_IP
  Port 22
  User USER
  IdentityFile ~/.ssh/id_ed25519
  IdentitiesOnly yes

Common tasks

Connectivity and OS info

ssh my-server "whoami && hostname && uname -a"
ssh my-server "cat /etc/os-release"

Disk and memory

ssh my-server "df -h"
ssh my-server "free -h"
ssh my-server "du -sh /var/log/* | sort -h | tail"

Processes and ports

ssh my-server "ps aux --sort=-%mem | head"
ssh my-server "ss -lntp"
ssh my-server "lsof -i :8080"

Logs (systemd)

ssh my-server "journalctl -u SERVICE -n 200 --no-pager"
ssh my-server "journalctl -u SERVICE -f --no-pager"
ssh my-server "journalctl -u SERVICE --since '1 hour ago' --no-pager"

Services (systemd)

Status:

ssh my-server "systemctl status SERVICE --no-pager"

Restart (often needs sudo and TTY):

ssh -tt my-server "sudo systemctl restart SERVICE"

Non-interactive sudo (fails if a password prompt would be required):

ssh my-server "sudo -n systemctl restart SERVICE"

Docker

ssh my-server "docker ps"
ssh my-server "docker logs --tail=200 -f CONTAINER"
ssh my-server "docker exec -it CONTAINER bash"
ssh my-server "docker stats --no-stream"

File editing shortcuts

# View a config file
ssh my-server "cat /etc/nginx/nginx.conf"

# Quick in-place line replacement
ssh my-server "sed -i 's/old/new/g' /path/to/file"

# Append a line
ssh my-server "echo 'new line' >> /path/to/file"

SSH Tunneling / Port Forwarding

Local forward

Access a remote service on a local port (server-side port does not need to be publicly open):

# remote MySQL → localhost:13306
ssh -L 13306:127.0.0.1:3306 my-server -N

# remote Redis → localhost:16379
ssh -L 16379:127.0.0.1:6379 my-server -N

Add -f to background the tunnel:

ssh -fNL 13306:127.0.0.1:3306 my-server

Remote forward

Expose a local service on the server (useful for demos or webhooks):

# local :8080 → server :18080
ssh -R 18080:127.0.0.1:8080 my-server -N

Jump host / bastion

# Single jump
ssh -J bastion user@target

# Multi-hop
ssh -J jump1,jump2 user@target

~/.ssh/config version:

Host target-internal
  HostName 10.0.0.50
  User ubuntu
  ProxyJump bastion

SSH Multiplexing

Reuse the same TCP connection for multiple SSH sessions — avoids repeated handshakes:

Host *
  ControlMaster auto
  ControlPath ~/.ssh/ctrl-%h-%p-%r
  ControlPersist 60s

Check active master sockets:

ls ~/.ssh/ctrl-*

Close a master socket manually:

ssh -O exit my-server

Safer host key handling

• Prefer verifying the host key fingerprint out-of-band on first connect.
• If you must automate first-connect for ephemeral hosts, use StrictHostKeyChecking=accept-new (OpenSSH 7.6+).
• If you see a "host key changed" warning, treat it as a potential security incident until you confirm the change is expected.

Remove a stale known_hosts entry:

ssh-keygen -R SERVER_IP

Troubleshooting quick hits

Symptom	Likely cause	Fix
Permission denied (publickey)	Wrong user/key, pub key not on server, sshd config	Check ~/.ssh/authorized_keys on server; verify key matches
Connection timed out	Firewall/security group, wrong port, server down	Check port, security group, ping
No route to host	Network path missing (VPN, ACL, subnet)	Check VPN, routing table
Host key changed warning	Server rebuilt, or MITM	Verify with out-of-band channel; ssh-keygen -R HOST if expected
Too many authentication failures	SSH agent offering too many keys	Use -o IdentitiesOnly=yes -i ~/.ssh/id_ed25519
sudo: a terminal is required	No TTY allocated	Use ssh -tt or pass -t to wrapper

Debug connection verbosely

ssh -vvv my-server

Check sshd config on server

ssh my-server "sudo sshd -T | grep -E 'pubkeyauthentication|passwordauthentication|permitrootlogin'"

Check authorized_keys permissions (common cause of publickey failure)

ssh my-server "ls -la ~/.ssh/ && cat ~/.ssh/authorized_keys"
# .ssh should be 700, authorized_keys should be 600