Cannot get SSH setup to work

[xfzv]

Hi,

First of all, thanks for sharing this tool! I'd like to try it but for some reason, I cannot figure the SSH setup.

The remote machine I'd like to "pacpush" is named archlinux-laptop.

From my main Arch system (archlinux-desktop) and with my regular user, I can run both

% ping archlinux-laptop
% ssh archlinux-laptop

just fine.

Relevant part of ~/.ssh/config from archlinux-desktop:

Host archlinux-laptop
  IdentityFile ~/.ssh/archlinux-desktop
  User xfzv

As mentioned in the README, from archlinux-laptop, I ran:

% sudo cp ~/.ssh/authorized_keys /root/.ssh
% sudo systemctl restart sshd

However, from archlinux-desktop:

% pacpush -n archlinux-laptop
[sudo] password for xfzv:
The authenticity of host 'archlinux-laptop (xxx.xxx.xxx.xxx)' can't be established.
ED25519 key fingerprint is SHA256:<redacted>.
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])? no
archlinux-laptop: ssh check failed. Have you set up root ssh access to archlinux-laptop?

I double checked:

• the public key in /root/.ssh/authorized_keys is correct on archlinux-laptop
• /root/.ssh has 0700 permissions and has root:root ownership
• /root/.ssh/authorized_keys has 0644 permissions and has root:root ownership

Note that

% ssh root@archlinux-laptop

doesn't work either. I'm prompted for root user password so the key authentication is definitely not working.

I have root SSH access with key authentication set up on another machine running Debian and it works just fine.

What am I doing wrong? I read SSH AND KEY CONFIGURATION multiple times but I'm out of ideas.

Thanks in advance!

[bulletmark]

ssh configuration can be tricky so I will add a new section to the README. Following is a draft. Please try these commands and let me know what you find.

TROUBLESHOOTING YOUR SSH CONFIGURATION

If you are having problems with your ssh configuration, you can try what is
described in this section to help diagnose the issue.

Assume we have a client machine from where we want to run the pacpush
command, and a server machine which we want to update from client remotely.

Run following commands on client:

$ ssh server ls -l .ssh
$ rsync server:.ssh

Both the above commands should list the contents of ~/.ssh/ on
server and prove you have your personal user key configured and
working correctly.

Again on client, run the following commands:

$ sudo SSH_AUTH_SOCK=$SSH_AUTH_SOCK ssh -F $HOME/.ssh/config root@server ls -l .ssh
$ sudo SSH_AUTH_SOCK=$SSH_AUTH_SOCK rsync -e "ssh -F $HOME/.ssh/config" root@server:.ssh/

Both the above commands should each list the contents of /root/.ssh/ on
server and prove you have your personal user key configured and
working for root user.

[xfzv]

Thanks for the reply!

I've made some progress, I can now

% ssh root@archlinux-laptop

with key authentication.

This was all my fault, I completely forgot I had added PermitRoot login no in /etc/ssh/sshd_config.d/custom_settings.conf a long time ago since I never needed to log as root via SSH (until now).

The drop-in config was read by:

# /etc/ssh/sshd_config

# Include drop-in configurations
Include /etc/ssh/sshd_config.d/*.conf

However, I still can't use pacpush:

% pacpush -n archlinux-laptop
[sudo] password for xfzv:
no such identity: /root/.ssh/archlinux-desktop: No such file or directory
root@archlinux-laptop's password:

Likewise, the last two commands you suggested produce the same output:

% sudo SSH_AUTH_SOCK=$SSH_AUTH_SOCK ssh -F $HOME/.ssh/config root@archlinux-laptop ls -l .ssh
[sudo] password for xfzv:
no such identity: /root/.ssh/archlinux-desktop: No such file or directory
root@archlinux-laptop's password:

After running:

% sudo cp ~/.ssh/archlinux-desktop /root/.ssh
% sudo systemctl daemon-reload ; sudo systemctl restart sshd

It now works:

% pacpush -n archlinux-laptop
archlinux-laptop: pushing x86_64 package lists ..
archlinux-laptop: pushed /var/lib/pacman/sync/core.db
archlinux-laptop: pushed /var/lib/pacman/sync/core.files
archlinux-laptop: pushed /var/lib/pacman/sync/extra.db
archlinux-laptop: pushed /var/lib/pacman/sync/extra.files
archlinux-laptop: pushed /var/lib/pacman/sync/multilib.db
archlinux-laptop: pushed /var/lib/pacman/sync/multilib.files
archlinux-laptop: getting list of needed package updates ..

[...]

---

I doubt that

% sudo cp ~/.ssh/archlinux-desktop /root/.ssh

is required though, is it?

pacpush/README.md

Lines 114 to 124 in b243738

	Note that the `sudo` invoked by `pacpush` on itself when you run it as
	your normal user passes on `$SSH_AUTH_SOCK` so that the remote root ssh
	session authenticates against your personal ssh key.
	For rsync/ssh configuration convenience, `rsync` and `ssh` commands run
	by root are specified with your personal user's ssh configuration file
	`~/.ssh/config` if it exists (unless you specify an explicit
	`--ssh-config-file` for pacpush usage as described in the next section).
	Note `rsync` and `ssh` commands run on behalf of root user explicitly
	specify target `root@host` so that any `User` specification in your
	personal ssh configuration for any host is ignored.

I also noticed that

% echo $SSH_AUTH_SOCK

didn't return anything, so I ran (on both machines)

% eval $(ssh-agent)
Agent pid 72540

$SSH_AUTH_SOCK is no longer empty but this doesn't seem to make any difference:

% pacpush -n archlinux-laptop
[sudo] password for xfzv:
no such identity: /root/.ssh/archlinux-desktop: No such file or directory
root@archlinux-laptop's password:

[bulletmark]

Pacpush is never going to work without debugging and fixing your ssh configuration which is the point of the new section I want to add.

What environment are you running in? E.g. I use GNOME on Arch Linux and I see that SSH_AUTH_SOCK is set automatically on reboot by GNOME (points to gcr which starts ssh-agent as needed). So I only get prompted once for my ssh key pass-phrase once after reboot. I must admit I guess I assumed that users would be using an environment which does the same which is why I put that SSH_AUTH_SOCK on that sudo command.

[bulletmark]

BTW, since you have IdentityFile ~/.ssh/archlinux-desktop in your ssh config then when running as root the ~ will resolve to roots home dir so you can either cp that same file there (which is ugly) or change the ~ to your actual home dir path.

[bulletmark]

Another thing, while waiting for you to reply, whenever you have ssh issues a tip is to check/monitor your server side sshd log. E.g. run journalctl -f -u sshd in a server side terminal window as you try to make the connection. Also, add a -v to an ssh command to make it verbosely say what it is trying. So in the two sudo commands I quote above, try also adding a -v before the -F.

[xfzv]

BTW, since you have IdentityFile ~/.ssh/archlinux-desktop in your ssh config then when running as root the ~ will resolve to roots home dir so you can either cp that same file there (which is ugly) or change the ~ to your actual home dir path.

Indeed, that was the issue. As expected, it works fine with

- # IdentityFile ~/.ssh/archlinux-desktop
+ IdentityFile /home/xfzv/.ssh/archlinux-desktop

I'll stick with that, it's probably better than copying /home/xfzv/archlinux-desktop key to /root/.ssh.

Regarding the README modifications, I don't have much to suggest, it's already pretty good in its current state in my opinion. Maybe this use case could be covered (i.e. don't use ~ in IdentityFile). Your troubleshooting draft is also good.

My setup is a bit unusual, my main distro on my desktop is Gentoo and I also have Arch installed on a separate SSD that is mounted to /media/Arch so that I can chroot into it from Gentoo. Therefore, ideally I'd like to be able to run pacpush from my Gentoo system with:

sudo chroot /media/Arch zsh -c "su $(whoami) -c 'pacpush archlinux-laptop'"

This would be part of my maintenance/sync routine with my laptop so doing it with a single command and no other manual intervention would be nice.

This works but only if adding a drop-in in /etc/sudoers.d/9999-pacpush:

Defaults env_keep += "SSH_AUTH_SOCK"
xfzv ALL=(ALL) NOPASSWD: /usr/bin/pacpush

I'm not sure about the security implications of this so I might stick with a manual process (first chroot into /media/Arch, then run pacpush archlinux-laptop as my regular user, which will call sudo and so on). I only do it weekly so it's not that big of a deal.

[bulletmark]

I added new section TROUBLESHOOTING YOUR SSH CONFIGURATION to the README.