ssh to fuel slave nodes doesn't work after re-deploy

Hi all.

If there is some known workaround for this problem, please kindly let us know. We have tried to create "/etc/fuel-bootstrap-image.conf" and put key there before re-deploy:
echo >> /etc/fuel-bootstrap-image.conf
echo 'BOOTSTRAP_SSH_KEYS="/root/.ssh/id_rsa.pub"' >> /etc/fuel-bootstrap-image.conf

but it doesn't help in our case.

BR/Anton.

Anton, I see some misunderstanding of settings page in that case.

First of all, "Operating System Access" -> "Authorized SSH keys" has no relation to access to root account. That is all about fueladmin user (or someone else you choose as "Username"). Check the settings above "Authorized SSH keys". And PasswordAuthentication setting to "no" is expected.

Besides it seems to me, that you regenerate /root/.ssh/id_rsa on Fuel Master. Am I right? That procedure is not defined and not allowed. To recover from that issue please open support ticket.

As I was not able to repeat you trouble on 9.2 lab, so I will change state of that bug to Incomplete.

Hi Anton.

Thank you for your inputs!

Yes, you are right - /root/.ssh/id_rsa has been regenerated on Fuel master node before second deploy attempt (it was not mentioned anywhere that it must be the same between deploys). I didn't see any restriction regarding this part and supposed that each new deploy should be independent from previous (i.e. I didn't expect that previous key will be considered after new deploy attempt).

BTW, please let me know where support ticket should be created?

Thank you in advance!
BR/Anton.

BTW, will it work if I will create "fueladmin" user on jump server, generate key for it and paste public key to "Authorized SSH keys" before next re-deploy? Will I have access to slave nodes from jump server then (from "fueladmin" user)? Or there is better alternative?

Just posted question: https://answers.launchpad.net/fuel/+question/556902 hope this is correct action.

There is no needs to create fueladmin on jump host. You can always invoke "ssh -i <path_to_private_key> fueladmin@<node_ip>".

You are right it was never mentioned that it works like that, but that is reality. That keys have been generated on Master node installation and mentioned in /etc/nailgun/settings.yaml. So you have to put right pub key into that file and restart Nailgun. That keys is not for customer use.

Hi Anton.

To be honest, I can't find where in "settings.yaml" old key might be mentioned.

I've attached our "settings.yaml" file.
As far as I understand next 3 parameters might be important:
PATH_TO_SSH_KEY: "/root/.ssh/id_rsa" - currently in this file we have actual key (not those, which was there after Master node installation):

[root@fuel opnfv]# ssh-keygen -y -f /root/.ssh/id_rsa
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC7Mymded2E3JSyV+VTR37hK//pH1UzROIQhx5r+d+WgbnFBPH2wVLk8yxMDZtHAkNsHQuWaIOcP8ra26/tLkMoZnN4nbLHzbKLdzqBYyOepBxqQ4G+T886V2vVXQGZanX65szohipkjj2moajc9H+JB9AckuxCv/cZuQIiJnQlxLp/2YCz++VA+izmwHAsiVkfSoM2+iI64hxhCR3EU1+6AdHWIDxIlhG3Zjm6VMrMKLZhLAcIxJRYKMvM8fuvSRAeZnMZN4wDPMorOdWQOQPr8QnUtqNuEnTHto12NSXsjogP9+9cef3Y/ebBfNDKBarWfkwR9OMTBlATu/R1j8bl
[root@fuel opnfv]# cat /root/.ssh/id_rsa.pub
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC7Mymded2E3JSyV+VTR37hK//pH1UzROIQhx5r+d+WgbnFBPH2wVLk8yxMDZtHAkNsHQuWaIOcP8ra26/tLkMoZnN4nbLHzbKLdzqBYyOepBxqQ4G+T886V2vVXQGZanX65szohipkjj2moajc9H+JB9AckuxCv/cZuQIiJnQlxLp/2YCz++VA+izmwHAsiVkfSoM2+iI64hxhCR3EU1+6AdHWIDxIlhG3Zjm6VMrMKLZhLAcIxJRYKMvM8fuvSRAeZnMZN4wDPMorOdWQOQPr8QnUtqNuEnTHto12NSXsjogP9+9cef3Y/ebBfNDKBarWfkwR9OMTBlATu/R1j8bl root@fuel

PATH_TO_BOOTSTRAP_SSH_KEY: "/root/.ssh/bootstrap.rsa" - this file doesn't exist in our case.

AUTHORIZED_KEYS:
  - "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFUulCbREWAVvoAKrL/zDOzVBK2GwhTQhn9U8148Uwhq3wsS3Ulum0OLz3R+oa1qygzP/TuqCry8h04QHcAFLGhATQXO3zMX6VUEjtQ0GkkcJq2L5aO0Qg8wS8h7HFmPn6dl8BUideVfD6lwQkBxuyVHUrEj/VqBlDIn2zZhDqYEa/vCfeS2tYxs/81JsCsFVHNJ3yVKPruM3dx+2ZXB+6LwSLGB05nBUXrxl0EBJ15jyWboBcRnMhdlCHFb77wF75Ins1QcRUcOa49nMc9el/sKg1SSHLWJkzB/IuHI8XSa+PNIyFzniBRoaBb5Ko7xFn8qNCXKmeDPJcMtN40FX <email address hidden>"
here we have only key for "fuel.domain.tld" domain. I'm not 100% sure, but if I remember correctly, after each re-deploy, key for this domain was new (when we logged to slave nodes manually we saw different key for this domain each time)

Could you please point out which place of this file should be updated? Should we add key for "root@fuel" to section AUTHORIZED_KEYS:
  - "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFUulCbREWAVvoAKrL/zDOzVBK2GwhTQhn9U8148Uwhq3wsS3Ulum0OLz3R+oa1qygzP/TuqCry8h04QHcAFLGhATQXO3zMX6VUEjtQ0GkkcJq2L5aO0Qg8wS8h7HFmPn6dl8BUideVfD6lwQkBxuyVHUrEj/VqBlDIn2zZhDqYEa/vCfeS2tYxs/81JsCsFVHNJ3yVKPruM3dx+2ZXB+6LwSLGB05nBUXrxl0EBJ15jyWboBcRnMhdlCHFb77wF75Ins1QcRUcOa49nMc9el/sKg1SSHLWJkzB/IuHI8XSa+PNIyFzniBRoaBb5Ko7xFn8qNCXKmeDPJcMtN40FX <email address hidden>"
  - "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC7Mymded2E3JSyV+VTR37hK//pH1UzROIQhx5r+d+WgbnFBPH2wVLk8yxMDZtHAkNsHQuWaIOcP8ra26/tLkMoZnN4nbLHzbKLdzqBYyOepBxqQ4G+T886V2vVXQGZanX65szohipkjj2moajc9H+JB9AckuxCv/cZuQIiJnQlxLp/2YCz++VA+izmwHAsiVkfSoM2+iI64hxhCR3EU1+6AdHWIDxIlhG3Zjm6VMrMKLZhLAcIxJRYKMvM8fuvSRAeZnMZN4wDPMorOdWQOQPr8QnUtqNuEnTHto12NSXsjogP9+9cef3Y/ebBfNDKBarWfkwR9OMTBlATu/R1j8bl root@fuel"

Thank you in advance!

You could add new key, but that is useless, because you wiped private key. That means that you have to replace AUTHORIZED_KEYS section of settings.yaml by contents of /root/.ssh/id_rsa.pub (from master node).

Hi Anton.

Ok, thank you! At next re-deploy attempt I will try to replace key in "AUTHORIZED_KEYS" section of settings.yaml file and put there public key from /root/.ssh/id_rsa.pub (from master node). Hope, it will really help.

Thanks again for your help!
BR/Anton.

Do not forget to restart nailgun.

Ok, thanks for reminding! I'll execute "service nailgun restart" after update. BTW, currently key in "settings.yaml" is for "<email address hidden>" and I'm going to replace it with key for "root@fuel". Is "root@fuel" key good enough for normal operation of environment, or there can be some drawback?

It does not matter.