Fuel for OpenStack

[9.1][timmy] Impossible to create a diagnostic snapshot if nailgun password was changed

Bug #1618965 reported by Vladimir Khlyunev
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Committed
High
Georgy Kibardin
Fuel for OpenStack 10.0
Mitaka
Fix Released
High
Georgy Kibardin
Fuel for OpenStack 9.1

Bug Description

9.1 snapshot 202

Steps:
1 - Deploy & update Fuel master to 9.1
2 - Change nailgun password (UI or shell)
3 - Generate diag snapshot

Result:
Task in error state - Timmy exit code: 4

timmy.log
...
fuel: error: unrecognized arguments: --user admin --password admin
...

This happens even if /root/.config/fuel/fuel-client.yaml and /etc/fuel/astute.yaml contains correct password.

Critical because it dramatically slows down debugging of upgrade jobs

See original description
Vladimir Khlyunev (vkhlyunev)
description: updated
Alexander Gordeev (a-gordeev)
Changed in fuel:
assignee: nobody → Georgy Kibardin (gkibardin)
Roman Prykhodchenko (romcheg)
summary: - [9.1][timmy] Not able to create diag snapshot if nailgun password was
- changed
+ [9.1][timmy] Impossible to create a diagnostic snapshot if nailgun
+ password was changed
Changed in fuel:
status: New → Confirmed
Georgy Kibardin (gkibardin)
Changed in fuel:
status: Confirmed → In Progress
Revision history for this message
Dmitry Sutyagin (dsutyagin) wrote :

To properly fix this bug there are two options:
A. Fix UI password update code so that config files are updated with the new password. This will also improve UX.
B. Pass auth (token or password) to astute and then to timmy when the snapshot button is pressed.

We are currently implementing several workarounds but they cannot overcome this issue fully, generated snapshot will be incomplete without access to Fuel.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-web (master)

Fix proposed to branch: master
Review: https://review.openstack.org/365607

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-astute (master)

Fix proposed to branch: master
Review: https://review.openstack.org/365609

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-web (master)

Reviewed: https://review.openstack.org/365607
Committed: https://git.openstack.org/cgit/openstack/fuel-web/commit/?id=cf26770465a1eb6cd05d68e7d618dd953a96413f
Submitter: Jenkins
Branch: master

commit cf26770465a1eb6cd05d68e7d618dd953a96413f
Author: Georgy Kibardin <email address hidden>
Date: Mon Sep 5 15:16:45 2016 +0300

    Pass auth token to diagnostic snapshot dumper

    Since Timmy is selfsufficient utility it requires auth token to login to
    Fuel to get nodes to dump.
    Also getting rid of snapshot configuration unit tests since Timmy
    doesn't use most of it.

    Change-Id: I559776a701c76bf9f9153550d2989d939d30eb3f
    Partial-Bug: #1618965

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-web (stable/mitaka)

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/366761

Dmitry Pyzhov (dpyzhov)
Changed in fuel:
milestone: 9.1 → 10.0
importance: Critical → High
Revision history for this message
Vladimir Khlyunev (vkhlyunev) wrote :

Strange that bug still in progress - https://product-ci.infra.mirantis.net/view/upgrades/job/9.x.backup-restore.ubuntu.upgrade_ceph_ha_tests/21/ I see diag snapshot here, previous run doesn't contains it (password is changed ofc). Fix released?

Revision history for this message
Georgy Kibardin (gkibardin) wrote :

This fix is still in progress and will unlikely be fixed by itself.

Revision history for this message
Vladimir Khlyunev (vkhlyunev) wrote :

Update:
Snapshot is now generating but it does not contains logs from nodes; timmy.log contains known "fuel: error: unrecognized arguments: --user admin --password admin" string

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-astute (master)

Reviewed: https://review.openstack.org/365609
Committed: https://git.openstack.org/cgit/openstack/fuel-astute/commit/?id=5bd2ba869acdb95a2f8da85b68fb542147009cd0
Submitter: Jenkins
Branch: master

commit 5bd2ba869acdb95a2f8da85b68fb542147009cd0
Author: Georgy Kibardin <email address hidden>
Date: Mon Sep 5 15:21:09 2016 +0300

    Pass auth token to Timmy

    So that it could authenticate in Fuel.

    Change-Id: I67461f63eb80ef982e3d6431a3840a1f6fc92843
    Closes-Bug: #1618965

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-astute (stable/mitaka)

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/369398

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-astute (stable/mitaka)

Reviewed: https://review.openstack.org/369398
Committed: https://git.openstack.org/cgit/openstack/fuel-astute/commit/?id=8b40eab8fc5143ff17498adc77d7ddaffe34e48e
Submitter: Jenkins
Branch: stable/mitaka

commit 8b40eab8fc5143ff17498adc77d7ddaffe34e48e
Author: Georgy Kibardin <email address hidden>
Date: Mon Sep 5 15:21:09 2016 +0300

    Pass auth token to Timmy

    So that it could authenticate in Fuel.

    Change-Id: I67461f63eb80ef982e3d6431a3840a1f6fc92843
    Closes-Bug: #1618965
    (cherry picked from commit 5bd2ba869acdb95a2f8da85b68fb542147009cd0)

Dmitriy Kruglov (dkruglov)
tags: added: on-verification
Revision history for this message
Dmitriy Kruglov (dkruglov) wrote :

Verified on MOS 9.1, snapshot #260. The issue is fixed.

tags: removed: on-verification
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/fuel-astute 10.0.0rc1

This issue was fixed in the openstack/fuel-astute 10.0.0rc1 release candidate.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/fuel-astute 10.0.0

This issue was fixed in the openstack/fuel-astute 10.0.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.